Discrete Time Markov Chains: Joint, Conditional, and Marginal Distributions

Every day for lunch you have either a sandwich (state 1), a burrito (state 2), or pizza (state 3). Suppose your lunch choices from one day to the next follow a MC with transition matrix

$$\mathbf{P}=\left[\begin{array}{ccc}0& 0.5& 0.5\\ 0.1& 0.4& 0.5\\ 0.2& 0.3& 0.5\end{array}\right]$$

Suppose today is Monday and consider your upcoming lunches.

• Monday is day 0, Tuesday is day 1, etc.
• You start with pizza on day 0 (Monday).
• Let $T$ be the first time (day) you have a sandwich.
• Let $V$ be the number of times (days) you have a burrito this five-day work week.
• Pizza costs $5, burrito $7, and sandwich $9.

1. Compute and interpret in context $\text{P}(T>4)$.
2. Find the marginal distribution of $V$, and interpret in context $\text{P}(V=2)$.
3. Compute the expected total cost of your lunch this work week (Monday through Friday). Interpret this value as a long run average in context.
4. Describe in detail how, in principle, you could use physical objects (coins, dice, spinners, cards, boxes, etc) to perform by hand a simulation to approximate $\text{E}(V|T=4)$. Note: this is NOT asking you to compute $\text{E}(V|T=4)$ or how you would compute it using matrices/equations. Rather, you need to describe in words how you would set up and perform the simulation, and how you would use the simulation results to approximate $\text{E}(V|T=5)$.