Markov Chains

Every day for lunch you have either a sandwich (state 1), a burrito (state 2), or pizza (state 3). Suppose your lunch choices from one day to the next follow a Markov chain with transition matrix

$$\mathbf{P}=\left[\begin{array}{ccc}0& 0.5& 0.5\\ 0.1& 0.4& 0.5\\ 0.2& 0.3& 0.5\end{array}\right]$$

Suppose today is Monday and consider your upcoming lunches.

• Monday is day 0, Tuesday is day 1, etc.
• You start with pizza on day 0 (Monday).
• Let $T$ be the first time (day) you have a sandwich. (Note: it is possible for $T$ to be greater than 4.)
• Let $V$ be the number of times (days) you have a burrito this five-day work week.
• Pizza costs $5, burrito $7, and sandwich $9.
• Let $X_n$ be the cost of your lunch on day $n$.
• Let $W=X_0+\cdots +X_4$ be your total lunch cost for this five-day work week.

Write code to setup and run a simulation to investigate the following.

1. Approximate the marginal distribution, along with the expected value and standard deviation, of each of the following
   1. $X_4$
   2. $T$
   3. $V$
   4. $W$
2. Approximate the joint distribution, along with the correlation, of each of the following
   1. $X_4$ and $X_5$.
   2. $T$ and $V$
   3. $T$ and $W$
   4. $W$ and $V$
3. Approximate the conditional distribution of $V$ given $T=4$, along with its (conditional) mean and standard deviation.
4. Your choice. Choose at least one other joint, conditional, or marginal distribution to investigate. You can work with $X_n,T,V,W$, but you are also welcome to define other random variables in this context. You can also look at time frames other than a single week.

For each of the approximate distributions, display the results in an appropriate plot, and write a sentence or two describing in words in context some of the main features.