Question

Problem 7.1 (10 points) Let pXnqn"0,1,... be a Markov chain with state space S " t1, 2, 3u and transition probability matrix P " ¨ ˝ 0.5 0.4 0.1 0.3 0.4 0.3 0.2 0.3 0.5 ˛ ‚. (a) Compute the stationary distribution π. (b) Is the stationary distribution π also the limiting distribution? Give a reason for your answer.

Answer 1

Answer:

The responses to the given can be defined as follows:

Step-by-step explanation:

For point a:

fixing probability vector that is $W = [a b c]$

$\therefore$

relation: $WT =W$

$\to 0.5a+0.3b+0.2c=a ..................(1)\\\\ \to 0.4a+0.4b+0.3c=b..............(2)\\\\ \to 0.1a+0.3b+0.5c=c.................(3)$

solving the value:

$a=0.3387 \\\\ b=0.3710\\\\ c=0.2903$

Therefore the stationary distribution $\pi =[0.3387 \ 0.3710\ 0.2903]$

For point b:

$\pi$ will be limiting distribution if $\pi_j=\lin_{n\to \infity} (X_n=\frac{j}{X_0}=i) \Sigma_{\sqrt{j}} n_j=1$

$\pi$ satisfies the above condition so, it is limiting the distribution.