[markov chain] reading expected value from the transition matrix

[rahl___]

Hello there,

yet another trivial problem:

We have a transition matrix of some markov chain: [tex]\left[\begin{array}{ccc}e_{11}&...&e_{1n}\\...&...&...\\e_{n1}&...&e_{nn}\end{array}\right][/tex].
at the beginning our chain is in the state [tex]e_1[/tex]. let T be the moment, when the chain reaches [tex]e_n[/tex] for the first time. What is the expected value of T?

I've attended the 'stochastic process' course some time ago but the only thing I remember is that this kind of problem is really easy to compute, there is some simple pattern for this I presume.

thanks for your help,
rahl.

 

[willem2]

I don't think there's an easy answer to that. You can modify the matrix so, the chain will remain in state [itex] e_n [/itex] if it gets there, and compute [tex] \sum_{k=1}^\infty k (i M^k - i M^{k-1})[/tex]

where i is the initial state of (1, 0, ... , 0) and M the transition matrix. You need the last component of this of course.

 

[Architeuthis Dux]

Hello rahl,

Thank you for your question. The expected value of T in this case can be calculated by summing the products of the transition probabilities and the number of steps it takes to reach the absorbing state (e_n). This can be expressed as a mathematical formula: E[T] = \sum_{n=1}^{n} n * e_{1n}. In other words, we multiply each transition probability by the number of steps it takes to reach the absorbing state and then sum those values together. This is a common method for calculating expected values in Markov chains and can be easily applied to this problem.

I hope this helps. If you have any further questions, please don't hesitate to ask.

Best,