# "Two step" Markov chain is also a Markov chain.

#### caffeinemachine

##### Well-known member
MHB Math Scholar
Let $X$ be a compact metric space and $\mathcal X$ be its Borel $\sigma$-algebra. Let $\mathscr P(X)$ be the set of all the Borel probability measures on $X$. A **Markov chain** on $X$ is a measurable map $P:X\to \mathscr P(X)$. We write the image of $x$ under $P$ as $P_x$. (Here $\mathscr P(X)$ is quipped with the Borel $\sigma$-algebra coming from the weak* topology).

Intuitively, for $E\in \mathcal X$, we think of $P_x(E)$ as the probability of landing inside $E$ in the next step given that we are sitting at $x$ at the current instant.

Now let $\mu$ be a probability measure on $X$. We define a new measure $\nu$ on $X$ as follows:

$$\nu(E) = \int_X P_x(E)\ d\mu(x)$$

Intuitively, suppose in the firt step we land in $X$ according to $\mu$. The probability of landing on $x\in X$ according to the measure $\mu$ is $d\mu(x)$.
Now let the Markov chain $P$ drive us. Then the probability of landing in $E$ in the next step given that we are at $x$ is $P_x(E)$.
So the probability of landing in $E$ in two steps is $\int_X P_x(E)\ d\mu(x)$.

So we have a map $\mathscr P(X)\to \mathscr P(X)$ which takes a probability measure $\mu$ and produces a new measure $\nu$ as defined above.

Composing $P:X\to \mathscr P(X)$ with $\mathscr P(X)\to \mathscr P(X)$ we again get a map $X\to \mathscr P(X)$, which I will denote by $P^2$.

Question. Is $P^2$ aslo a Markov chain, that is, is $P^2$ aslo Borel measurable?

My guess is that the map $\mathscr P(X)\to \mathscr P(X)$ is actually continuous, which would answer the question in the affirmative. But I am not sure.

#### caffeinemachine

##### Well-known member
MHB Math Scholar
I am not sure what is wrong. It's appearing on my computer just fine.

Perhaps you are viewing it on another device?

#### Janssens

##### Well-known member
I am not sure what is wrong. It's appearing on my computer just fine.
When I quote-reply your original post, I find that each paragraph is inside a set of color-tags with color code #cbd2ac. When I remove those tags, the text becomes readable.

Staff member