- Thread starter
- #1

#### chisigma

##### Well-known member

- Feb 13, 2012

- 1,704

Scope of this thread is to give a complete as possible answer to the question proposed two days ago by the user simon11 on Basic Probability and Statistic forum...

My 'almost automatic' answer has been 'yes!... P(X|Y) must necessarly be a normal distribution too...', but other members of MHB expressed critics or doubts about that, so I intend to clarify all the aspects not enough clear of the problem. The first step to perform the task is to remember the definition of

Conditional probability distribution - Wikipedia, the free encyclopedia

... if the r.v. X has p.d.f. $f_{X} (x)$, the r.v. Y has p.d.f. $f_{Y}(y)$, and X and Y have joint density function $f_{X,Y} (x.y)$, then the

$\displaystyle f_{Y} (y|X=x) = f_{Y|X} (x,y) = \frac{f_{X,Y} (x,y)}{f_{X}(x)}$ (1)

$\displaystyle f_{X} (x|Y=y) = f_{X|Y} (x,y) = \frac{f_{X,Y} (x,y)}{f_{Y}(y)}$ (2)

Very well!... now the basic definitions (1) and (2) give the answer to the question of simon11... why?... observing (1) and (2) it is fully evident their

$\displaystyle f_{X}(x)= \frac{1}{\sqrt{2\ \pi}\ \sigma_{X}}\ e^{- \frac{(x-\mu_{X})^{2}}{2\ \sigma^{2}_{X}}}$ (3)

$\displaystyle f_{Y}(y)= \frac{1}{\sqrt{2\ \pi}\ \sigma_{Y}}\ e^{- \frac{(y-\mu_{Y})^{2}}{2\ \sigma^{2}_{Y}}}$ (4)

... but how to say about $f_{X,Y}$?... 'Monster Wolfram' helps us...

Bivariate Normal Distribution -- from Wolfram MathWorld

$\displaystyle f_{X,Y} (x,y)= \frac{1}{2\ \pi\ \sigma_{X}\ \sigma_{Y}\ \sqrt{1-\rho^{2}}}\ e^{- \frac{z}{2\ (1-\rho^{2})}}$ (5)

... where...

$\displaystyle z= \frac{(x-\mu_{X})^{2}}{\sigma^{2}_{X}} - 2\ \frac{\rho\ (x-\mu_{X})\ (y-\mu_{Y})}{\sigma_{X}\ \sigma_{Y}} + \frac{(y-\mu_{Y})^{2}}{\sigma^{2}_{Y}}$ (6)

$\displaystyle \rho= \text{cor}\ (X,Y)= \frac{V_{X,Y}}{\sigma_{X}\ \sigma_{Y}}$ (7)

Usually $\rho$ is called 'correlation' of X and Y and $V_{X,Y}$ is called 'covariance' of X and Y. The (6) and (7) are very interesting and 'suggestive' because the presence of the term $\rho$. In X and Y independent [or more precisely

$\displaystyle f_{Y|X} (x,y)= \frac{1}{\sqrt{2\ \pi}\ \sigma_{Y}\ \sqrt{1-\rho^{2}}}\ e^{- \frac{u}{2\ (1-\rho^{2})}}$ (8)

... where...

$\displaystyle u= \frac{(y-\mu_{Y})^{2}}{\sigma^{2}_{Y}} -2\ \rho\ \frac{(y-\mu_{Y})\ (x-\mu_{X})}{\sigma_{Y}\ \sigma_{X}} + \rho^{2}\ \frac{(x-\mu_{X})^{2}}{\sigma^{2}_{X}}$ (9)

$\displaystyle f_{X|Y} (x,y)= \frac{1}{\sqrt{2\ \pi}\ \sigma_{X}\ \sqrt{1-\rho^{2}}}\ e^{- \frac{v}{2\ (1-\rho^{2})}}$ (10)

... where...

$\displaystyle v= \frac{(x-\mu_{X})^{2}}{\sigma^{2}_{X}} -2\ \rho\ \frac{(x-\mu_{X})\ (y-\mu_{Y})}{\sigma_{X}\ \sigma_{Y}} + \rho^{2}\ \frac{(y-\mu_{Y})^{2}}{\sigma^{2}_{Y}}$ (11)

So 'finally' we are arrived to an explicit expression for $f_{Y|X}$ and $f_{X|Y}$ in the general case where X and Y are not independent. A I said before $f_{Y|X}$ is obtained from $f_{X|Y}$ swapping the role of X and Y... of course!... now by integration one can compute, if desired, $\mu_{Y|X}$, $\mu_{X|Y}$,$\sigma^{2}_{Y|X}$, $\sigma^{2}_{X|Y}$ and other interesting parameters... now I'm a little tired and that will be made, in case, in a successive post...

Kind regards

$\chi$ $\sigma$

*Assume two random variables X and Y are not independent, if P(X), P(Y) and P(Y|X) are all normal, then does P(X|Y) also can only be normal or not necessarily?...*My 'almost automatic' answer has been 'yes!... P(X|Y) must necessarly be a normal distribution too...', but other members of MHB expressed critics or doubts about that, so I intend to clarify all the aspects not enough clear of the problem. The first step to perform the task is to remember the definition of

*conditional distribution function*. According to...Conditional probability distribution - Wikipedia, the free encyclopedia

... if the r.v. X has p.d.f. $f_{X} (x)$, the r.v. Y has p.d.f. $f_{Y}(y)$, and X and Y have joint density function $f_{X,Y} (x.y)$, then the

*conditional probability distribution functions*of X and Y, one conditioned by the other, are...$\displaystyle f_{Y} (y|X=x) = f_{Y|X} (x,y) = \frac{f_{X,Y} (x,y)}{f_{X}(x)}$ (1)

$\displaystyle f_{X} (x|Y=y) = f_{X|Y} (x,y) = \frac{f_{X,Y} (x,y)}{f_{Y}(y)}$ (2)

Very well!... now the basic definitions (1) and (2) give the answer to the question of simon11... why?... observing (1) and (2) it is fully evident their

*intrinsic symmetry*respect to the X and Y, so that is always possible to swap the roles of X and Y and if $f_{X}$,$f_{Y}$ and $f_{Y|X}$ have the same property,*no matter which is the property*, also $f_{X|Y}$ has that property. After some marginal clarifications, simon11 seems to have been satisfied by the answer. One member of the staff of MHB however wasn’t and required a ‘formal proof’. Well!... in order to do that the first step is to find, under the assumption that X and Y are*normal r.v.*, the explicit expressions of $f_{X}$,$f_{Y}$ and $f_{X,Y}$ and use (1) and (2) to obtain $f_{X|Y}$ and $f_{Y|X}$. No matter of course for the first two...$\displaystyle f_{X}(x)= \frac{1}{\sqrt{2\ \pi}\ \sigma_{X}}\ e^{- \frac{(x-\mu_{X})^{2}}{2\ \sigma^{2}_{X}}}$ (3)

$\displaystyle f_{Y}(y)= \frac{1}{\sqrt{2\ \pi}\ \sigma_{Y}}\ e^{- \frac{(y-\mu_{Y})^{2}}{2\ \sigma^{2}_{Y}}}$ (4)

... but how to say about $f_{X,Y}$?... 'Monster Wolfram' helps us...

Bivariate Normal Distribution -- from Wolfram MathWorld

$\displaystyle f_{X,Y} (x,y)= \frac{1}{2\ \pi\ \sigma_{X}\ \sigma_{Y}\ \sqrt{1-\rho^{2}}}\ e^{- \frac{z}{2\ (1-\rho^{2})}}$ (5)

... where...

$\displaystyle z= \frac{(x-\mu_{X})^{2}}{\sigma^{2}_{X}} - 2\ \frac{\rho\ (x-\mu_{X})\ (y-\mu_{Y})}{\sigma_{X}\ \sigma_{Y}} + \frac{(y-\mu_{Y})^{2}}{\sigma^{2}_{Y}}$ (6)

$\displaystyle \rho= \text{cor}\ (X,Y)= \frac{V_{X,Y}}{\sigma_{X}\ \sigma_{Y}}$ (7)

Usually $\rho$ is called 'correlation' of X and Y and $V_{X,Y}$ is called 'covariance' of X and Y. The (6) and (7) are very interesting and 'suggestive' because the presence of the term $\rho$. In X and Y independent [or more precisely

*unrelated*...], then $\rho=0$, if not [and that is the case proposed by simon11...] an 'extra term' must be taken into account. Now we are able, using (1) and (2), to compute $f_{Y|X} (x,y)$ and $f_{X|Y} (x,y)$ with a symple division...$\displaystyle f_{Y|X} (x,y)= \frac{1}{\sqrt{2\ \pi}\ \sigma_{Y}\ \sqrt{1-\rho^{2}}}\ e^{- \frac{u}{2\ (1-\rho^{2})}}$ (8)

... where...

$\displaystyle u= \frac{(y-\mu_{Y})^{2}}{\sigma^{2}_{Y}} -2\ \rho\ \frac{(y-\mu_{Y})\ (x-\mu_{X})}{\sigma_{Y}\ \sigma_{X}} + \rho^{2}\ \frac{(x-\mu_{X})^{2}}{\sigma^{2}_{X}}$ (9)

$\displaystyle f_{X|Y} (x,y)= \frac{1}{\sqrt{2\ \pi}\ \sigma_{X}\ \sqrt{1-\rho^{2}}}\ e^{- \frac{v}{2\ (1-\rho^{2})}}$ (10)

... where...

$\displaystyle v= \frac{(x-\mu_{X})^{2}}{\sigma^{2}_{X}} -2\ \rho\ \frac{(x-\mu_{X})\ (y-\mu_{Y})}{\sigma_{X}\ \sigma_{Y}} + \rho^{2}\ \frac{(y-\mu_{Y})^{2}}{\sigma^{2}_{Y}}$ (11)

So 'finally' we are arrived to an explicit expression for $f_{Y|X}$ and $f_{X|Y}$ in the general case where X and Y are not independent. A I said before $f_{Y|X}$ is obtained from $f_{X|Y}$ swapping the role of X and Y... of course!... now by integration one can compute, if desired, $\mu_{Y|X}$, $\mu_{X|Y}$,$\sigma^{2}_{Y|X}$, $\sigma^{2}_{X|Y}$ and other interesting parameters... now I'm a little tired and that will be made, in case, in a successive post...

Kind regards

$\chi$ $\sigma$

Last edited: