Minimum mean square error for two random variables

[Dustinsfl]

Homework Statement

Determine the minimum mean square error for the joint PMF. You will need to evaluate ##E_{X, Y}[(Y - 14/11\cdot X - 1/11)^2]##.

Homework Equations

The Attempt at a Solution

The answer is ##\frac{3}{22}##, but when I work it out, I get ##\frac{203}{484}##. From my values, I have the same expected values, covariance, and variance of X as the book. Therefore, since my numbers are on point, I can't figure out why my solution is so off.

The ##mse(a, b) = E_{X, Y}[(Y - \hat{Y})^2] = E_{X, Y}[(Y - aX - B)^2]## since ##\hat{Y} = aX + b##. Expanding out ##mse(a, b)## and taking the derivatives of ##\frac{\partial mse(a, b)}{\partial a} = 0## and ##\frac{\partial mse(a, b)}{\partial b} = 0##, we get that
\begin{alignat*}{2}
a_{opt} &= \frac{cov(X, Y)}{var(X)} &&={} \frac{14}{11}\\
b_{opt} &= E[Y] - \frac{cov(X, Y)}{var(X)}E[X] &&={} \frac{1}{11}
\end{alignat*}
Then the ##mse(a, b)## is
\begin{align*}
mse(a, b) &= E[Y^2] - 2aE[XY] + a^2E[X^2] - 2bE[Y] + 2abE[X] + b^2\\
&= E[Y^2] - \frac{28}{11}E[XY] + \frac{196}{121}E[X^2] -
\frac{2}{11}E[Y] + \frac{28}{121}E[X] + \frac{1}{121}
\end{align*}
In order to find ##mse(a, b)##, we need to know the expected value of ##X##, ##Y##, ##E[X^2]##, and ##E[X^2]## as well as ##E[XY]##.
\begin{alignat*}{4}
E[X] &= \sum_ix_ip_X[x_i] &\quad &
E[Y] &&={} \sum_jy_jp_Y[y_j]\\
&= 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{2} & &
&&={} 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{4} + 3\frac{1}{4}\\
&= \frac{5}{4} & &
&&={} \frac{3}{2}\\
E[X^2] &= \sum_ix_i^2p_X[x_i] & &
E[Y^2] &&={} \sum_jy_j^2p_Y[y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{2} & &
&&={} 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} + 3^2\frac{1}{4}\\
&= \frac{9}{4} & &
&&={} \frac{15}{4}\\
E[XY] &= \sum_{i, j}x_iy_jp_{X, Y}[x_i, y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} +
2\cdot 3\frac{1}{4}\\
&= \frac{11}{4}
\end{alignat*}
Now, that we have remaining components, we can obtain ##mse(a, b)##.
\begin{align*}
&= \frac{15}{4} - \frac{28}{11}\frac{11}{4} +
\frac{196}{121}\frac{9}{4} - \frac{2}{11}\frac{3}{2} +
\frac{28}{121}\frac{5}{4} + \frac{1}{121}\\
&= \frac{203}{484}
\end{align*}

 

[Ray Vickson]

Homework Statement

Determine the minimum mean square error for the joint PMF. You will need to evaluate ##E_{X, Y}[(Y - 14/11\cdot X - 1/11)^2]##.

Homework Equations

The Attempt at a Solution

The answer is ##\frac{3}{22}##, but when I work it out, I get ##\frac{203}{242}##. From my values, I have the same expected values, covariance, and variance of X as the book. Therefore, since my numbers are on point, I can't figure out why my number is so off.

The ##mse(a, b) = E_{X, Y}[(Y - \hat{Y})^2] = E_{X, Y}[(Y - aX - B)^2]## since ##\hat{Y} = aX + b##. Expanding out ##mse(a, b)## and taking the derivatives of ##\frac{\partial mse(a, b)}{\partial a} = 0## and ##\frac{\partial mse(a, b)}{\partial b} = 0##, we get that
\begin{alignat*}{2}
a_{opt} &= \frac{cov(X, Y)}{var(X)} &&={} \frac{14}{11}\\
b_{opt} &= E[Y] - \frac{cov(X, Y)}{var(X)}E[X] &&={} \frac{1}{11}
\end{alignat*}
Then the ##mse(a, b)## is
\begin{align*}
mse(a, b) &= E[Y^2] - 2aE[XY] + a^2E[X^2] - 2bE[Y] + 2abE[X] + b^2\\
&= E[Y^2] - \frac{28}{11}E[XY] + \frac{196}{121}E[X^2] -
\frac{2}{11}E[Y] + \frac{28}{121}E[X] + \frac{1}{121}
\end{align*}
In order to find ##mse(a, b)##, we need to know the expected value of ##X##, ##Y##, ##E[X^2]##, and ##E[X^2]## as well as ##E[XY]##.
\begin{alignat*}{4}
E[X] &= \sum_ix_ip_X[x_i] &\quad &
E[Y] &&={} \sum_jy_jp_Y[y_j]\\
&= 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{2} & &
&&={} 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{4} + 3\frac{1}{4}\\
&= \frac{5}{4} & &
&&={} \frac{3}{2}\\
E[X^2] &= \sum_ix_i^2p_X[x_i] & &
E[Y^2] &&={} \sum_jy_j^2p_Y[y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{2} & &
&&={} 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} + 3^2\frac{1}{4}\\
&= \frac{9}{4} & &
&&={} \frac{15}{4}\\
E[XY] &= \sum_{i, j}x_iy_jp_{X, Y}[x_i, y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} +
2\cdot 3\frac{1}{4}\\
&= \frac{11}{4}
\end{alignat*}
Now, that we have remaining components, we can obtain ##mse(a, b)##.Y - (14/11)X -
\begin{align*}
&= \frac{15}{4} - \frac{28}{11}\frac{11}{4} +
\frac{196}{121}\frac{9}{4} - \frac{2}{11}\frac{3}{2} +
\frac{28}{121}\frac{5}{4} + \frac{1}{121}\\
&= \frac{203}{242}
\end{align*}

I do not fully understand the notation ##(Y - 14/11\cdot X - 1/11)^2##. Do you mean
[tex] \left( Y - \frac{14}{11} X - \frac{1}{11} \right)^2 ? [/tex]
If so, use the "\frac" command in TeX. If you mean something else, try using appropriate parentheses. Or, you could/should even use parentheses without the "\frac" command---for example, write ##(Y - (14/11)X - 1/11)^2## if that is what you mean. Alternatively, you could write ##(Y - aX - b)^2,## where ##a = 14/11, \, b = 1/11##. Personally, I like this last way the best, as it helps keep different factors and influences separate.

 

[Dustinsfl]

I do not fully understand the notation ##(Y - 14/11\cdot X - 1/11)^2##. Do you mean
[tex] \left( Y - \frac{14}{11} X - \frac{1}{11} \right)^2 ? [/tex]
If so, use the "\frac" command in TeX. If you mean something else, try using appropriate parentheses. Or, you could/should even use parentheses without the "\frac" command---for example, write ##(Y - (14/11)X - 1/11)^2## if that is what you mean. Alternatively, you could write ##(Y - aX - b)^2,## where ##a = 14/11, \, b = 1/11##. Personally, I like this last way the best, as it helps keep different factors and influences separate.

I have no idea what to say if you don't understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.

 

[Mark44]

I have no idea what to say if you don't understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.

Dustin, I'm pretty sure what you wrote is exactly what you intended to write. The reason for Ray's comment is that we get so many people who write x - 2/y - 3 when they really mean (x - 2)/(y - 3).

 

[Dustinsfl]

Dustin, I'm pretty sure what you wrote is exactly what you intended to write. The reason for Ray's comment is that we get so many people who write x - 2/y - 3 when they really mean (x - 2)/(y - 3).

Even if one is unsure, look past the question into the attempt. If that was done, one would see the proper definition Y - aX - B and realize X is not in the denominator. Critiquing a question in which you didn't fully read is pointless since the reader was premature in their analysis when the clarity was already present and verified later.

 

[haruspex]

I get a different number for E[Y²], but it still doesn't get me to 3/22.

 

[Dustinsfl]

I get a different number for E[Y²], but it still doesn't get me to 3/22.

Did you get 7/2 and then 41/242 for the mean square error? I see my addition error in E[Y^2].

 

[LCKurtz]

I have no idea what to say if you don't understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.

Even if one is unsure, look past the question into the attempt. If that was done, one would see the proper definition Y - aX - B and realize X is not in the denominator. Critiquing a question in which you didn't fully read is pointless since the reader was premature in their analysis when the clarity was already present and verified later.

Sorry, but I agree with Ray here. It isn't our job to look past what is written to see if we can figure out what was meant. When posters neglect needed parentheses we suggest they use them just as we make other suggestions to help posters solve their problems or how to write their solutions up properly. Often their misuse of parentheses is the cause of later algebra mistakes. Instead of telling us you don't need an education in LaTeX perhaps you could say something like "Thanks, I will try to be more careful." If you aren't "trying to be rude", then don't be rude.

 

[Dustinsfl]

Sorry, but I agree with Ray here. It isn't our job to look past what is written to see if we can figure out what was meant. When posters neglect needed parentheses we suggest they use them just as we make other suggestions to help posters solve their problems or how to write their solutions up properly. Often their misuse of parentheses is the cause of later algebra mistakes. Instead of telling us you don't need an education in LaTeX perhaps you could say something like "Thanks, I will try to be more careful." If you aren't "trying to be rude", then don't be rude.

For me, this is the 2nd time Ray hasn't read and made a mistake.

On this post: https://www.physicsforums.com/threads/cov-w-z-where-w-x-and-z-ax-y.775444/ he mentioned and identity to use that I already listed. So he has a recorded with me of not reading the question in full and making a suggestion mistake.

Additionally, I know how to format my questions and wouldn't make the error of saying times when I want to divide that as a quantity. I wasn't being careless and didnt need to be more careful since that format is what I meant and is correct.

 

[haruspex]

I get a different number for E[Y²], but it still doesn't get me to 3/22.

The other error is in your calculation of B.
As you probably realize, it is very easy to get the 3/22 result once you have A and B. Just calculate Y errors.

 

[Dustinsfl]

The other error is in your calculation of B.
As you probably realize, it is very easy to get the 3/22 result once you have A and B. Just calculate Y errors.

Thanks sign error. All is good now.