Welcome to our community

Be a part of something great, join today!

Linear MMSE

OhMyMarkov

Member
Mar 5, 2012
83
Hello Everyone!

What $b$ minimizes $E[(X-b)^2]$ where $b$ is some constant, isn't it $b=E[X]$? Is it right to go about the proof as follows:

$E[(X-b)^2] = E[(X^2+b^2-2bX)] = E[X^2] + E[b^2]-2bE[X]$, but $E = b$, we differentiate with respect to $b$ and set to zero, we obtain that $b=E[X]$. Is this proof correct? I was thinking it was until I got this problem:

What $Y$ minimizes $E[(Y-aX-b)^2]$? The given expression contains variances and covariances, but all I get was $Y=aE[X]+b$.

What am I doing wrong here?

Any help is appreciated! :D
 

CaptainBlack

Well-known member
Jan 26, 2012
890
Hello Everyone!

What $b$ minimizes $E[(X-b)^2]$ where $b$ is some constant, isn't it $b=E[X]$? Is it right to go about the proof as follows:

$E[(X-b)^2] = E[(X^2+b^2-2bX)] = E[X^2] + E[b^2]-2bE[X]$, but $E = b$, we differentiate with respect to $b$ and set to zero, we obtain that $b=E[X]$. Is this proof correct?


Correct


I was thinking it was until I got this problem:

What $Y$ minimizes $E[(Y-aX-b)^2]$? The given expression contains variances and covariances, but all I get was $Y=aE[X]+b$.

What am I doing wrong here?

Any help is appreciated! :D
The problem with this second question is that with normal naming conventions \(Y\) is a random variable not a constant, if it were a constant what you get would be correct. If it is a RV then it leaves you with a minimisation problem where the variables are \( \overline{Y}\), \(Var(Y)\), \( Covar(Y,aX+b)\). This is a constrained minimisation problem as \( | Covar(Y,aX+b)| \le \sqrt{Var(Y)Var(aV+b)}\)

CB
 
Last edited:

OhMyMarkov

Member
Mar 5, 2012
83
Thank you CaptainBlack!

But, you mentioned [FONT=MathJax_Math-italic]E[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]Y[/FONT][FONT=MathJax_Main]][/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]V[/FONT][FONT=MathJax_Math-italic]a[/FONT][FONT=MathJax_Math-italic]r[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]Y[/FONT][FONT=MathJax_Main]][/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]C[/FONT][FONT=MathJax_Math-italic]o[/FONT][FONT=MathJax_Math-italic]v[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]Y[/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]a[/FONT][FONT=MathJax_Math-italic]X[/FONT][FONT=MathJax_Main]+[/FONT][FONT=MathJax_Math-italic]b[/FONT][FONT=MathJax_Main]][/FONT] , what about [FONT=MathJax_Math-italic]E[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]X[/FONT][FONT=MathJax_Main]][/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]V[/FONT][FONT=MathJax_Math-italic]a[/FONT][FONT=MathJax_Math-italic]r[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]X[/FONT][FONT=MathJax_Main]][/FONT] , can we sub them for [FONT=MathJax_Math-italic]C[/FONT][FONT=MathJax_Math-italic]o[/FONT][FONT=MathJax_Math-italic]v[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]Y[/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]a[/FONT][FONT=MathJax_Math-italic]X[/FONT][FONT=MathJax_Main]+[/FONT][FONT=MathJax_Math-italic]b[/FONT][FONT=MathJax_Main]][/FONT] ? Or are the variables intentionally used in this fashion so that the hand calculation becomes easier?
 
Last edited:

CaptainBlack

Well-known member
Jan 26, 2012
890
Thank you CaptainBlack!

But, you mentioned [FONT=MathJax_Math-italic]E[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]Y[/FONT][FONT=MathJax_Main]][/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]V[/FONT][FONT=MathJax_Math-italic]a[/FONT][FONT=MathJax_Math-italic]r[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]Y[/FONT][FONT=MathJax_Main]][/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]C[/FONT][FONT=MathJax_Math-italic]o[/FONT][FONT=MathJax_Math-italic]v[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]Y[/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]a[/FONT][FONT=MathJax_Math-italic]X[/FONT][FONT=MathJax_Main]+[/FONT][FONT=MathJax_Math-italic]b[/FONT][FONT=MathJax_Main]][/FONT] , what about [FONT=MathJax_Math-italic]E[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]X[/FONT][FONT=MathJax_Main]][/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]V[/FONT][FONT=MathJax_Math-italic]a[/FONT][FONT=MathJax_Math-italic]r[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]X[/FONT][FONT=MathJax_Main]][/FONT] , can we sub them for [FONT=MathJax_Math-italic]C[/FONT][FONT=MathJax_Math-italic]o[/FONT][FONT=MathJax_Math-italic]v[/FONT][FONT=MathJax_Main][[/FONT][FONT=MathJax_Math-italic]Y[/FONT][FONT=MathJax_Main],[/FONT][FONT=MathJax_Math-italic]a[/FONT][FONT=MathJax_Math-italic]X[/FONT][FONT=MathJax_Main]+[/FONT][FONT=MathJax_Math-italic]b[/FONT][FONT=MathJax_Main]][/FONT] ? Or are the variables intentionally used in this fashion so that the hand calculation becomes easier?
\(E(X)\) and \(Var(X)\) are constants for this problem, while \(u=Covar(Y,aX+b)\) is one of the variable in the optimisation problem.

CB