Positive definite matrices

Fernando Revilla

Well-known member
MHB Math Helper
I quote a question from Yahoo! Answers

1. Prove that if X ∈ R^(d×n) then XX^T and X^TX are both positive semidefinite.
6. Prove that if X ∈ R^(d×n) has rank d, then XX^T is positive definite (invertible).
I have given a link to the topic there so the OP can see my response.

Fernando Revilla

Well-known member
MHB Math Helper
For all $x\in\mathbb{R}^{d\times 1}:$
$$x^T(XX^T)x=(X^Tx)^T(X^Tx)=(y_1,\ldots,y_n) \begin{pmatrix}{y_1}\\{\vdots}\\{y_n}\end{pmatrix}=y_1^2+\ldots+y_n^2\geq 0$$
which implies $XX^T$ is positive semidefinite (or positive definite). Similar arguments for $X^TX$.

If $\text{rank }X=d$, then $\text{rank }(XX^T)=\text{rank }X=d$, which implies $XX^T$ is invertible. This means that $XX^T$ is congruent to a matrix $\text{diag }(\alpha_1,\ldots,\alpha_d)$ with $\alpha_i>0$ for all $i$, as a consequence $XX^T$ is positive definite

MrJava

New member
When I try to solve the case for XTX I get stuck at the following:
xT(XTX)x = xTXTXx = (Xx)TXx

Please kindly guide me next step.

Fernando Revilla

Well-known member
MHB Math Helper
When I try to solve the case for XTX I get stuck at the following: xT(XTX)x = xTXTXx = (Xx)TXx
Right. Now, the difference is that $x\in \mathbb{R}^{n\times 1}$ instead of $\mathbb{R}^{d\times 1}.$ So, for all $x\in \mathbb{R}^{n\times 1}$
$$(Xx)^T(Xx)=(w_1,\ldots,w_d) \begin{pmatrix}{w_1}\\{\vdots}\\{w_d}\end{pmatrix} =w_1^2+\ldots+w_d^2\geq 0$$

MrJava

New member
Right. Now, the difference is that $x\in \mathbb{R}^{n\times 1}$ instead of $\mathbb{R}^{d\times 1}.$ So, for all $x\in \mathbb{R}^{n\times 1}$
$$(Xx)^T(Xx)=(w_1,\ldots,w_d) \begin{pmatrix}{w_1}\\{\vdots}\\{w_d}\end{pmatrix} =w_1^2+\ldots+w_d^2\geq 0$$
Ok I get the point, thank you.