Symmetric Matrix

Proposition 9.2.11: PSD

symmetric matrix $A$ is said to be Positive Semidefinite (PSD) if all its eigenvalues are non-negative. If all the eigenvalues of $A$ are strictly positive then we say $A$ is Positive Definite (PD).

Using Rayleigh Quotient PSD $⟺x^{\top }Ax\ge 0\forall x\in {\mathbb{R}}^n$ PD $⟺x^{\top }Ax>0\forall x\in {\mathbb{R}}^n$

Eine diagonal-dominant matrix is always PSD $⟹A\in {\mathbb{R}}^{n\times n}$ symmetric and $A_{ii}>{\sum }_{j\ne i}|A_{ij}|\forall i\in \{1,\dots ,n\}$

Definition: Gram Matrix

Let $A\in {\mathbb{R}}^{m\times n}$, the matrix $G=A^{\top }A\in {\mathbb{R}}^{n\times n}$ is a gram matrix

Equivalent Notations

A matrix $G$ is a gram matrix $⟺$ $G$ ist PSD

$⟹$ $x^{\top }Gx=x^{\top }{A}^{\top }Ax=(Ax{)}^{\top }(Ax)=\Vert Ax{\Vert }^2\ge 0$

$⟸$ $M=V\Lambda V^{\top }$ (Spectral Theorem) Let $B\in {\mathbb{R}}^{n\times n}$ such that $\Lambda =B^2$ with ${\Lambda }_{ii}=B_{ii}{B}_{ii}$ (since all entries of $\Lambda$ are positive) $M=VBB{V}^{\top }=VB(VB{)}^{\top }$ let $C=(VB{)}^{\top }⟹M=C^{\top }C$ ($M$ is gram)

Proposition 9.12.15

NOTE

Given a real matrix $A\in {\mathbb{R}}^{m\times n}$ .The non-zero eigenvalues of $A^{\top }A$ and $A{A}^{\top }$ are the same.

Proof: $A^{\top }Av=\lambda v$ with $(\lambda \ne 0)$ $⟹A{A}^{\top }Av=A\lambda v⟹(A{A}^{\top })(Av)=\lambda (Av)$ Let $w=Av$ $A{A}^{\top }w=\lambda w$ We still need to prove that $w\ne 0$: $N(A^{\top }A)=N(A)\ne \{0\}$ $⟹w=Av\ne 0$ (otherwise $Av=0⟹N(A)=\{0\}$)