Two subspaces $V$ and $W$ are orthogonal if for all $v \in V$ and $w \in W$ , the vectors $v$ and $w$ are orthogonal

Lemma 5.1.2 To prove this we only need to prove that the basis of the two subspaces are orthogonal to each other. The basis are denoted as $v_{1} \dots, v_{k}$ and $w_{1}, \dots, w_{l}$ In such case the set ${v_{1}, \dots, v_{k}, w_{1}, \dots, w_{l}}$ are linearly independent.

Proof Consider the linear combination

i = 1 \sum k λ_{i} v_{i} + j = 1 \sum l μ_{j} w_{j} = 0

we want to show that $λ_{i} = 0$ and $μ_{j} = 0$

define

v = i = 1 \sum k λ_{i} v_{i}

then

v = - j = 1 \sum l μ_{j} w_{j} = 0

then $v^{⊤} v = - \sum_{j = 1}^{l} μ_{j} v^{⊤} w_{j} = 0$ Since $v^{⊤} v = ∣∣ v ∣∣$ , this means $v = 0$

Hence, we get

{\sum_{i = 1}^{k} λ_{i} v_{i} = 0 \sum_{j = 1}^{l} μ_{j} w_{j} = 0

Since $v_{1}, \dots v_{k}$ and $w_{1}, \dots, w_{l}$ are linear independent vectors, $λ_{i} = 0$ and $μ_{j} = 0$ for all $i, j$

This means ${v_{1}, \dots, v_{k}, w_{1}, \dots w_{l}}$ form the basis of a new subspace denoted as $V + W$ and we must have $d im (V) + d im (W) = d im (V + W)$

Examples

If $v, w \in R^{n}$ are orthogonal, then $∣∣ v + w ∣ ∣^{2} = ∣∣ v ∣ ∣^{2} + ∣∣ w ∣ ∣^{2}$

Fact 1

$v, w$ orthogonal $V, W$ orthogonal subspaces

Fact 2

${v, w}$ are linear independent $⟹ v, w$ are basis of $R^{2}$ and we can decompose $R^{2}$ as $R^{2} = {λ v ∣ λ \in R} + {μ w ∣ μ \in R}$ (MinKonski sum)

Lemma 5.1.3

Orthogonal Complement

Definition: Orthogonal Complement

$V^{⊥} = {w \in R^{n} ∣ w^{⊤} v = 0 for all v \in V}$

Theorem 5.1.6

Important

$N (A) = C (A^{⊤})^{⊥} = R (A)^{⊥}$

of course

N (A)^{⊥} = C (A^{⊤})

Show $N (A) \subseteq C (A^{⊤})^{⊥}$ Let $x \in N (A)$ (meaning $A x = 0$ ) By definition if $x \in C (A^{⊤})^{⊥}$ then $x^{⊤} b = 0 \forall b \in C (A^{⊤})$ Lets write $\forall b \in C (A^{⊤})$ as $b = A^{⊤} y$ for all $y \in R^{m}$ (recall the definition of column space Column space) $b^{⊤} x = (A^{⊤} y)^{⊤} x = y^{⊤} A x = y^{⊤} 0 = 0$ (recall the rules of transpose Transpose (Transposition))

Show $C (A^{⊤})^{⊥} \subseteq N (A)$ Let $x \in C (A^{⊤})^{⊥}$ By definition $x^{⊤} b = 0 \forall b \in C (A^{⊤})$ again we write $b$ as $b = A^{⊤} y$ for some $y \in R^{m}$ Let’s choose $y = A x$ (which is a valid vector in $R^{m})$ : Then $x^{⊤} b = x^{⊤} (A^{⊤} A x) = (A x)^{⊤} A x = ∣∣ A x ∣ ∣^{2} = 0$ $⟹ A x = 0$ so $x \in N (A)$

Hence the theorem is proved

Example

Find $N (A)^{⊥}$ Just find $C (A^{⊤})$ or $R (A)$

Example 2

Let $V \subseteq R^{n}$ a subspace dimension: $n - 1$ Theres is $α_{1}, \dots, α_{n} \in R$ with $V = {x \in R^{n} ∣ \sum_{i = 1}^{n} α_{i} x_{i} = 0}$ Therefore, $V = α^{⊥}$

If we let $A = [α_{1}, α_{2}, \dots, α_{n}]$ be a $1 \times n$ matrix, then

N (A) = C (A^{⊤})^{⊥} = R (A)^{⊥} = V

$V$ is the null space of the matrix A, and it is a hyperplane

Theorem 5.1.7

Proof: Let $v_{1} \dots v_{k}$ be basis from $V$ , and let $w_{1} \dots w_{l}$ be basis from $W$ $\forall_{i} \in [k], j \in [l] (v_{i}^{⊤} w_{j} = 0)$ $(i) \to (ii)$ Let $A = [v_{1}^{⊤} v_{k}^{⊤}] \in R^{k \times n}$ $C (A^{⊤}) = V$ We know $W = V^{⊥}$ $⟹ V^{⊥} = W = N (A) ⟹ d im (V) + d im (W) = n$

$(ii) \to (iii)$ …

$(iii) \to (i)$ $W \subseteq V^{⊥}$ since $W$ is orthogonal to $V$ show $V^{⊥} \subseteq W$ …

Lemma 5.1.8

NOTE

$R^{n} = V + V^{⊥} = V^{⊥} + (V^{⊥})^{⊥} ⟹ V = (V^{⊥})^{⊥}$

Lemma 5.1.10

Title

$N (A) = N (A^{⊤} A)$ and $C (A^{⊤}) = C (A^{⊤} A)$

Proof: $N (A) \subseteq N (A^{⊤} A)$ $x \in N (A)$ then $A x = 0$ $⟹ A^{⊤} A x = A^{⊤} 0 = 0$ $⟹ x \in N (A^{⊤} A)$

$N (A^{⊤} A) \subseteq N (A)$ $x \in N (A^{⊤} A)$ then $A^{⊤} A x = 0$ $x^{⊤} A^{⊤} A x = (A x)^{⊤} A x = ∥ A x ∥^{2} = 0 ⟹ A x = 0 ⟹ x \in N (A)$

$⟹ N (A) = N (A^{⊤} A)$

Example

$A = [1, 2] \in R^{1 \times 2}$ $N (A) = {x \in R^{2} ∣ x_{1} + 2 x_{2} = 0}$ $A^{⊤} A = [1224]$ $N (A^{⊤} A) = {x \in R^{2} ∣ x_{1} + 2 x_{2} = 0, 2 x_{1} + 4 x_{2} = 0} = {x \in R^{2} ∣ x_{1} + 2 x_{2} = 0} = N (A)$

Thomas Second Brain

Explorer

Orthogonality

Examples

Fact 1

Fact 2

Lemma 5.1.3

Orthogonal Complement

Definition: Orthogonal Complement

Theorem 5.1.6

Example

Example 2

Theorem 5.1.7

Lemma 5.1.8

Lemma 5.1.10

Example

Graph View

Table of Contents

Backlinks