The Essentials
A projection of vector a onto vector b is when you take only the part of vector a that is going in the same direction of vector b.
The projection of
\( \vec{u} \) onto
\( \vec{x} \) is given as:
\[
proj_{\vec{x}}(\vec{u})=\frac{\vec{u}\cdot\vec{x}}{\vec{x}\cdot\vec{x}}\vec{x}
\]
When two vectors are orthogonal their dot product is equal to zero.
To make a vector orthogonal to another vector, all that is needed is to subtract the projection of the vector onto the second vector from the first vector.
For example, to make u orthogonal to x, this operation would be performed:
\[
\vec{u_2}=\vec{u}-\frac{\vec{u}\cdot\vec{x}}{\vec{x}\cdot\vec{x}}\vec{x}
\]
A unit vector is a vector that has a magnitude of 1. To make a vector a unit vector, divide the vector by its magnitude:
\[
\vec{u}=\frac{\vec{x}}{||\vec{x}||}
\]
An orthogonal set of vectors, is a set of vectors where each vector is orthogonal to all other vectors in the set. An orthonormal set of vectors is an orthogonal set of unit vectors.
An orthogonal matrix is a matrix whose columns are not an orthogonal set of vectors, but an orthonormal set.
To make a matrix orthogonal (make the column vectors an orthonormal set), the Gram-Schmidt process may be used.
Given a matrix A with column vectors:
\[
A=
\begin{bmatrix}
| & | && | \\
v_1 & v_2 & \ldots & v_n \\
| & | && |
\end{bmatrix}
\]
The first part is to make the column vectors orthogonal to each other. This is done by taking each vector and subtracting from it the projection of it onto all the new vectors:
\[
x_1 = v_1
\]
\[
x_2 =v_2- \frac{\vec{v}_2\cdot\vec{x}_1}{\vec{x}_1\cdot\vec{x}_1}\vec{x}_1
\]
\[
x_3=v_3-\frac{\vec{v}_3\cdot\vec{x}_1}{\vec{x}_1\cdot\vec{x}_1}\vec{x}_1 -\frac{\vec{v}_3\cdot\vec{x}_2}{\vec{x}_2\cdot\vec{x}_2}\vec{x}_2
\]
\[
x_n=v_n-\frac{v_n\cdot x_1}{x_1\cdot x_1}x_1 -\frac{v_n\cdot
x_2}{x_2\cdot
x_2}x_2-\ldots- \frac{v_n\cdot x_{n-1}}{x_{n-1}\cdot x_{n-1}}x_{n-1}
\]
Note that until this point, the vectors should not be scaled. After the vectors are all orthogonal to each other, divide each one by its magnitude to make it a unit vector:
\[
Q=
\begin{bmatrix}
| & | && | \\
\frac{x_1}{||x_1||} & \frac{x_2}{||x_2||} & \ldots & \frac{x_n}{||x_n||} \\
| & | && |
\end{bmatrix}
\]
Example
Use the Gram-Schmidt process to make this matrix an orthogonal matrix:
\[
A =
\begin{bmatrix}
0 & 4 & 3 \\
2 & -1 & 2 \\
0 & -3 & 4
\end{bmatrix}
\]
First, we take the column vectors. The first stays the same:
\[
x_1=v_1=\begin{bmatrix}0\\
2\\
0
\end{bmatrix}
\]
Next, we subtract the projection of the second column vector onto the new first from the second to get the new second vector:
\[
x_2=v_2-proj_{x_1}(v_2)
\]
\[
x_2=v_2-\frac{v_2\cdot x_1}{x_1\cdot x_1}x_1
\]
\[
x_2=
\begin{bmatrix}4\\
-1\\
-3\end{bmatrix}-\frac{\begin{bmatrix}4\\
-1\\
-3\end{bmatrix}\cdot \begin{bmatrix}0\\
2\\
0\end{bmatrix}}{\begin{bmatrix}0\\
2\\
0\end{bmatrix}\cdot \begin{bmatrix}0\\
2\\
0\end{bmatrix}}\begin{bmatrix}0\\
2\\
0\end{bmatrix}
\]
\[
x_2=\begin{bmatrix}4\\
-1\\
-3
\end{bmatrix}-\frac{-2}{4}\begin{bmatrix}0\\
2\\
0\end{bmatrix}
\]
\[
x_2=\begin{bmatrix}4\\
0\\
-3\end{bmatrix}
\]
Then, to make the new third vector, we take the old third vector and subtract the projections of the old third vector onto the new first and second vectors:
\[
x_3=v_3-proj_{x_1}(v_3)-proj_{x_2}(v_3)
\]
\[
x_3=v_3-\frac{v_3\cdot x_1}{x_1\cdot x_1}x_1-\frac{v_3\cdot x_2}{x_2\cdot x_2}x_2
\]
\[
x_3=
\begin{bmatrix}
3\\
2\\
4
\end{bmatrix}-\frac{
\begin{bmatrix}
3\\
2\\
4
\end{bmatrix}\cdot
\begin{bmatrix}
0\\
2\\
0\end{bmatrix}}
{\begin{bmatrix} 0\\
2\\
0\end{bmatrix}\cdot
\begin{bmatrix}
0\\
2\\
0
\end{bmatrix}}
\begin{bmatrix}
0\\
2\\
0
\end{bmatrix}-\frac{
\begin{bmatrix}3\\
2\\
4
\end{bmatrix}\cdot
\begin{bmatrix}4\\
0\\
-3
\end{bmatrix}}{\begin{bmatrix}4\\
0\\
-3\end{bmatrix}\cdot
\begin{bmatrix}
4\\
0\\
-3\end{bmatrix}}\begin{bmatrix}4\\
0\\
-3\end{bmatrix}
\]
\[
x_3=
\begin{bmatrix}
3\\
2\\
4
\end{bmatrix}-\frac{4}{4}
\begin{bmatrix}0\\
2\\
0
\end{bmatrix}-\frac{0}{25}
\begin{bmatrix}
4\\
0\\
-3
\end{bmatrix}
\]
\[
x_3=\begin{bmatrix}
3\\
0\\
4\end{bmatrix}
\]
Lastly, we scale the three vectors to make them unit vectors:
\[
Q =
\begin{bmatrix}
0 & 4/5 & 3/5\\
1 & 0 & 0\\
0 & -3/5 & 4/5
\end{bmatrix}
\]
Practice
Make this set of vectors orthonormal:
\[
\begin{Bmatrix}
\begin{bmatrix}
1\\
0\\
1
\end{bmatrix},
\begin{bmatrix}
0\\
1\\
1
\end{bmatrix},
\begin{bmatrix}
1\\
-1\\
3
\end{bmatrix},
\end{Bmatrix}
\]
Solution:
\[
\begin{Bmatrix}
\frac{1}{\sqrt{2}}
\begin{bmatrix}
1\\
0\\
1\end{bmatrix},
\frac{1}{\sqrt{3}}
\begin{bmatrix}
-1/2\\
1\\
1/2
\end{bmatrix},
\sqrt{\frac{3}{2}}
\begin{bmatrix}
-1\\
-1\\
1
\end{bmatrix},
\end{Bmatrix}
\]
