Review : Matrices & Vectors — Question 5

Question 5 – Solution

Strategy. Complete the square to find the best coefficient, then express that coefficient as a linear function of the input.

Step 1: Minimize the distance. Using $a^Ta=5$, $a^Tb=4$ and $b^Tb=5$, $$\|b-ta\|^2=5t^2-8t+5 =5(t-4/5)^2+9/5.$$ The unique minimizer is $t=4/5$, so $$\boxed{Qb=(4/5,8/5)^T,\qquad \text{minimum distance}=3/\sqrt 5.}$$

Step 2: Build the projection matrix. For any input $x$, the minimizing coefficient is $(a^Tx)/(a^Ta)$. Therefore $$Q=\frac{aa^T}{5}=\frac 15\begin{pmatrix}1&2\\2&4\end{pmatrix}, \qquad Q^T=Q,\qquad Q^2=Q.$$ The last equality follows from $aa^Taa^T=a(a^Ta)a^T=5aa^T$.

Step 3: Describe the two perpendicular directions. The condition $Qx=0$ is equivalent to $x_1+2x_2=0$, so the vectors sent to zero form the line through $(2,-1)^T$. The fixed vectors are exactly the multiples of $a$: the image lies on that line, and $Q(ta)=ta$. For the stated input, $b-Qb=(6/5,-3/5)^T$, whose dot product with $a$ is zero.

Step 4: Prove the length identity and equality case. In general $a^T(x-Qx)=0$, so $Qx$ is perpendicular to $x-Qx$. Expanding the squared length of their sum gives $$\boxed{\|x\|^2=\|Qx\|^2+\|x-Qx\|^2,\qquad \|Qx\|\le\|x\|.}$$ Equality holds exactly when $x-Qx=0$, that is, when $x$ lies on the line through $a$, including the zero vector. For $b$, the squared lengths are $5=16/5+9/5$. The plot uses equal axis scales and marks the right angle.