Find the closest allowed prediction

Before fitting both features across all three records, work with a smaller version. Use the first two records and predict from hours alone, with no intercept. Their hours are $a=[1,2]^\mathsf T$ and their observed scores are $y=[2,5]^\mathsf T$.

With one weight $t$, the prediction vector must be $ta=[t,2t]^\mathsf T$. It lies on a line through the origin. The observed vector does not: matching its first entry requires $t=2$, while matching its second requires $t=2.5$.

We need to say what “closest” means. Here it means minimizing the sum of squared prediction errors, which is squared Euclidean distance in the two-output space.

Why the error must be perpendicular

Choose a point $p=ta$ on the allowed line. The remaining error is $r=y-p$. If part of this error points along the line, moving along the line can remove that part. At the closest point, the error has no component along the line:

$$a\cdot(y-ta)=0.$$

Distribute the dot product and solve for $t$:

$$a\cdot y-t(a\cdot a)=0, \qquad t=\frac{a\cdot y}{a\cdot a}=\frac{1(2)+2(5)}{1^2+2^2}=\frac{12}{5}.$$

The closest allowed vector is therefore

$$p=\frac{12}{5}\begin{bmatrix}1\\2\end{bmatrix} =\begin{bmatrix}12/5\\24/5\end{bmatrix}, \qquad r=y-p=\begin{bmatrix}-2/5\\1/5\end{bmatrix}.$$

Check the perpendicularity: $a\cdot r=-2/5+2/5=0$. This closest vector $p$ is the orthogonal projection of $y$ onto the line spanned by $a$.

Check that it is a minimum

Its squared error is $(-2/5)^2+(1/5)^2=1/5$. Weight $t=2$ gives squared error 1; weight $t=2.5$ gives squared error $1/4$. The projection improves on both.

There is also a reason it beats every other point on the line. Any other prediction is $p+ca$ for some scalar $c$. Its error is $r-ca$. Because $r\cdot a=0$,

$$\lVert r-ca\rVert^2=\lVert r\rVert^2+c^2\lVert a\rVert^2.$$

The extra term is nonnegative and is zero only when $c=0$. Moving away from the projection cannot improve squared error.

The coefficient formula requires $a\ne0$, since $a\cdot a$ is the denominator. Projection here also depends on our chosen distance: changing how output errors are weighted changes which prediction is closest.

Keep the three objects separate: $t=12/5$ is the fitted weight, $p$ contains the two predictions, and $r$ contains the two remaining errors. A question asking for the projection wants $p$, not the weight alone.

Try it

Project $z=[3,6]^\mathsf T$ onto the same line. Do you need to change it?

Work through the answer

$t=(3+12)/5=3$, so the projection is $3a=[3,6]^\mathsf T=z$. The target is already allowed, and its residual is zero.