Which outputs can the model reach?

We have made predictions from known weights. Now reverse the question: can any weights reproduce all three observed scores, $y=[2,5,8]^\mathsf T$?

Keep the same input matrix, but leave the weights unknown:

$$\begin{bmatrix}1&0\\2&1\\3&1\end{bmatrix} \begin{bmatrix}w_1\\w_2\end{bmatrix} =\begin{bmatrix}2\\5\\8\end{bmatrix}.$$

The first row requires $w_1=2$. The second requires $2w_1+w_2=5$, so $w_2=1$. But the third then gives $3(2)+1=7$, not 8. No weight vector satisfies all three equations. This is an inconsistent system, meaning it has no exact solution.

That does not make the records invalid. It tells us that this particular model, with two weights and no intercept, cannot fit these three scores exactly.

Read down the columns

Name the columns $h=[1,2,3]^\mathsf T$ and $s=[0,1,1]^\mathsf T$. The matrix product can also be written

$$Aw=w_1h+w_2s.$$

The first weight scales the hours column, the second scales the sets column, and we add the resulting vectors. An expression made by scaling vectors and adding them is a linear combination.

All combinations of these columns form their span, also called the column space of $A$. It contains every prediction vector this model can produce. The question “does $Aw=y$ have a solution?” therefore means “is $y$ in that column space?”

These column vectors have three coordinates because there are three records. Their coordinates are outputs across records, not hours and sets within one record. A two-feature model can therefore produce a vector with three entries, but it cannot necessarily produce every such vector.

For these columns, every prediction satisfies a useful constraint: its third entry equals its first plus its second. Check it directly:

$$3w_1+w_2=w_1+(2w_1+w_2).$$

Our observed scores violate that constraint: $8\ne2+5$. The target $[2,5,7]^\mathsf T$ does satisfy it and has the exact weights $[2,1]^\mathsf T$.

Can different weights give the same outputs?

Here the columns are linearly independent: neither is a multiple of the other. More precisely, $ah+bs=0$ forces $a=0$ from the first entry, then $b=0$ from the second. Their span has dimension 2, so the matrix has rank 2.

Compare a matrix whose columns are $h$ and $2h$. Weights $[2,0]^\mathsf T$ and $[0,1]^\mathsf T$ both give $2h$. Those columns are dependent, and the data cannot distinguish the two contributions. Having two column names does not guarantee two independent directions.

Try it

Can our original matrix produce the target $[1,4,5]^\mathsf T$? Find the weights, then check every row.

Work through the answer

The first equation gives $w_1=1$. The second gives $2+w_2=4$, so $w_2=2$. The third gives $3+2=5$. Thus $w=[1,2]^\mathsf T$ satisfies all three equations.