Make every prediction at once

Our candidate weights are $w=[2,1]^\mathsf T$. Calculating a prediction for each record gives $2$, $5$, and $7$. We can organize the three dot products in one expression:

$$A=\begin{bmatrix}1&0\\2&1\\3&1\end{bmatrix}, \qquad Aw=\begin{bmatrix}1\times2+0\times1\\2\times2+1\times1\\3\times2+1\times1\end{bmatrix} =\begin{bmatrix}2\\5\\7\end{bmatrix}.$$

The matrix $A$ contains the inputs: one record per row and one feature per column. It is often called a design matrix in regression. Its entries are the same measurements as before, arranged for a repeated calculation.

Check the shapes first

$A$ has shape $3\times2$: three rows, two columns. The weight vector has shape $2\times1$. Their product has shape $3\times1$:

$$(3\times2)(2\times1)\longrightarrow(3\times1).$$

The matching inner sizes mean every row has one entry for each weight. The outer sizes tell us there will be three predictions. A shape check can catch an impossible calculation before you do any arithmetic.

Shape is necessary but does not check meaning. A row ordered as “sets, hours” still has two entries, so the multiplication runs. It gives the wrong interpretation if the weights expect “hours, sets.” Keep the labels attached while learning the notation.

Entry $A_{ij}$ means row $i$, column $j$. Here $A_{32}=1$: the third record has one practice set. This differs from $A_{23}$, which is not present because the matrix has only two columns.

Several weight vectors

Suppose we also want predictions from weights $[1,2]^\mathsf T$. Put both candidate weight vectors in columns:

$$W=\begin{bmatrix}2&1\\1&2\end{bmatrix}, \qquad AW=\begin{bmatrix}2&1\\5&4\\7&5\end{bmatrix}.$$

The first result column contains predictions from the first model; the second contains predictions from the second. Each output entry comes from a row of $A$ dotted with a column of $W$. For example, row 3, column 2 is $3\times1+1\times2=5$.

In general, $(m\times d)(d\times k)$ produces an $m\times k$ matrix. This is not entry-by-entry multiplication: that is a different operation. Order also matters. Here $AW$ is defined, but $WA$ is not: its inner sizes would be 2 and 3.

What transpose does

Transposing exchanges rows and columns:

$$A^\mathsf T=\begin{bmatrix}1&2&3\\0&1&1\end{bmatrix}.$$

It has shape $2\times3$. The hours column of $A$ becomes the hours row of $A^\mathsf T$. Transpose rearranges entries; it does not undo a matrix multiplication.

Try it

Using $w=[3,-1]^\mathsf T$, calculate $Aw$. Does agreeing with the first model on record 2 mean agreeing everywhere?

Work through the answer

The predictions are $[3,5,8]^\mathsf T$. Both candidates predict 5 for record 2, but the first predicts $[2,5,7]^\mathsf T$. One matching prediction does not determine the weights.

Georgia Tech’s matrix multiplication chapter develops the same row-by-column rule.