Part 0 · 1 chapters · ~8 min

Vectors and Matrices

Vectors as points and directions, addition and scaling, norms, the dot product, matrices and their shapes, matrix-vector and matrix-matrix multiplication, the identity and transpose, shapes as the first debugging tool, and why GPUs and NumPy make this fast.

The operations in code

code

const dot = (a: number[], b: number[]) => a.reduce((s, x, i) => s + x * b[i], 0);
const norm = (a: number[]) => Math.sqrt(dot(a, a));
const matvec = (M: number[][], x: number[]) => M.map(row => dot(row, x));
const matmul = (A: number[][], B: number[][]) => A.map(r => B[0].map((_, j) => dot(r, B.map(row => row[j]))));

matvec([[2, 0], [0, 0.5]], [1, 1])    // [2, 0.5]: stretch x, squash y

# NumPy: the same, vectorised in C
import numpy as np
W = np.random.randn(1536, 768); x = np.random.randn(768)
y = W @ x                              # shape (1536,): check shapes first when anything breaks

Shapes: an (m × n) matrix times an n-vector gives an m-vector; (m × n) times (n × p) gives (m × p). Most ML bugs are shape bugs, so print shapes before printing values.

VECTORS AND MATRICES IN ONE PAGE

the objects and the operations

swipe the figure sideways, or tap expand for full screen

1/4

vectors

A vector is a list of numbers, read as a point in space or an arrow from the origin. An embedding is a vector with hundreds or thousands of entries.

lists of numberspoints or arrows