From a single straight line to the transformer. Every idea begins as a story, becomes something you can hold and bend, and ends worked out in full.
From one straight line to a machine that writes and paints. Each stop uses the ones before it — tap any open stop to jump in.
Parts I–V are open now. Part VI is being written, and each of its cards shows what it will build on.
Where vectors live, and the matrices that move them.
A matrix is a machine that moves space. Three windows on Ax = b, the three fates, and elimination — the algorithm that never lies.
The universe where vectors live: groups, subspaces, span, independence, basis, dimension — the architecture of ML math.
The board gets its ruler: norms, inner products, angles, orthogonality — and Gram–Schmidt, the cleanest grid a geometry can have.
A matrix's fingerprints: the volume dial, the directions it cannot turn, A = QΛQᵀ — and Cholesky, the covariance square root.
Prime factorization for transformations: the ladder from spectral to eigen to SVD — ending with a real photograph compressed live.
How a tiny nudge in the input changes the output — the language of learning.
Which way is down? From a shrinking secant to the Jacobian matrices behind backpropagation — the language every learning machine speaks.
The chain rule, industrialized: watch blame flow backwards through a graph, train a neuron with your own hands, and see why a million derivatives cost one sweep.
Polynomial impostors: where Taylor's formula comes from, exactly how big the lie (the remainder) is — and the Hessian, the judge that tells a bowl from a dome from a saddle.
Walking downhill on purpose: the engines that train every model.
Walk downhill in thick fog: the descent rule proved, the step-size speed limit and the zig-zag, exact line search, schedules, gradient checking — and why a random spoonful of the data (stochastic gradient descent) points the right way.
The four ways the walk goes wrong — a bad start, memorising, the wrong valley or a salt flat, a canyon that makes every step bounce — and the cheapest fix in machine learning: change the units of the data, not the model, and watch the canyon become a bowl.
One stride can never suit two directions, so give the walker memory, a private stride per knob, and the ability to forget — momentum, AdaGrad, RMSProp and Adam racing the same valley. Then someone builds a wall, the answer moves onto it while the gradient is still tilted, and you need a new test: the Lagrangian as a fine, complementary slackness, the five conditions, and the price of the wall.
Everything so far, earning its keep on real data.
A cloud of a thousand columns has only a few directions that matter. Find the one where the data spreads the most — the leash that turns it into an eigenvector, the spread budget, the rebuild error, the SVD route, power iteration and the Gram trick for wide data — then run the whole recipe on real numbers.
Draw the widest street between two groups. How wide it is, a price on every point, the dual, the few support vectors that hold the street up, soft margins for messy data — and the kernel trick that lifts the data so a straight cut up there becomes a curve down here.
From geometry to uncertainty, then networks that read — words as points, a running memory, then attention, each unit solving the problem the last one left.
Every unit so far gave crisp answers; the world gives maybes. Distributions and their balance point, Bayes for turning evidence around, the bell curve and why it keeps showing up, the many-dimensional Gaussian that was Unit 12’s ellipse all along, maximum likelihood — under which least squares falls out — then softmax, entropy, cross-entropy and KL.
A real network with every number visible. A layer is a matrix, a shift and a bend; without the bend every stack collapses into one matrix, and with it enough folds trace any shape. The full forward and backward pass worked by hand — at the output the gradient is just prediction − truth — then a playground where you build a network and watch the blame flow back.
Teaching a machine what words mean. Guess the next word by counting, make room for the unseen, and score the guesser by its surprise (perplexity). Know a word by the company it keeps, weighed by PMI and squeezed by the SVD. Then word2vec’s two games, CBOW and skip-gram, made cheap by negative sampling — and king − man + woman lands near queen.
Reading one word at a time. A recurrent cell keeps a running note, and a tiny model learns to talk. Blame flows back through time, multiplied by the same matrix again and again, so its eigenvalues decide whether memory fades or explodes. Then clipping, the LSTM’s express lane, the GRU’s blend dial, reading both ways, and translation graded by BLEU.
Look back instead of remembering. Every word asks every other word a question: queries, keys and values worked by hand, why we divide by √d, and how “bank” finds its meaning. Then an attention lab, many heads, three ways to mark position, the block, the no-peeking mask that trains every guess at once, BERT versus GPT — and a tiny transformer walked end to end.
The two things modern AI does — predict the next word, and turn noise into data — and how almost the whole course lives inside them.
What happens when a model writes. Next-token likelihood at scale, sampling with temperature, top-k and top-p, rotary positions, the KV cache, LoRA fine-tuning, alignment with a reward and a KL leash (and DPO), and scaling laws.
The aha Fine-tuning a giant model is Eckart–Young: change only a rank-8 slice and leave the rest alone.
How a model paints. Sampling and the reparameterisation trick, the evidence lower bound, adding noise step by step, the score as a compass pointing towards real data, and walking back from static to a picture.
The aha Generating an image is Unit 9’s walker going downhill on a learned landscape — with a little noise at every step.
Spotted a mistake, want a topic covered next, or would like to use these units with your class — write to me.