Unit 1 · Systems of Linear Equations — The Math Behind the Machine

1

Why are we even talking about lines?

Imagine this

Stand in an open field. The Earth is a giant ball, yet the ground under your feet looks perfectly flat. You could lay a ruler on it and never notice the curve.

Most curvy things in life behave like this. Zoom out and they bend. Zoom in close enough and they look like straight lines.

Let's test that on a real curve, $y=\sin x$ , standing at $x=1$ . Pretend the curve is a straight line (its tangent, the line that just touches it there) and predict how much it rises:

0.1Take a step of 0.1. The curve really rises 0.0497. The straight line guesses 0.0540. Error: about 0.004.
0.01Take a step 10 times smaller. The curve rises 0.005361. The line guesses 0.005403. Error: about 0.00004, which is 100 times smaller.

A 10 times smaller step made the error 100 times smaller. That is why "pretend it's a line" works so well. See it for yourself:

Try: press ▶ and watch the curve melt into its tangent line. Then read the last row of the table: it settles on one number.

Zoom1×

window width W	5.00
biggest gap curve ↔ tangent	—
gap ÷ W²	—

Zoom in and watch the gap die.

Make the window 10 times narrower and the gap gets 100 times smaller. So gap ÷ W² stops changing: it settles near 0.105, which is sin(1)/8, a number set by how much the curve bends at x = 1.

The realization

Up close, every smooth curve is a straight line. The error of pretending shrinks like the square of how far you look. So we can study the easy, straight version and trust the answer nearby.

This one trick runs all of machine learning. Training a neural network means: zoom in, solve an easy straight-line problem, take a small step, and repeat, millions of times. You will meet this loop as gradient descent in Unit 9.

And straight lines have two more gifts. A computer can always solve a straight-line problem, or prove that it has no answer. And you can draw them: lines, flat planes, flat sheets. Curvy models are built by stacking many straight pieces with a little bend (like a ReLU) between them, the way plain rectangular LEGO bricks can build a castle.

Here is the road through this unit:

①Lists and equations: a vector is a list of numbers; a system is several straight-line rules at once.
②The three fates: a system has no answer, one answer, or infinitely many. Never two.
③The matrix as a machine: it moves all of space; the determinant says how much area it keeps.
④Undo and blind spots: the inverse, and the null space, the directions a matrix cannot see.
⑤Elimination and rank: one method that solves any system, and one number that predicts its fate.

In one sentence: almost everything looks straight when you zoom in, so linear equations are the tool machine learning reaches for first.

2

What is a vector, really?

Imagine this

At the kirana shop, the owner writes your order on a slip: 2 kg rice, 1 kg dal, 3 packets of biscuits. Your neighbour's slip says 1, 0 and 2.

Put both orders in one bag: 3 rice, 1 dal, 5 biscuits. Want the same order again next week? Double it: 4, 2, 6. The shop owner just did vector maths without knowing it.

Write each slip as an ordered list: yours is $(2,1,3)$ , your neighbour's is $(1,0,2)$ . The two moves from the story are the only two moves a vector needs:

\underbrace{(2,1,3)+(1,0,2)=(3,1,5)}_{\text{add: slot by slot}}\qquad\underbrace{2\cdot(2,1,3)=(4,2,6)}_{\text{scale: every slot}}

That is all a vector is: an ordered list of numbers that you can add and scale, and the answer is again a list of the same kind. (That "you get the same kind back" promise is called closure. Unit 2 builds the idea of a vector space on it.)

Once you see it, lists are everywhere. A house in a dataset is (area, bedrooms, age, price). A photo is one number per pixel. A word inside a language model is a list of 768 numbers, called an embedding. Even you are a list of numbers inside your bank's risk model.

Four faces of one vector (swipe sideways on a phone).

The realization

A vector is not a mysterious arrow. It is a list of numbers you are allowed to add and scale. Houses, photos, words and shopping slips all follow the same two rules, so one set of tools works on all of them. That is why linear algebra powers machine learning.

Pause & predict

Which of these can be treated as a vector?

In one sentence: a vector is an ordered list of numbers that you can add slot by slot and scale, and almost any data can be written that way.

3

A small problem you can solve by hand

Imagine this

You run a small workshop that makes chairs, tables and cabinets. Today you have exactly 9 units of wood, 8 hours of labour, 7 hours of machine time and 11 units of shipping.

How many of each should you make so that every resource is used up exactly, with nothing left over and nothing short?

Say you make $x_1$ chairs, $x_2$ tables and $x_3$ cabinets. A chair uses 1 unit of wood, a table 2, a cabinet 1. So the wood rule is $x_1+2x_2+x_3=9$ . Each resource gives one rule like that:

\begin{aligned} x_1+2x_2+x_3 &= 9 &&\text{(wood)}\\ 2x_1+x_2+x_3 &= 8 &&\text{(labour)}\\ x_1+x_2+2x_3 &= 7 &&\text{(machine time)}\\ 3x_1+x_2+2x_3 &= 11 &&\text{(shipping)} \end{aligned}

Four rules, three unknowns. Before any theory, just play. Find the plan with your hands:

Try: press + on chairs once and watch which bars move. Then hunt for the plan where all four bars read ✓.

Chairs x₁0

Tables x₂0

Cabinets x₃0

4 targets to hit.

Each bar is one resource being used by your plan; the bright tick is the target. Notice: one button moves all four bars. The rules are tied together, and that is what makes a system harder than four separate sums.

On paper you would not guess. You would play a small game instead: scale one rule and subtract it from another, so an unknown disappears. Take the labour rule minus 2 times the wood rule:

\begin{aligned}(2x_1+x_2+x_3)-2(x_1+2x_2+x_3)&=8-2\cdot 9\\ -3x_2-x_3&=-10\end{aligned}

The $x_1$ is gone. That is what eliminate means. Keep going until one rule has a single unknown left, then substitute back up. The answer is 2 chairs, 3 tables, 1 cabinet. (Check the new rule: $-9-1=-10$ . ✓)

The realization

Everything fancy in this unit (matrices, Gaussian elimination, echelon form) is just these two moves, done by a machine at scale: remove an unknown by subtracting rules, then substitute back.

One more thing: four rules for three unknowns usually have no answer at all. This one does because the rules happen to agree. When real data disagrees, which is the everyday case in machine learning, linear regression finds the best almost-answer instead.

In one sentence: a system of linear equations is a set of rules that must all be true at once, and you solve it by subtracting rules to remove unknowns one at a time.

4

The three fates: zero, one, or infinity

Imagine this

Treasure is buried on a flat island. Two friends each give you one clue, and each clue is a straight line: "the treasure is somewhere on this line."

If the lines cross, you dig at the crossing. If they run side by side, the clues disagree and there is no treasure. If both friends describe the same line, you have to dig along all of it.

Here are the three cases in numbers. The first clue is always $x+y=4$ .

1Second clue $x-y=0$ : add the two and $2x=4$ , so $x=2,\ y=2$ . One answer.
0Second clue $x+y=1$ : now $x+y$ must be 4 and 1. Impossible. No answer.
∞Second clue $2x+2y=8$ : that is the first clue doubled. Every point on the line works. Infinitely many answers.

In the stage below, each clue stands up as a glowing wall of light. Where two walls cross, a beacon marks the treasure.

The realization

A linear system never has exactly two answers. Suppose $\mathbf{u}$ and $\mathbf{v}$ are both answers. Then the point halfway between them is an answer too, and so is every point on the line through them. Two answers always grow into infinitely many. So the only possible counts are zero, one, or infinity.

The same three fates hold with more unknowns. With three unknowns, each rule is a flat plane in 3D. Take ① $x_1+x_2+x_3=3$ and ② $x_1-x_2+2x_3=2$ . Add them: $2x_1+3x_3=5$ . Now a third rule decides the fate. If it says $2x_1+3x_3=1$ , it fights the other two: no answer. If it says $2x_1+3x_3=5$ , it adds nothing new: infinitely many. If it says something new, like $x_2+x_3=2$ , the three planes meet at one point, $(1,1,1)$ . In Section 13 you will fly around those planes in 3D.

Pause & predict

A classmate says she found a linear system with exactly two solutions. What do you tell her?

In one sentence: each equation is a straight line (or flat plane), and they can meet nowhere, at one point, or along a whole line, so a system has zero, one or infinitely many solutions.

5

Meet the matrix — a spreadsheet with superpowers

Imagine this

Diwali shopping. Asha buys 2 laddoos, 1 barfi and 3 jalebis. Ravi buys 0, 4 and 1. At the sweet shop a laddoo costs ₹10, a barfi ₹20 and a jalebi ₹5.

Asha's bill: take her basket and the price list, multiply item by item, then add: 20 + 20 + 15 = ₹55. You have just done one "row times column".

A matrix is a grid of numbers: $m$ rows and $n$ columns (we say it is $m\times n$ ). Put the two baskets in rows, and put the prices of two shops in columns:

\underbrace{\begin{pmatrix}2&1&3\\0&4&1\end{pmatrix}}_{\text{baskets (Asha, Ravi)}}\underbrace{\begin{pmatrix}10&12\\20&15\\5&6\end{pmatrix}}_{\text{prices (shop 1, shop 2)}}=\underbrace{\begin{pmatrix}55&57\\85&66\end{pmatrix}}_{\text{every bill}}

Each entry of the answer is one basket times one price list: row of the first, times column of the second. Ravi at shop 2: $0\cdot12+4\cdot15+1\cdot6=66$ . In symbols, $c_{ij}=\sum_{l} a_{il}\,b_{lj}$ . This only works when a row of the first is as long as a column of the second. In short, $(m\times k)(k\times n)\to m\times n$ : the inner sizes must match.

Now strip the furniture problem of its letters and plus signs. Keep only the numbers:

\underbrace{\begin{pmatrix}1&2&1\\2&1&1\\1&1&2\\3&1&2\end{pmatrix}}_{A\ \text{(the recipe)}}\ \underbrace{\begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix}}_{\mathbf{x}\ \text{(unknowns)}} = \underbrace{\begin{pmatrix}9\\8\\7\\11\end{pmatrix}}_{\mathbf{b}\ \text{(targets)}}

The whole system is now three letters: $A\mathbf{x}=\mathbf{b}$ . Adding two matrices of the same shape and scaling a matrix work cell by cell, just like vectors.

Try: tap a cell of C, then press ▶ to light up the row and column behind every cell. Switch to BA to see a different answer, and to AA to see the shapes refuse.

—

Inner sizes must agree: $(m\times k)(k\times n)\to m\times n$ . If they don't, the product does not exist. Even when both AB and BA exist, they are different matrices (here one is 2×2, the other 3×3).

Swap the boxes (darken first, then blur) and the photo comes out different. That is why AB ≠ BA.

The realization

A matrix is a function, and multiplying matrices means doing one function after another. "Blur, then darken" is one combined filter, and that filter is $AB$ (read right to left: $B$ acts first). The strange row-times-column rule is exactly the arithmetic that makes $(AB)\mathbf{x}=A(B\mathbf{x})$ true.

That is also why order matters. Socks then shoes is not shoes then socks. In general $AB\neq BA$ .

A few rules you will use forever: $(AB)C=A(BC)$ , $A(B+C)=AB+AC$ , and $IA=AI=A$ . The transpose $A^T$ turns rows into columns, and it flips the order of a product: $(AB)^T=B^TA^T$ . Some matrices to greet by name: the identity $I$ (the do-nothing filter), the zero matrix (destroys everything), diagonal matrices (stretch each axis on its own), and symmetric matrices, $A=A^T$ , which Unit 4 leans on heavily.

If you want the algebra · why (AB)ᵀ = BᵀAᵀ

Why (AB)ᵀ = BᵀAᵀ — derived

Claim. For $A\in\mathbb{R}^{m\times k}$ and $B\in\mathbb{R}^{k\times n}$ : $(AB)^T=B^TA^T$ . Transposing a product flips the order.

1

Write one entry of the product:

(AB)_{ij}=\sum_{l=1}^{k} a_{il}\,b_{lj}

Row

i

of

A

meets column

j

of

B

. That is all "row times column" says.

2

Transposing swaps the two positions of every entry:

\big((AB)^T\big)_{ji}=(AB)_{ij}=\sum_{l=1}^{k} a_{il}\,b_{lj}

This is the definition of transpose,

(M^T)_{ji}=M_{ij}

, used with

M=AB

.

3

Rename each number through its own transpose,

a_{il}=(A^T)_{li}

and

b_{lj}=(B^T)_{jl}

, and swap the two numbers in each term:

\sum_{l=1}^{k} a_{il}\,b_{lj}=\sum_{l=1}^{k} (B^T)_{jl}\,(A^T)_{li}

Nothing changed in value. We only renamed entries and swapped two ordinary numbers.

4

The right side is row

j

of

B^T

times column

i

of

A^T

:

\sum_{l=1}^{k} (B^T)_{jl}\,(A^T)_{li}=(B^TA^T)_{ji}

So the two matrices agree in every entry, and

(AB)^T=B^TA^T

. ∎ It is like taking off socks and shoes: the last thing on is the first thing off.

Pause & predict

Without computing anything: $(AB)^T$ equals…

In one sentence: a matrix is a grid of numbers, and multiplying two matrices means "row times column" for every cell, which is the same as doing one filter after another.

6

One equation, three windows

Imagine this

Look at a coin from the top: it is a circle. Look from the side: it is a thin bar. It is the same coin. Some questions are easy from the top ("how wide is it?") and some from the side ("how thick is it?").

$A\mathbf{x}=\mathbf{b}$ is like that coin. Learn to walk around it.

This is the most important section of the unit. Here is one tiny system, which we will look at in three ways:

\begin{aligned}2x_1+x_2&=5\\ x_1+3x_2&=5\end{aligned}\qquad\Longleftrightarrow\qquad A=\begin{pmatrix}2&1\\1&3\end{pmatrix},\ \ \mathbf{b}=\begin{pmatrix}5\\5\end{pmatrix}

The answer is $\mathbf{x}=(2,1)$ . Check: $2\cdot2+1=5$ and $2+3\cdot1=5$ . Now the three windows:

①Rows: each row is a rule. Each rule is a line; the answer is where the lines cross.
②Columns: $\mathbf{b}$ is a recipe. How much of column 1 and column 2 must you mix to make $\mathbf{b}$ ?
③Machine: $A$ moves all of space. Which input did it move onto $\mathbf{b}$ ?

Try: in the Columns window, set 2 scoops of a₁ and 1 scoop of a₂ and watch the blend land on b. Then lift b off the plane: now no recipe works.

Each row of $A$ , with its number from $\mathbf{b}$ , is one rule. Rule 1: $2x_1+x_2=5$ . Rule 2: $x_1+3x_2=5$ . Each rule is a line (a plane in 3D). Solving means finding the point that obeys every rule at once: here, the crossing at $(2,1)$ .

This is the school view, and it grows: ten equations, ten flat sheets, one crossing point.

drag the picture to orbit

Now forget the rows and look at the columns: $\mathbf{a}_1=\binom{2}{1}$ and $\mathbf{a}_2=\binom{1}{3}$ . It turns out $A\mathbf{x}=x_1\mathbf{a}_1+x_2\mathbf{a}_2$ . So $\mathbf{x}$ is a shopping list: how many scoops of each column make exactly $\mathbf{b}$ ?

x₁ (scoops of a₁)0.8

x₂ (scoops of a₂)2.4

Mix the columns to hit b.

Everything you could ever cook from these columns is called the column space of $A$ . If $\mathbf{b}$ lies inside it, a recipe exists. If not, there is no solution. Section 13 turns this into a test.

The deepest view. $A$ is a function: feed it any arrow $\mathbf{x}$ and it gives back $A\mathbf{x}$ . It moves all of space. Then $A\mathbf{x}=\mathbf{b}$ asks backwards: you can see the output $\mathbf{b}$ ; which input made it?

Run the machine0%

î → column 1 = (2,1)ĵ → column 2 = (1,3)x = (2,1) → b

As the machine runs, î and ĵ land on the two columns of $A$ , and the violet input $\mathbf{x}=(2,1)$ is carried onto $\mathbf{b}=(5,5)$ . Section 7 makes this the main event.

The realization

$A\mathbf{x}=x_1\mathbf{a}_1+x_2\mathbf{a}_2+\cdots+x_n\mathbf{a}_n$

Rows give you rules to obey. Columns give you ingredients to mix. The machine view asks which input made this output. Three pictures, one equation. And the big questions (is there an answer? just one?) belong to the machine $A$ itself, not to the particular $\mathbf{b}$ you happen to be holding.

If you want the algebra · why Ax = x₁a₁ + x₂a₂

Why Ax = x₁a₁ + x₂a₂ — derived

Claim. For any $2\times2$ matrix with columns $\mathbf{a}_1,\mathbf{a}_2$ : $A\mathbf{x}=x_1\mathbf{a}_1+x_2\mathbf{a}_2$ . The row window and the column window are the same arithmetic.

1

Multiply row by row, as the row window reads it:

A\mathbf{x}=\begin{pmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{pmatrix}\begin{pmatrix}x_1\\x_2\end{pmatrix}=\begin{pmatrix}a_{11}x_1+a_{12}x_2\\a_{21}x_1+a_{22}x_2\end{pmatrix}

Entry

i

is row

i

times

\mathbf{x}

. That is the definition of the product.

2

Split each entry into its

x_1

part and its

x_2

part:

\begin{pmatrix}a_{11}x_1+a_{12}x_2\\a_{21}x_1+a_{22}x_2\end{pmatrix}=\begin{pmatrix}a_{11}x_1\\a_{21}x_1\end{pmatrix}+\begin{pmatrix}a_{12}x_2\\a_{22}x_2\end{pmatrix}

Vectors add slot by slot, so this is just vector addition read backwards.

3

Pull the common number out of each vector:

\begin{pmatrix}a_{11}x_1\\a_{21}x_1\end{pmatrix}+\begin{pmatrix}a_{12}x_2\\a_{22}x_2\end{pmatrix}=x_1\begin{pmatrix}a_{11}\\a_{21}\end{pmatrix}+x_2\begin{pmatrix}a_{12}\\a_{22}\end{pmatrix}=x_1\mathbf{a}_1+x_2\mathbf{a}_2

The two vectors on the right are exactly the columns of

A

. ∎ With our numbers:

2\binom{2}{1}+1\binom{1}{3}=\binom{5}{5}

, the recipe the column window finds. The same three steps work for any size:

A\mathbf{x}=x_1\mathbf{a}_1+\cdots+x_n\mathbf{a}_n

.

Pause & predict

In the column window, "solve $A\mathbf{x}=\mathbf{b}$ " means…

In one sentence: Ax = b can be read as rules to obey (rows), ingredients to mix (columns), or a machine to run backwards, and switching windows is the key skill of linear algebra.

7

A matrix is a Photoshop filter for space

Imagine this

Draw a square grid on a rubber sheet. Now pull the sheet: stretch it, tilt it, lean it sideways. Every point moves, but straight lines stay straight, the centre stays put, and parallel lines stay parallel.

That is exactly what a matrix does to space. And there is a shortcut: to know the whole stretch, you only need to watch two arrows.

The two arrows are $\hat\imath=(1,0)$ , one step right, and $\hat\jmath=(0,1)$ , one step up. Take $A=\begin{pmatrix}2&0\\0&3\end{pmatrix}$ :

î $A\hat\imath=(2,0)$ , the first column. One step right becomes two steps right.
ĵ $A\hat\jmath=(0,3)$ , the second column. One step up becomes three steps up.
□The 1×1 square becomes a 2×3 rectangle. Its area went from 1 to 6, and $2\cdot3-0\cdot0=6$ .

That last number is the determinant: how much the machine scales area. For any $2\times2$ matrix, $\det A=a_{11}a_{22}-a_{12}a_{21}$ . Its sign tells you whether the plane was flipped over like a pancake. Watch it happen:

Try: press "Reflect" and watch the tile flip over to its orange back side. Then "Squash flat": the tile collapses to a line and the area reads 0. You can also drag the tips of î and ĵ.

drag the picture to orbit · drag an arrow tip to move it

A = type any entries
(−10 … 10)

det A = 1.00 area ×1.00

Space intact.

det > 0: the tile keeps its blue face up. det < 0: the plane is mirrored and the orange back shows. The size of det is the area factor. det = 0 is the event to watch: the tile goes flat, a whole direction dies, and Undo goes dark. Then two lines appear: violet, the null space (every arrow on it is crushed to the origin), and aqua, the column space (where every output now lives).

The realization

The columns of a matrix are where î and ĵ land. Once you know those two landing spots, you know where every point goes: $A\mathbf{x}=x_1(A\hat\imath)+x_2(A\hat\jmath)$ . And $\det A$ is the area of the tile those two arrows make: how much the machine grows or shrinks area, with a minus sign if it flips space over.

If you want the algebra · why Aî is column 1, and why ad − bc is an area

Why Aî is column 1 — derived

Claim. $A\hat\imath=\mathbf{a}_1$ and $A\hat\jmath=\mathbf{a}_2$ . So the whole map is fixed by where two arrows land: $A\mathbf{x}=x_1(A\hat\imath)+x_2(A\hat\jmath)$ .

1

Multiply out with

\mathbf{x}=\hat\imath=(1,0)^T

:

A\hat\imath=\begin{pmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix}=\begin{pmatrix}a_{11}\\a_{21}\end{pmatrix}=\mathbf{a}_1

The 0 wipes out the second column and the 1 keeps the first. With

(0,1)^T

you get

\mathbf{a}_2

.

2

Any input is a mix of the two basic arrows:

\mathbf{x}=\begin{pmatrix}x_1\\x_2\end{pmatrix}=x_1\begin{pmatrix}1\\0\end{pmatrix}+x_2\begin{pmatrix}0\\1\end{pmatrix}=x_1\hat\imath+x_2\hat\jmath

Scale each arrow by its coordinate and add. Check the two entries.

3

Use the column rule from Section 6 and step 1:

A\mathbf{x}=x_1\mathbf{a}_1+x_2\mathbf{a}_2=x_1(A\hat\imath)+x_2(A\hat\jmath)

Two landing spots decide every output. ∎ This is what "linear" buys you: the machine respects adding and scaling, so knowing it on two arrows means knowing it everywhere.

Why det = ad − bc is an area — derived

Claim. For $A=\begin{pmatrix}a&b\\c&d\end{pmatrix}$ with $a\neq0$ , the tile made by the columns $(a,c)^T$ and $(b,d)^T$ has signed area $ad-bc$ .

1

Lean the tile along its first side: replace the second side by

\mathbf{a}_2-\tfrac{b}{a}\mathbf{a}_1

.

\mathbf{a}_2-\tfrac{b}{a}\mathbf{a}_1=\begin{pmatrix}b-\tfrac{b}{a}a\\ d-\tfrac{b}{a}c\end{pmatrix}=\begin{pmatrix}0\\ \dfrac{ad-bc}{a}\end{pmatrix}

Sliding one side parallel to the other (a shear) changes the shape but not the area: same base, same height. We picked the slide so the new side points straight up.

2

Now lean the other way, so the first side lies flat:

\mathbf{a}_1-\frac{c}{(ad-bc)/a}\begin{pmatrix}0\\ \tfrac{ad-bc}{a}\end{pmatrix}=\begin{pmatrix}a\\ c-c\end{pmatrix}=\begin{pmatrix}a\\0\end{pmatrix}

Another shear, so the area is still the same. (If

ad-bc=0

, the upright side from step 1 is already zero: area 0, and the formula agrees.)

3

The tile is now a rectangle with sides

(a,0)

and

(0,\tfrac{ad-bc}{a})

:

\text{area}=\text{base}\times\text{height}=a\cdot\frac{ad-bc}{a}=ad-bc

Two shears, one rectangle, and the number falls out. ∎ The sign comes for free: if the second column sits clockwise from the first, the "height" is negative. That is the flip the widget shows as the orange back side. (If

a=0

but

c\neq0

, do the same steps with the two coordinates swapped.)

Watch out

When $\det A=0$ , one fact is told in three languages: ① $A$ has no inverse, so there is no undo. ② The columns are dependent: one is a mix of the others, so it brings no new direction. ③ Many inputs share one output, like a filter so strong that a hundred different photos come out identical. Such a matrix is called singular.

Up a dimension: the determinant is a volume

Nothing about the area story was special to flat 2D. In 3D, a matrix turns the unit cube (volume 1) into a slanted box, and $\det A$ is the box's volume. Example: this matrix makes a 2 × 1 × 3 box, so the volume is 6:

A=\begin{pmatrix}2&0&0\\0&1&0\\0&0&3\end{pmatrix},\qquad \det A=2\cdot1\cdot3=6

In more dimensions it is the same idea, just past where drawing works. Grab the box and spin it:

Try: press "Shear" and watch the box lean while det stays at 1. Then "Flatten" and watch the volume die.

drag the picture to orbit

A =

det A = 1.00 volume ×1.00 rank 3

A full 3-D volume — all three directions alive.

The dashed orange outline is the original cube. Shear is the one to watch: the box leans hard, yet det stays 1, because leaning is not shrinking. Rank 2 flattens the box to a sheet (volume 0) and rank 1 crushes it to a line. det = 0 and "rank below 3" are the same event, seen with two different instruments.

Pause & predict

You work out $\det A = 0$ for a $2\times2$ matrix. Which of these must be true?

In one sentence: a matrix moves space by sending î and ĵ to its columns, and its determinant is how much it scales area (or volume), with det = 0 meaning space got squashed flat.

8

The undo button — the inverse

Imagine this

You type a sentence into a translator: English to Hindi. Then you press the swap button and translate it back. If the translator is perfect, you get your original sentence.

A matrix can have a "translate back" button too. It is called the inverse, written $A^{-1}$ .

Tiny example. $A=\begin{pmatrix}2&0\\0&3\end{pmatrix}$ doubles the first number and triples the second: it sends $(1,1)$ to $(2,3)$ . The undo halves the first and divides the second by three:

A^{-1}=\begin{pmatrix}\tfrac12&0\\0&\tfrac13\end{pmatrix},\qquad A^{-1}\begin{pmatrix}2\\3\end{pmatrix}=\begin{pmatrix}1\\1\end{pmatrix}

In symbols, the inverse satisfies $A^{-1}A=AA^{-1}=I$ : do, then undo, and nothing has changed. If it exists, solving is one line: $A\mathbf{x}=\mathbf{b}\Rightarrow\mathbf{x}=A^{-1}\mathbf{b}$ . For any $2\times2$ matrix there is a ready formula:

A^{-1}=\frac{1}{a_{11}a_{22}-a_{12}a_{21}}\begin{pmatrix}a_{22}&-a_{12}\\-a_{21}&a_{11}\end{pmatrix}

Swap the two diagonal numbers, flip the sign of the other two, and divide by the determinant. Dividing by the determinant is the catch: if $\det A=0$ , there is no inverse.

If you want the algebra · why the 2×2 inverse formula works

Why the 2×2 inverse formula works — derived

Claim. For $A=\begin{pmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{pmatrix}$ with $\det A=a_{11}a_{22}-a_{12}a_{21}\neq0$ : $\ A^{-1}=\dfrac{1}{\det A}\begin{pmatrix}a_{22}&-a_{12}\\-a_{21}&a_{11}\end{pmatrix}$ .

1

Take

B=\begin{pmatrix}a_{22}&-a_{12}\\-a_{21}&a_{11}\end{pmatrix}

(swap the diagonal, flip the other signs) and compute

AB

entry by entry:

AB=\begin{pmatrix}a_{11}a_{22}-a_{12}a_{21} & -a_{11}a_{12}+a_{12}a_{11}\\ a_{21}a_{22}-a_{22}a_{21} & -a_{21}a_{12}+a_{22}a_{11}\end{pmatrix}

Row of

A

times column of

B

, four times. This is the step to redo yourself on paper.

2

The off-diagonal entries cancel. Both diagonal entries are the same number:

AB=\begin{pmatrix}\det A&0\\0&\det A\end{pmatrix}=(\det A)\,I_2

-a_{11}a_{12}+a_{12}a_{11}=0

because ordinary numbers can be multiplied in any order.

3

Divide by the number

\det A

, which is allowed exactly when it is not zero:

A\Big(\tfrac{1}{\det A}B\Big)=\tfrac{1}{\det A}(AB)=I_2

A number can slide through a matrix product. The same four multiplications give

BA=(\det A)I_2

too, so this undoes

A

from both sides.

4

So

A^{-1}=\tfrac{1}{\det A}B

, legal exactly when

\det A\neq0

. If

\det A=0

, step 2 says

AB=0

, and no number can rescue that. No inverse exists. ∎ The algebra agrees with the picture: an undo exists exactly when space was not squashed flat.

Try it yourself

For the Section 6 matrix $A=\begin{pmatrix}2&1\\1&3\end{pmatrix}$ : $\det A=2\cdot3-1\cdot1=5$ , so $A^{-1}=\tfrac15\begin{pmatrix}3&-1\\-1&2\end{pmatrix}$ . Work out $A^{-1}\binom{5}{5}$ by hand. You should land on $\binom{2}{1}$ , the same answer the crossing lines gave. Three windows, one truth.

Watch out

If the inverse is so neat, why do professionals avoid computing it?

It often does not exist. Only square matrices with $\det\neq0$ have one. A data matrix with 10,000 rows and 50 columns never does.
It is slow. Computing the whole inverse costs more than solving the one system you care about.
It is fragile. When $A$ is close to squashed, the inverse is full of huge numbers, and tiny rounding errors blow up.
It is wasteful. You wanted one door opened, not a master key for the whole building.

Rule of thumb: replace np.linalg.inv(A) @ b with np.linalg.solve(A, b). Same answer, faster and safer.

"Fragile" is best felt. Here is a system whose true answer is always $(1,1)$ , with a dial that slides the matrix toward squashed:

Try: first nudge δ a little with ε = 1 (not much happens). Then drag ε toward 0 and watch the same small nudge throw the answer off the map.

ε (det A)1

nudge b₂ by δ0

The system $\;x+y=2,\;\; x+(1+\varepsilon)y=2+\varepsilon\;$ has $\det A=\varepsilon$ and

A⁻¹ = 2−1−11

det = 1 error amplified ×1.4 (= √2/ε)

Comfortably invertible.

δ plays a tiny measurement error, a wobble in your data. The true answer is (1, 1); the computed one is (1 − δ/ε, 1 + δ/ε). So the wobble comes out multiplied by √2/ε. As ε shrinks, the lines close like scissors and the crossing point skates away on a hair-trigger.

The realization

The inverse exists exactly when $\det A\neq0$ , because you cannot un-squash something that was squashed flat. And even when it exists, a determinant close to zero makes the inverse huge and the answer shaky. In practice we solve with elimination and never build $A^{-1}$ .

Pause & predict

You need $\mathbf{x}$ for one system $A\mathbf{x}=\mathbf{b}$ in real code. What is the professional move?

In one sentence: the inverse is the undo button for a matrix; it exists only when det A is not zero, and in real work we solve with elimination instead of computing it.

9

A system with training wheels

Imagine this

At a sandwich shop the rule is: every sandwich needs bread and one protein. Beyond that, add any toppings you like.

So there is one base sandwich that follows the rule, and then any mix of toppings you pile on. Every legal sandwich is "base + toppings". Solutions to equations work the same way.

Here is a system so friendly it almost solves itself. Two equations, four unknowns:

\begin{pmatrix}1&0&8&-4\\0&1&2&12\end{pmatrix}\begin{pmatrix}x_1\\x_2\\x_3\\x_4\end{pmatrix}=\begin{pmatrix}42\\8\end{pmatrix}

Use the column window. Columns 1 and 2 are $\binom{1}{0}$ and $\binom{0}{1}$ , so they can build any target directly: 42 of column 1, 8 of column 2, none of the rest. That is the base sandwich, called a particular solution: $\mathbf{x}_p=(42,\,8,\,0,\,0)$ .

Now the toppings. Column 3 is $\binom{8}{2}$ , which is 8 of column 1 plus 2 of column 2. So "8 of column 1, 2 of column 2, minus one column 3" adds up to nothing:

\begin{pmatrix}1&0&8&-4\\0&1&2&12\end{pmatrix}\begin{pmatrix}8\\2\\-1\\0\end{pmatrix}=\begin{pmatrix}8+0-8+0\\0+2-2+0\end{pmatrix}=\begin{pmatrix}0\\0\end{pmatrix}

Column 4 gives a second topping in the same way, $(-4,\,12,\,0,\,-1)$ . Adding "nothing" to a solution keeps it a solution, so every answer is

\mathbf{x}=\underbrace{\begin{pmatrix}42\\8\\0\\0\end{pmatrix}}_{\text{base}}+\ \lambda_1\underbrace{\begin{pmatrix}8\\2\\-1\\0\end{pmatrix}}_{\text{topping 1}}+\ \lambda_2\underbrace{\begin{pmatrix}-4\\12\\0\\-1\end{pmatrix}}_{\text{topping 2}}

for any numbers $\lambda_1,\lambda_2$ .

Try: push both sliders to their ends. Every entry of x moves, yet A·x reads (42, 8) the whole time.

λ₁0

λ₂0

x = 42800 → A·x = 428

—

✓ output pinned at b = (42, 8)

The readout works out each output from the current x. The λ parts always cancel, because both toppings were built to give A·x = 0. Four unknowns but only two real rules leave two free directions.

The realization

all solutions = one solution + every "invisible" direction

The invisible directions are the arrows the matrix sends to zero: solutions of $A\mathbf{x}=\mathbf{0}$ . Together they are called the null space of $A$ . You will meet "one answer plus the null space" again in regression, in optimization, and inside neural networks.

Two honest notes. The base is not unique: any single solution can play the base. And the toppings are not unique either: other mixes of them describe the same set. What is unique is the whole set of solutions, a flat sheet inside 4D space. Those invisible directions are so important that they get the next section to themselves.

In one sentence: when a system has infinitely many solutions, they all look like one particular answer plus any mix of the directions the matrix sends to zero.

10

The null space — the machine's blind spot

Imagine this

A torch throws the shadow of your hand onto a wall. Hand in, shadow out: it is a machine.

Now slide your hand straight along the beam, toward the torch. The shadow does not move at all. That direction is invisible to the machine. So when you see a shadow, the hand could be anywhere along that beam: infinitely many hands, one shadow.

Tiny example. Take $A=\begin{pmatrix}1&2\\2&4\end{pmatrix}$ (the second row is just double the first). Feed it $\mathbf{x}_n=(-2,1)$ :

A\mathbf{x}_n=\begin{pmatrix}1\cdot(-2)+2\cdot1\\ 2\cdot(-2)+4\cdot1\end{pmatrix}=\begin{pmatrix}0\\0\end{pmatrix}

The machine sends it to zero. That arrow is the "beam". Every input the machine sends to zero is collected in the null space:

N(A)=\{\,\mathbf{x}\;:\;A\mathbf{x}=\mathbf{0}\,\}

It always contains $\mathbf{0}$ , because $A\mathbf{0}=\mathbf{0}$ . The interesting question is whether anything else is in there. If yes, solutions come in families. Suppose $A\mathbf{x}_p=\mathbf{b}$ . Add any amount of a null arrow:

\begin{aligned}A(\mathbf{x}_p+\lambda\,\mathbf{x}_n)&=A\mathbf{x}_p+\lambda\,A\mathbf{x}_n\\&=\mathbf{b}+\mathbf{0}=\mathbf{b}\end{aligned}

And the other way round: if $\mathbf{u}$ and $\mathbf{v}$ both solve it, then $A(\mathbf{u}-\mathbf{v})=\mathbf{b}-\mathbf{b}=\mathbf{0}$ , so any two solutions differ by a null arrow. Here $\mathbf{b}=(4,8)$ and $\mathbf{x}_p=(0,2)$ . Watch the hand slide along the beam:

Try: press ▶. The arrow x travels the whole beam, past its faint ghosts, and the shadow on the wall never twitches. Below, the same story in 2D: the dot walks the blue line while the output stays put.

solution line xₚ + λxₙ (input)null space N(A) and the λxₙ stepcolumn space (every possible output)

INPUT SPACE — where x lives

OUTPUT SPACE — where Ax lands

λ (walk the line)0.6

—

✓ output frozen at b

Left: the violet dashed line through the origin is $N(A)$ itself. The solid blue line is the same line, picked up and parked at $\mathbf{x}_p$ . It misses the origin, so the solution set is a shifted copy of the null space, not a space of its own. Right: every possible output lives on the aqua line, the column space. This squashed machine can only hit targets on that line.

The realization

solutions of $A\mathbf{x}=\mathbf{b}$ = $\mathbf{x}_p$ + all of $N(A)$

One anchor, plus the whole blind spot. If the blind spot is only $\{\mathbf{0}\}$ , there are no look-alikes, and a solution (when there is one) is unique. You already know that case by its other names: $\det\neq0$ , invertible.

Watch out — this is everywhere in ML

Put two perfectly linked columns in your data, say temperature in °C and the same temperature in °F. You have just created a null space: some mix of those two features is invisible to the model. Result: infinitely many weight vectors give identical predictions, and the fitted weights can swing wildly without changing the fit. The usual fix is to pick the shortest weight vector among all the look-alikes, or to add a small penalty on size (ridge regression). You will meet both later.

How big is the blind spot? Each independent null direction adds one dimension: a line, a plane, or more. Elimination will count it for us in Sections 11–12: one free variable for each missing pivot.

Pause & predict

$\mathbf{x}_0$ solves $A\mathbf{x}=\mathbf{b}$ (with $\mathbf{b}\neq\mathbf{0}$ ), and $\mathbf{x}_n\neq\mathbf{0}$ is in $N(A)$ . Which of these is sure to be another solution?

In one sentence: the null space holds every input the matrix sends to zero, and adding any of them to a solution gives another solution, which is why answers come in families.

11

Gaussian elimination — the algorithm

Imagine this

Three friends split a restaurant bill, and you get three tangled clues about who paid what. By hand, you would use one clue to remove a person from the other clues. Then again. Soon one clue mentions just one person, and everything unravels backwards.

A computer needs the same trick, but with no guessing and no choices: the same kind of step every time.

Tiny example: $x_1+2x_2=5$ and $3x_1+4x_2=11$ . Write only the numbers, with a bar where the "=" was. This is the augmented matrix $[A\,|\,\mathbf{b}]$ . One move clears the 3:

\left[\begin{array}{cc|c}1&2&5\\3&4&11\end{array}\right]\xrightarrow{\;R_2\to R_2-3R_1\;}\left[\begin{array}{cc|c}1&2&5\\0&-2&-4\end{array}\right]

The last row now says $-2x_2=-4$ , so $x_2=2$ . Then the first row gives $x_1=5-2\cdot2=1$ . That is the whole algorithm. You only ever need three legal moves:

①Swap two rows: write the equations in a different order.
②Scale a row by a number that is not zero: multiply both sides of one equation.
③Add a multiple of one row to another: add equal things to equal things.

The goal is a staircase: each row starts further right than the row above, with only zeros below the steps. That shape is called row-echelon form (REF). The first number on each step is a pivot.

Try: press ▶ on the full example. Rows light up as they change, blocks below each pivot sink to zero, and a glowing staircase draws itself. At the last step, a slider for a appears.

Step 0/0

drag the picture to orbit

parameter a0

—

With $a=-1$ : set the free variables $x_2,x_5$ to 0 and solve upward to get one solution. Then set one free variable to 1 (the other to 0) and solve $A\mathbf{x}=\mathbf{0}$ to get each invisible direction:

\mathbf{x}=\begin{pmatrix}2\\0\\-1\\1\\0\end{pmatrix}+\lambda_1\begin{pmatrix}2\\1\\0\\0\\0\end{pmatrix}+\lambda_2\begin{pmatrix}2\\0\\-1\\2\\1\end{pmatrix}

Base sandwich plus two toppings, just like Section 9. The null space is $5-3=2$ -dimensional: one direction per free variable. The staircase told us everything: pivots in columns 1, 3, 4 make $x_1,x_3,x_4$ bound; columns 2 and 5 have no pivot, so $x_2,x_5$ are free.

The realization

Elimination rewrites the same problem until the answer is obvious. Every move can be undone, so the solutions never change. At the end, the staircase tells you everything: pivot columns are pinned-down unknowns, columns without a pivot are free, and a row that says $0=$ something non-zero means no solution. In the big example the last row became " $0=a+1$ ", so there is a solution only when $a=-1$ .

If you want the algebra · why row operations never change the solutions

Why row operations never change the solutions — derived

Claim. If $[A'\,|\,\mathbf{b}']$ comes from $[A\,|\,\mathbf{b}]$ by any of the three moves, then $A\mathbf{x}=\mathbf{b}$ and $A'\mathbf{x}=\mathbf{b}'$ have exactly the same solutions.

1

Every old solution solves the new system. A solution

\mathbf{x}

makes each equation true (left side = right side), and each move keeps true equations true:

\begin{aligned}\text{swap:}\quad&\text{the same true equations, new order}\\ \text{scale:}\quad&\lambda\cdot\text{LHS}_i=\lambda\cdot\text{RHS}_i\\ \text{add:}\quad&\text{LHS}_i+c\,\text{LHS}_j=\text{RHS}_i+c\,\text{RHS}_j\end{aligned}

Multiplying a true equation by a number keeps it true. Adding two true equations gives a true equation.

2

Every move can be undone by a move of the same kind:

\begin{aligned}R_i\leftrightarrow R_j\quad&\text{is undone by}\quad R_i\leftrightarrow R_j\\ R_i\to\lambda R_i\quad&\text{is undone by}\quad R_i\to\tfrac1\lambda R_i\\ R_i\to R_i+cR_j\quad&\text{is undone by}\quad R_i\to R_i-cR_j\end{aligned}

This is exactly where

\lambda\neq0

is needed:

\tfrac1\lambda

must exist. (Check c7 below asks what goes wrong otherwise.)

3

Apply step 1 to the undo move: every new solution solves the old system. So each solution set sits inside the other:

\text{solutions}(A,\mathbf{b})=\text{solutions}(A',\mathbf{b}')

The solution sets are identical. ∎ A chain of moves is a chain of equalities, so the staircase at the end has exactly the solutions you started with. That is why reading the answer off it is fair.

The staircase of the big example (with a = −1): pivot columns 1, 3, 4 pin down x₁, x₃, x₄; columns 2 and 5 are free.

Pause & predict

Move ② lets you scale a row by any $\lambda \neq 0$ . Why exactly is $\lambda = 0$ not allowed?

In one sentence: Gaussian elimination uses swap, scale and add-a-multiple to turn the augmented matrix into a staircase, without ever changing the solutions, and the staircase then shows the answer and the system's fate.

12

RREF — finish the job, and two tricks fall out

Imagine this

A staircase (REF) is like a half-tidied room: everything is in the right corner, but you still have to dig a little to find things.

Keep tidying until every item sits alone on its own labelled shelf. Now you can read off anything at a glance. That fully tidied form is RREF.

Reduced row-echelon form adds two rules to the staircase: every pivot is a 1, and it is the only non-zero number in its column (so we also clear upward). Tiny example:

\left[\begin{array}{cc|c}1&2&5\\0&-2&-4\end{array}\right]\xrightarrow{\;R_2\to -\frac12R_2\;}\left[\begin{array}{cc|c}1&2&5\\0&1&2\end{array}\right]\xrightarrow{\;R_1\to R_1-2R_2\;}\left[\begin{array}{cc|c}1&0&1\\0&1&2\end{array}\right]

The last matrix simply says $x_1=1$ , $x_2=2$ . No back-substitution needed. And unlike REF, which depends on the moves you chose, RREF is always the same for a given matrix: it is its fingerprint.

The realization

In RREF, every pivot column is a clean "1 and zeros" column, so the answer can be read straight off the bar. Every column without a pivot is a free variable, and each one gives exactly one invisible direction. So the size of the null space is

$\dim N(A) = n - \operatorname{rank}(A)$

unknowns minus pivots: freedom left over = unknowns minus the rules that really pin something down. (This is the rank–nullity theorem.)

Trick 1: reading the null space by eye. Take this RREF from the slides:

A=\begin{pmatrix}1&3&0&0&3\\0&0&1&0&9\\0&0&0&1&-4\end{pmatrix}

Column 2 is 3 times column 1, so "3 of column 1, minus column 2" gives zero: the null arrow $(3,-1,0,0,0)$ . Column 5 is $3$ ·col 1 $+\ 9$ ·col 3 $-\ 4$ ·col 4, which gives $(3,0,9,-4,-1)$ . The pattern: copy the free column's recipe, and put −1 in its own slot. Add a right-hand side, say $(2,1,-1)$ , and one solution is read off the bar: put each right-hand number in its pivot's slot and zeros in the free slots, $\mathbf{x}_p=(2,0,1,-1,0)$ . The full answer:

\mathbf{x}=\begin{pmatrix}2\\0\\1\\-1\\0\end{pmatrix}+\lambda_1\begin{pmatrix}3\\-1\\0\\0\\0\end{pmatrix}+\lambda_2\begin{pmatrix}3\\0\\9\\-4\\-1\end{pmatrix}

Trick 2: the inverse for free. If $A$ is invertible, its RREF is $I$ . So glue $I$ to its right, $[A\,|\,I]$ , and reduce the left half to $I$ . The right half becomes $A^{-1}$ . The second tab above does it for the Section 6 matrix and lands on $\tfrac15\begin{pmatrix}3&-1\\-1&2\end{pmatrix}$ , the same inverse Section 8 found.

The null space is also a real space: if $A\mathbf{u}=\mathbf{0}$ and $A\mathbf{v}=\mathbf{0}$ , then $A(\mathbf{u}+\mathbf{v})=\mathbf{0}$ and $A(\lambda\mathbf{u})=\mathbf{0}$ . Add or scale invisible arrows and they stay invisible. Unit 2 gives these spaces their full treatment.

If you want the algebra · why dim N(A) = n − rank

Why dim N(A) = n − rank — derived

Claim. If the RREF of $A$ has $r$ pivots among $n$ columns, then $N(A)$ has a basis of exactly $n-r$ vectors: one per free column.

1

Row moves do not change

N(A)

(Section 11 with

\mathbf{b}=\mathbf{0}

), so work with the RREF

R

. Each pivot row of

R

reads

x_{p_i}+\sum_{f\ \text{free}} r_{if}\,x_f=0\ \ \Longrightarrow\ \ x_{p_i}=-\sum_{f\ \text{free}} r_{if}\,x_f

In RREF a pivot column has a single 1, so each pivot variable appears in exactly one equation and can be written using free variables only. Zero rows say

0=0

and add nothing.

2

The free variables can be anything. For each free column

f

, let

\mathbf{v}_f

be the solution with

x_f=1

and every other free variable

0

. Then every solution is

\mathbf{x}=\sum_{f\ \text{free}} x_f\,\mathbf{v}_f

Step 1 makes each pivot entry a fixed mix of the free values, so plugging in one free variable at a time and adding rebuilds the general solution.

3

The

n-r

vectors

\mathbf{v}_f

are independent. Suppose

\sum_f c_f\mathbf{v}_f=\mathbf{0}

. Look at the slot of free variable

g

:

\Big(\sum_f c_f\mathbf{v}_f\Big)_{g}=c_g\cdot1+\sum_{f\neq g}c_f\cdot0=c_g

so every

c_g=0

. They span the null space and are independent, so they are a basis with

n-r

members. ∎ The 1-and-0 pattern in the free slots is the whole argument. In Trick 1,

(3,-1,0,0,0)

and

(3,0,9,-4,-1)

carry

-1

in slots 2 and 5: same idea, opposite sign.

Pause & predict

In the stepper above, the left block of $[A\,|\,I]$ became $I$ and the right block became $\tfrac15\begin{pmatrix}3&-1\\-1&2\end{pmatrix}$ . Why is that right block $A^{-1}$ ?

In one sentence: RREF keeps eliminating until every pivot is a lone 1, so solutions, null-space directions and even the inverse can be read straight off the page.

13

Rank — one number that settles everything

Imagine this

A family WhatsApp group has three members. Whenever one of them posts an opinion, the other two just repeat it.

Three people, but only one real opinion. The group is really a one-person chat. Rank counts the real voices in a matrix: how many rows (or columns) are not just copies or mixes of the others.

One digit can change the count:

B=\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix}\ \ \text{rank }2,\qquad C=\begin{pmatrix}1&2&3\\4&5&6\\7&8&\mathbf{10}\end{pmatrix}\ \ \text{rank }3

In $B$ , row 3 = 2 × row 2 − row 1 (check: $8-1=7$ , $10-2=8$ , $12-3=9$ ). Row 3 is an echo. Change the 9 to a 10 and the echo breaks: three real voices. Elimination shows it directly: $B$ ends with two pivots and a row of zeros, $C$ keeps three pivots. Rank = the number of pivots.

Now count $B$ 's columns. Column 3 = 2 × column 2 − column 1, so only two columns are independent too. Two real rules (rows), two real ingredients (columns). That is not luck:

The staircase is counted once and read twice: down the rows, and across the columns.

Row rank = column rank, always. Each pivot marks one independent row and one independent column at the same time, so the two counts must match. That is why we just say "rank". Rank can never be bigger than the number of rows or columns: $\operatorname{rank}(A)\le\min(m,n)$ . A matrix at that ceiling is full rank.

Rank also joins our list of "squashed" signals. For a square $n\times n$ matrix: invertible $\iff$ rank $=n$ $\iff$ $\det\neq0$ $\iff$ columns independent. In 3D, drop one rank and the cube flattens to a sheet; drop two and it becomes a line. Rank is how many dimensions survive the machine.

The realization

Rank answers "how many solutions?" completely. Compare $\operatorname{rank}(A)$ with $\operatorname{rank}([A|\mathbf{b}])$ and with $n$ , the number of unknowns:

① At least one solution $\iff\operatorname{rank}(A)=\operatorname{rank}([A|\mathbf{b}])$ . Adding $\mathbf{b}$ brought no new direction, so $\mathbf{b}$ is in the column space.

② Exactly one $\iff$ ① holds and $\operatorname{rank}(A)=n$ : no free variables.

③ Infinitely many $\iff$ ① holds and $\operatorname{rank}(A)<n$ : free variables exist, and the null space is alive.

If you want the algebra · why "solvable ⇔ rank(A) = rank([A|b])"

Why "consistent ⇔ rank(A) = rank([A|b])" — derived

Claim. $A\mathbf{x}=\mathbf{b}$ has at least one solution exactly when adding the column $\mathbf{b}$ to $A$ does not raise the rank.

1

Run elimination on

[A\,|\,\mathbf{b}]

to reach a staircase

[R\,|\,\mathbf{c}]

. The solutions do not change (Section 11), and

R

is a staircase for

A

on its own:

\begin{aligned}\operatorname{rank}(A)&=\#\{\text{pivots of }R\}\\ \operatorname{rank}([A\,|\,\mathbf{b}])&=\#\{\text{pivots of }[R\,|\,\mathbf{c}]\}\end{aligned}

The same moves act on the whole row, so one elimination run gives both ranks.

2

Every pivot of

R

is also a pivot of

[R\,|\,\mathbf{c}]

. The only possible extra pivot sits in the last column, and it appears exactly when some row is all zeros on the left but has

c\neq0

on the right:

\big[\,0\ \ 0\ \cdots\ 0\ \big|\ c\,\big],\qquad c\neq0

A pivot is the first non-zero number in a row. For such a row, that first non-zero number is in the

\mathbf{b}

column: one new pivot. No other kind of row can make one.

3

Read that row as an equation:

0x_1+\cdots+0x_n=c

, that is,

0=c\neq0

. Impossible for every

\mathbf{x}

. If no such row exists, back-substitution works: set free variables to 0 and solve each pivot row from the bottom up.

\text{ranks differ}\iff\text{a row }0=c\neq0\text{ exists}\iff\text{no solution}

So equal ranks and "solvable" are the same statement. ∎ Practice Problem 3 hits exactly such a row,

[\,0\ 0\ 0\,|\,2\,]

. Problem 4, with the same

A

and a different

\mathbf{b}

, does not.

Watch all three lines come true in 3D. The first two planes never change; only the third equation does. The rank chips call each fate before your eyes do:

Try: orbit until you can see the dashed lines where pairs of planes meet: three lines through one point, one shared line, or three parallel lines that never meet.

drag the picture to orbit

rank(A) = 3 rank([A|b]) = 3 n = 3 n − rank = 0 free direction(s)

—

Pause & predict

Elimination on a system with 3 unknowns gives $\operatorname{rank}(A)=\operatorname{rank}([A|\mathbf{b}])=2$ . What do the solutions look like?

Pause & predict

$A$ is $5\times 7$ with $\operatorname{rank}(A)=5$ . For a given $\mathbf{b}$ , how many solutions can $A\mathbf{x}=\mathbf{b}$ have?

In one sentence: rank is the number of pivots, the number of truly independent rows and of columns, and comparing rank(A), rank([A|b]) and n tells you whether a system has zero, one or infinitely many solutions.

14

The whole story, in five lines

Imagine this

Someone hands you a sealed machine and one output it produced. You want to know: could it have made this? Was there only one way? You don't need to open the machine. You need one picture, one method and one number.

①The world looks straight up close, and machine learning is stacks of straight pieces with a little bend between them.
② $A\mathbf{x}=\mathbf{b}$ is three pictures at once: rules to obey, ingredients to mix, a machine to run backwards.
③ $\mathbf{x}=A^{-1}\mathbf{b}$ is beautiful on paper and poor in practice: solve, don't inv.
④Elimination builds a staircase; from it you read the solutions, the rank, the null space and even the inverse.
⑤Every later method in the course (LU, Cholesky, QR, even the SVD) is a smarter version of the same elimination idea.

The realization

One picture: a matrix is a machine that moves space; solving $A\mathbf{x}=\mathbf{b}$ asks which input the machine turned into that output. One method: elimination on $[A|\mathbf{b}]$ until the staircase appears. One number: rank, which counts the dimensions that survive and decides all three fates.

What's coming — every later method is elimination in disguise

Method	In one sentence	Where
LU	Elimination, written down once, so you can solve for many b's cheaply.	later (Unit 5)
Cholesky	Elimination that uses symmetry to do half the work.	later (Unit 5)
QR / least squares	Elimination by rotations: the right tool when no exact answer exists, and the maths behind regression.	Units 3 and 5
Eigen / SVD	The deepest split: any matrix = rotate · stretch · rotate. The road to PCA.	Units 4–5, then PCA

In one sentence: a matrix moves space, elimination reveals what it did, and rank tells you whether Ax = b has zero, one or infinitely many answers.

15

Practice arena — the unit's problem set, solved in full

These are Prof. Saurabh's ten practice problems for this unit — seven core problems at easy and medium level, then three hard ones. Attempt each one on paper first, honestly, before unfolding the solution: every fold-out works the problem step by step, entry by entry, with nothing skipped.

Problem 1Easy

Solve the system below. Do it first by reducing to REF and back substituting, then again by going all the way to RREF. State $\operatorname{rank}(A)$ , $\operatorname{rank}([A\,|\,\mathbf{b}])$ and the number of unknowns, and use them to justify the number of solutions.

\begin{aligned} x + 2y &= 8\\ 3x - y &= 3 \end{aligned}

What this tests. The mechanics of one elimination step, and the difference between stopping at REF and pushing on to RREF.

Show the worked solution — every step

Method 1 · Step 1 · Write the augmented matrix. Two unknowns, so

n=2

.

[A\,|\,\mathbf{b}]=\left[\begin{array}{cc|c}1&2&8\\3&-1&3\end{array}\right]

Method 1 · Step 2 · Clear below the first pivot. The pivot is the

1

in position

(1,1)

. To kill the

3

under it, use

R_2\to R_2-3R_1

. Compute the new row entry by entry:

3-3(1)=0

,

-1-3(2)=-7

,

3-3(8)=-21

.

\left[\begin{array}{cc|c}1&2&8\\3&-1&3\end{array}\right] \xrightarrow{\;R_2\to R_2-3R_1\;} \left[\begin{array}{cc|c}1&2&8\\0&-7&-21\end{array}\right]

This is REF. Pivots are

1

(column 1) and

-7

(column 2).

Method 1 · Step 3 · Read the ranks. Two pivots inside the first two columns, so

\operatorname{rank}(A)=2

. No pivot appears in the

\mathbf{b}

column, so

\operatorname{rank}([A\,|\,\mathbf{b}])=2

as well. And

n=2

.

Method 1 · Step 4 · Classify. Since

\operatorname{rank}(A)=\operatorname{rank}([A\,|\,\mathbf{b}])=2=n

, the system has exactly one solution.

Method 1 · Step 5 · Back substitute. The last row says

-7y=-21

, so

y=3

. The first row says

x+2y=8

, so

x=8-2(3)=2

.

Method 1 · Step 6 · Check in the originals.

2+2(3)=8\ \checkmark

and

3(2)-3=3\ \checkmark

.

Method 2 · Step 1 · Start from the REF above.

\left[\begin{array}{cc|c}1&2&8\\0&-7&-21\end{array}\right]

Method 2 · Step 2 · Make the second pivot equal to 1. Scale with

R_2\to -\tfrac{1}{7}R_2

:

\left[\begin{array}{cc|c}1&2&8\\0&1&3\end{array}\right]

Method 2 · Step 3 · Clear above that pivot. Column 2 still has a

2

sitting above the pivot. Use

R_1\to R_1-2R_2

:

1-0=1

,

2-2(1)=0

,

8-2(3)=2

.

\left[\begin{array}{cc|c}1&0&2\\0&1&3\end{array}\right]

This is the RREF.

Method 2 · Step 4 · Read the answer. The rows now say

x=2

and

y=3

directly. No back substitution was needed.

Answer.

x=2,\ y=3

— unique, because

\operatorname{rank}(A)=\operatorname{rank}([A\,|\,\mathbf{b}])=2=n

. Method 2 lands on the same solution; the RREF of

[A\,|\,\mathbf{b}]

is

[\,I \,|\, (2,3)^{\top}\,]

.

Key takeaway

When the left block of the RREF turns out to be the identity matrix, the right column is the solution. That only happens when $\operatorname{rank}(A)=n=m$ .

Problem 2Easy

Use Gauss–Jordan elimination to solve the system below. Confirm your classification with the rank criterion.

\begin{aligned} x + y + z &= 6\\ 2x - y + z &= 3\\ x + 2y - z &= 2 \end{aligned}

What this tests. Handling three rows without losing track, and using a row swap to dodge fractions.

Show the worked solution — every step

Step 1 · Augmented matrix. Here

n=3

.

\left[\begin{array}{ccc|c}1&1&1&6\\2&-1&1&3\\1&2&-1&2\end{array}\right]

Step 2 · Clear column 1 below the pivot. Two operations, both using

R_1

. For

R_2\to R_2-2R_1

:

2-2=0

,

-1-2=-3

,

1-2=-1

,

3-12=-9

. For

R_3\to R_3-R_1

:

1-1=0

,

2-1=1

,

-1-1=-2

,

2-6=-4

.

\left[\begin{array}{ccc|c}1&1&1&6\\0&-3&-1&-9\\0&1&-2&-4\end{array}\right]

Step 3 · Swap to get a friendly pivot. The next pivot should sit in column 2. Taking

-3

would force fractions immediately, so swap:

R_2\leftrightarrow R_3

.

\left[\begin{array}{ccc|c}1&1&1&6\\0&1&-2&-4\\0&-3&-1&-9\end{array}\right]

Step 4 · Clear below the second pivot. Use

R_3\to R_3+3R_2

:

-3+3(1)=0

,

-1+3(-2)=-7

,

-9+3(-4)=-21

.

\left[\begin{array}{ccc|c}1&1&1&6\\0&1&-2&-4\\0&0&-7&-21\end{array}\right]

This is REF, with three pivots.

Step 5 · Classify before finishing.

\operatorname{rank}(A)=3

,

\operatorname{rank}([A\,|\,\mathbf{b}])=3

,

n=3

. All equal, so exactly one solution exists.

Step 6 · Normalise the third pivot.

R_3\to -\tfrac{1}{7}R_3

gives

[\,0\ 0\ 1\,|\,3\,]

.

\left[\begin{array}{ccc|c}1&1&1&6\\0&1&-2&-4\\0&0&1&3\end{array}\right]

Step 7 · Clear column 3 above the pivot. Two operations:

R_2\to R_2+2R_3

gives

-4+2(3)=2

in the last slot and

-2+2=0

in column 3. Then

R_1\to R_1-R_3

gives

6-3=3

and

1-1=0

.

\left[\begin{array}{ccc|c}1&1&0&3\\0&1&0&2\\0&0&1&3\end{array}\right]

Step 8 · Clear column 2 above the pivot.

R_1\to R_1-R_2

gives

3-2=1

.

\left[\begin{array}{ccc|c}1&0&0&1\\0&1&0&2\\0&0&1&3\end{array}\right]

This is the RREF.

Step 9 · Check in the originals.

1+2+3=6\ \checkmark

;

2(1)-2+3=3\ \checkmark

;

1+2(2)-3=2\ \checkmark

.

Answer.

x=1,\ y=2,\ z=3

. Unique, since

\operatorname{rank}(A)=\operatorname{rank}([A\,|\,\mathbf{b}])=3=n

.

Watch out

Clear column 3 before column 2 when going upward. If you clean column 2 first, the later column-3 operation puts new junk back into column 2 and you loop forever. Going right to left on the way up is the rule.

Problem 3Easy

Show that the following system has no solution, and identify the exact row of the reduced matrix that proves it.

\begin{aligned} x + y + z &= 3\\ 2x + 3y + z &= 5\\ 3x + 4y + 2z &= 10 \end{aligned}

What this tests. Spotting inconsistency, and saying why in rank language rather than by feel.

Show the worked solution — every step

Step 1 · Augmented matrix.

\left[\begin{array}{ccc|c}1&1&1&3\\2&3&1&5\\3&4&2&10\end{array}\right]

Step 2 · Clear column 1. With

R_2\to R_2-2R_1

:

2-2=0

,

3-2=1

,

1-2=-1

,

5-6=-1

. With

R_3\to R_3-3R_1

:

3-3=0

,

4-3=1

,

2-3=-1

,

10-9=1

.

\left[\begin{array}{ccc|c}1&1&1&3\\0&1&-1&-1\\0&1&-1&1\end{array}\right]

Rows 2 and 3 now agree on the left of the bar but disagree on the right. That is the whole story, and the next step makes it official.

Step 3 · Clear below the second pivot.

R_3\to R_3-R_2

:

0-0=0

,

1-1=0

,

-1-(-1)=0

,

1-(-1)=2

.

\left[\begin{array}{ccc|c}1&1&1&3\\0&1&-1&-1\\0&0&0&2\end{array}\right]

Step 4 · Read the fatal row. Row 3 is

[\,0\ 0\ 0\,|\,2\,]

. As an equation it says

0\cdot x + 0\cdot y + 0\cdot z = 2, \quad\text{that is}\quad 0=2,

which no choice of

x,y,z

can make true.

Step 5 · Say it with ranks. Inside the first three columns there are two pivots, so

\operatorname{rank}(A)=2

. Across all four columns there are three pivots, because row 3 has its pivot in the

\mathbf{b}

column, so

\operatorname{rank}([A\,|\,\mathbf{b}])=3

. Since

\operatorname{rank}(A)=2 < 3=\operatorname{rank}([A\,|\,\mathbf{b}])

, the system is inconsistent.

Step 6 · Where it came from. Notice

R_1+R_2=[\,3\ 4\ 2\,]

on the left, matching row 3 exactly. So the left sides satisfy

\text{eq}_1+\text{eq}_2=\text{eq}_3

, but the right sides give

3+5=8\neq 10

. The equations contradict each other.

Answer. No solution. The row

[\,0\ 0\ 0\,|\,2\,]

proves it, and in rank terms

\operatorname{rank}(A)=2 < \operatorname{rank}([A\,|\,\mathbf{b}])=3

.

Watch out

Once you hit a row $[\,0\cdots 0\,|\,c\,]$ with $c\neq 0$ , stop. Do not carry on to RREF and do not try to write a solution. There is nothing further to compute.

Problem 4Medium

Solve the system below. Give the general solution in parametric form and in vector form, and state how many free variables there are and why.

\begin{aligned} x + y + z &= 3\\ 2x + 3y + z &= 5\\ 3x + 4y + 2z &= 8 \end{aligned}

What this tests. The same matrix $A$ as Problem 3 with a different $\mathbf{b}$ . One number changed and the answer went from "none" to "infinitely many".

Show the worked solution — every step

Step 1 · Augmented matrix.

\left[\begin{array}{ccc|c}1&1&1&3\\2&3&1&5\\3&4&2&8\end{array}\right]

Step 2 · Clear column 1.

R_2\to R_2-2R_1

gives

[\,0\ 1\ {-1}\,|\,{-1}\,]

exactly as before.

R_3\to R_3-3R_1

now gives

3-3=0

,

4-3=1

,

2-3=-1

,

8-9=-1

.

\left[\begin{array}{ccc|c}1&1&1&3\\0&1&-1&-1\\0&1&-1&-1\end{array}\right]

Step 3 · Clear below the second pivot.

R_3\to R_3-R_2

wipes row 3 out completely:

\left[\begin{array}{ccc|c}1&1&1&3\\0&1&-1&-1\\0&0&0&0\end{array}\right]

This is REF. The zero row is harmless: it says

0=0

.

Step 4 · Classify. Two pivots in the first three columns, so

\operatorname{rank}(A)=2

. No pivot in the

\mathbf{b}

column, so

\operatorname{rank}([A\,|\,\mathbf{b}])=2

too. They match, so the system is consistent. But

r=2 < 3=n

, so there are infinitely many solutions, with

n-r=3-2=1 \ \text{ free variable.}

Step 5 · Go to RREF. Both pivots are already 1. Only column 2 needs cleaning above, using

R_1\to R_1-R_2

:

1-0=1

,

1-1=0

,

1-(-1)=2

,

3-(-1)=4

.

\left[\begin{array}{ccc|c}1&0&2&4\\0&1&-1&-1\\0&0&0&0\end{array}\right]

Step 6 · Name the variables. Pivots sit in columns 1 and 2, so

x

and

y

are basic. Column 3 has no pivot, so

z

is free. Set

z=t

, where

t

is any real number.

Step 7 · Read the rows. Row 1 says

x+2z=4

, so

x=4-2t

. Row 2 says

y-z=-1

, so

y=-1+t

.

Step 8 · Parametric form.

x=4-2t,\quad y=-1+t,\quad z=t,\qquad t\in\mathbb{R}.

Step 9 · Vector form. Split the constant part from the

t

part:

\begin{bmatrix}x\\y\\z\end{bmatrix} = \begin{bmatrix}4\\-1\\0\end{bmatrix} + t\begin{bmatrix}-2\\1\\1\end{bmatrix}

The first vector is a particular solution, found by setting

t=0

. The second vector solves

A\mathbf{x}=\mathbf{0}

, and it spans the null space.

Step 10 · Check in the originals, keeping $t$ symbolic.

\begin{aligned} \text{eq}_1&:\ (4-2t)+(-1+t)+t = 3+(-2t+t+t) = 3\ \checkmark\\ \text{eq}_2&:\ 2(4-2t)+3(-1+t)+t = 5+(-4t+3t+t) = 5\ \checkmark\\ \text{eq}_3&:\ 3(4-2t)+4(-1+t)+2t = 8+(-6t+4t+2t) = 8\ \checkmark \end{aligned}

Every

t

term cancels, which is exactly what "the direction lies in the null space" means.

Answer. Infinitely many solutions:

(x,y,z)=(4,-1,0)+t(-2,1,1)

for

t\in\mathbb{R}

. One free variable, since

n-r=3-2=1

.

Wait — really?

Problems 3 and 4 share the identical matrix $A$ . Only $b_3$ moved, from 10 to 8. So $\operatorname{rank}(A)=2$ in both. Changing $\mathbf{b}$ can flip a system between "no solution" and "infinitely many", but it can never change $\operatorname{rank}(A)$ , and it can never produce a unique solution here. With $r=2 < n=3$ , a unique solution was impossible from the start.

Problem 5Medium

Determine whether the system below has a solution other than $x=y=z=0$ . If it does, find every solution and give a basis for the null space of the coefficient matrix.

\begin{aligned} x + 2y + 3z &= 0\\ 2x + 5y + 7z &= 0\\ 3x + 7y + 10z &= 0 \end{aligned}

What this tests. Whether you realise consistency is automatic here, so the only real question is $r$ versus $n$ .

Show the worked solution — every step

Step 1 · Augmented matrix, with a zero column on the right.

\left[\begin{array}{ccc|c}1&2&3&0\\2&5&7&0\\3&7&10&0\end{array}\right]

That last column will stay zero through every row operation, since every operation combines zeros. So we may track only

A

and remember the

0

.

Step 2 · Clear column 1.

R_2\to R_2-2R_1

:

2-2=0

,

5-4=1

,

7-6=1

.

R_3\to R_3-3R_1

:

3-3=0

,

7-6=1

,

10-9=1

.

\left[\begin{array}{ccc|c}1&2&3&0\\0&1&1&0\\0&1&1&0\end{array}\right]

Step 3 · Clear below the second pivot.

R_3\to R_3-R_2

gives a zero row:

\left[\begin{array}{ccc|c}1&2&3&0\\0&1&1&0\\0&0&0&0\end{array}\right]

Step 4 · Classify.

\operatorname{rank}(A)=2

and

\operatorname{rank}([A\,|\,\mathbf{0}])=2

; they are equal automatically, so the system is consistent. Since

r=2 < 3=n

, there are

3-2=1

free variables, so there are infinitely many solutions. So non-trivial solutions do exist.

Step 5 · Reduce to RREF. Clear column 2 above the pivot with

R_1\to R_1-2R_2

:

1-0=1

,

2-2=0

,

3-2=1

.

\left[\begin{array}{ccc|c}1&0&1&0\\0&1&1&0\\0&0&0&0\end{array}\right]

Step 6 · Solve. Pivots in columns 1 and 2, so

x

and

y

are basic and

z

is free. Set

z=t

. Row 1 gives

x+z=0

, so

x=-t

. Row 2 gives

y+z=0

, so

y=-t

.

Step 7 · Vector form.

\begin{bmatrix}x\\y\\z\end{bmatrix} = t\begin{bmatrix}-1\\-1\\1\end{bmatrix},\qquad t\in\mathbb{R}.

There is no constant vector out front. A homogeneous system always has

\mathbf{x}_p=\mathbf{0}

, so the answer is pure null space.

Step 8 · Check. Take

t=1

, giving

(-1,-1,1)

:

-1+2(-1)+3(1)=0\ \checkmark,\qquad -2-5+7=0\ \checkmark,\qquad -3-7+10=0\ \checkmark

Step 9 · Rank–nullity check.

\operatorname{rank}(A)=2

,

\operatorname{nullity}(A)=1

, and

2+1=3=n\ \checkmark

.

Answer. Non-trivial solutions exist. Every solution is

t(-1,-1,1)

, and a basis for the null space is

\{(-1,-1,1)\}

.

Watch out

Writing the null space as " $(-1,-1,1)$ " alone is incomplete. The null space is the whole set $\{\,t(-1,-1,1) : t\in\mathbb{R}\,\}$ . The single vector is a basis for it. Also note the trivial solution is still in there: it is the case $t=0$ .

Problem 6Medium

For the matrix

A=\begin{bmatrix}1&2&3&4\\2&4&6&8\\1&3&5&7\\2&5&8&11\end{bmatrix}

find $\operatorname{rank}(A)$ . Then decide whether $A\mathbf{x}=\mathbf{0}$ has a non-trivial solution, and if so, find a basis for the null space. Verify the rank–nullity theorem.

What this tests. Rank as a property of a matrix, without any $\mathbf{b}$ attached, plus the link from rank straight to the null space.

Show the worked solution — every step

Step 1 · Look before you compute. Row 2 is exactly

2\times

row 1. Also row 1 plus row 3 is

[\,2\ 5\ 8\ 11\,]

, which is row 4. So two of the four rows are combinations of the others, and the rank should come out as 2. The elimination below confirms it.

Step 2 · Clear column 1. Three operations, all using

R_1

:

\begin{aligned} R_2\to R_2-2R_1&:\ [\,2-2,\ 4-4,\ 6-6,\ 8-8\,]=[\,0\ 0\ 0\ 0\,]\\ R_3\to R_3-R_1&:\ [\,1-1,\ 3-2,\ 5-3,\ 7-4\,]=[\,0\ 1\ 2\ 3\,]\\ R_4\to R_4-2R_1&:\ [\,2-2,\ 5-4,\ 8-6,\ 11-8\,]=[\,0\ 1\ 2\ 3\,] \end{aligned}

\begin{bmatrix}1&2&3&4\\0&0&0&0\\0&1&2&3\\0&1&2&3\end{bmatrix}

Step 3 · Move the zero row down. REF requires zero rows at the bottom, so swap

R_2\leftrightarrow R_3

:

\begin{bmatrix}1&2&3&4\\0&1&2&3\\0&0&0&0\\0&1&2&3\end{bmatrix}

Step 4 · Clear the last row.

R_4\to R_4-R_2

gives another zero row:

\begin{bmatrix}1&2&3&4\\0&1&2&3\\0&0&0&0\\0&0&0&0\end{bmatrix}

This is REF. Two nonzero rows, so two pivots, in columns 1 and 2.

Step 5 · State the rank.

\operatorname{rank}(A)=2

.

Step 6 · Decide on non-trivial solutions. Here

n=4

and

r=2

, so

n-r=2 > 0

. A homogeneous system with a rank below the number of unknowns always has non-trivial solutions. So yes.

Step 7 · Reduce to RREF. Both pivots are already 1. Clear column 2 above with

R_1\to R_1-2R_2

:

1-0=1

,

2-2=0

,

3-4=-1

,

4-6=-2

.

\begin{bmatrix}1&0&-1&-2\\0&1&2&3\\0&0&0&0\\0&0&0&0\end{bmatrix}

Step 8 · Write the equations. Pivot columns 1 and 2, so

x_1

and

x_2

are basic; columns 3 and 4 have no pivots, so

x_3

and

x_4

are free.

x_1 - x_3 - 2x_4 = 0,\qquad x_2 + 2x_3 + 3x_4 = 0.

Step 9 · Introduce parameters. Put

x_3=s

and

x_4=t

. Then

x_1 = s+2t,\qquad x_2 = -2s-3t.

Step 10 · Split into two vectors. Group the

s

terms and the

t

terms:

\begin{bmatrix}x_1\\x_2\\x_3\\x_4\end{bmatrix} = s\begin{bmatrix}1\\-2\\1\\0\end{bmatrix} + t\begin{bmatrix}2\\-3\\0\\1\end{bmatrix}

Each vector comes from setting one free variable to 1 and the other to 0. That is why the pattern of 1 and 0 appears in the bottom two slots, and it also proves the two vectors are independent.

Step 11 · Check both against a row of the original. Use row 4,

[\,2\ 5\ 8\ 11\,]

:

\begin{aligned} 2(1)+5(-2)+8(1)+11(0) &= 2-10+8 = 0\ \checkmark\\ 2(2)+5(-3)+8(0)+11(1) &= 4-15+11 = 0\ \checkmark \end{aligned}

Step 12 · Rank–nullity.

\operatorname{nullity}(A)=2

because the basis has two vectors, and

\operatorname{rank}(A)+\operatorname{nullity}(A)=2+2=4=n\ \checkmark

Answer.

\operatorname{rank}(A)=2

. Non-trivial solutions exist, and a null-space basis is

\{(1,-2,1,0),\ (2,-3,0,1)\}

, so

\operatorname{nullity}(A)=2

and

2+2=4

.

Key takeaway

Free variables and null-space basis vectors are the same thing counted twice. Set one free variable to 1 and the rest to 0, and you have read off one basis vector. Repeat for each free variable.

Problem 7Medium

Solve the system below. Identify the pivot and free variables, and write the complete solution in vector form.

\begin{aligned} x_1 + 2x_2 - x_3 + x_4 &= 1\\ 2x_1 + 4x_2 + x_3 - x_4 &= 5 \end{aligned}

What this tests. That the free variables are decided by the pivot positions, not by which letters look convenient.

Show the worked solution — every step

Step 1 · Augmented matrix. Here

m=2

and

n=4

.

\left[\begin{array}{cccc|c}1&2&-1&1&1\\2&4&1&-1&5\end{array}\right]

Step 2 · Predict the outcome first. Rank cannot exceed the number of rows, so

r\le 2 < 4=n

. A unique solution is impossible. The only two options are "no solution" or "infinitely many".

Step 3 · Clear column 1.

R_2\to R_2-2R_1

:

2-2=0,\quad 4-4=0,\quad 1-(-2)=3,\quad -1-2=-3,\quad 5-2=3.

\left[\begin{array}{cccc|c}1&2&-1&1&1\\0&0&3&-3&3\end{array}\right]

Column 2 came out entirely zero below the first row. So the second pivot is not in column 2, it is in column 3.

Step 4 · Scale the second row.

R_2\to \tfrac{1}{3}R_2

:

\left[\begin{array}{cccc|c}1&2&-1&1&1\\0&0&1&-1&1\end{array}\right]

This is REF, with pivots in columns 1 and 3.

Step 5 · Classify.

\operatorname{rank}(A)=2=\operatorname{rank}([A\,|\,\mathbf{b}])

, so the system is consistent. And

n-r=4-2=2

, so there are two free variables and infinitely many solutions.

Step 6 · Reduce to RREF. Clear column 3 above the second pivot with

R_1\to R_1+R_2

:

1+0=1,\quad 2+0=2,\quad -1+1=0,\quad 1+(-1)=0,\quad 1+1=2.

\left[\begin{array}{cccc|c}1&2&0&0&2\\0&0&1&-1&1\end{array}\right]

Step 7 · Name the variables. Pivots are in columns 1 and 3, so

x_1

and

x_3

are basic. Columns 2 and 4 have no pivot, so

x_2

and

x_4

are free. Put

x_2=s

and

x_4=t

.

Step 8 · Read the rows. Row 1 says

x_1+2x_2=2

, so

x_1=2-2s

. Row 2 says

x_3-x_4=1

, so

x_3=1+t

.

Step 9 · Vector form. Collect constants, then the

s

part, then the

t

part:

\begin{bmatrix}x_1\\x_2\\x_3\\x_4\end{bmatrix} = \begin{bmatrix}2\\0\\1\\0\end{bmatrix} + s\begin{bmatrix}-2\\1\\0\\0\end{bmatrix} + t\begin{bmatrix}0\\0\\1\\1\end{bmatrix}

Step 10 · Check with symbols kept in.

\begin{aligned} \text{eq}_1&:\ (2-2s)+2s-(1+t)+t = 2-1 = 1\ \checkmark\\ \text{eq}_2&:\ 2(2-2s)+4s+(1+t)-t = 4-4s+4s+1 = 5\ \checkmark \end{aligned}

Answer. Infinitely many solutions with two free variables:

\mathbf{x}=(2,0,1,0)+s(-2,1,0,0)+t(0,0,1,1)

.

Watch out

The tempting mistake is to declare $x_3$ and $x_4$ free "because they are the last two". Look at the RREF. The pivots landed in columns 1 and 3, so the free ones are $x_2$ and $x_4$ . Free variables are the columns without pivots, full stop.

Problem 8Hard

Consider the system

\begin{aligned} x + y + z &= 6\\ x + 2y + 3z &= 14\\ x + 2y + az &= b \end{aligned}

where $a$ and $b$ are real constants. Find all values of $a$ and $b$ for which the system has (i) a unique solution, (ii) no solution, (iii) infinitely many solutions. In case (i) give the solution in terms of $a$ and $b$ . In case (iii) give the general solution.

What this tests. Eliminating while a pivot might secretly be zero. This is the classic textbook problem, and the classic way to go wrong is to divide by $a-3$ without checking whether it is zero.

Show the worked solution — every step

Step 1 · Augmented matrix. Here

n=3

.

\left[\begin{array}{ccc|c}1&1&1&6\\1&2&3&14\\1&2&a&b\end{array}\right]

Step 2 · Clear column 1. The pivot is a plain

1

, and it does not depend on

a

or

b

, so this step is always safe.

R_2\to R_2-R_1

:

1-1=0,\quad 2-1=1,\quad 3-1=2,\quad 14-6=8.

R_3\to R_3-R_1

:

1-1=0,\quad 2-1=1,\quad a-1,\quad b-6.

\left[\begin{array}{ccc|c}1&1&1&6\\0&1&2&8\\0&1&a-1&b-6\end{array}\right]

Step 3 · Clear below the second pivot. The second pivot is

1

, again free of

a

and

b

, so

R_3\to R_3-R_2

is safe:

1-1=0,\quad (a-1)-2=a-3,\quad (b-6)-8=b-14.

\left[\begin{array}{ccc|c}1&1&1&6\\0&1&2&8\\0&0&a-3&b-14\end{array}\right]

Step 4 · Stop and look at row 3. Everything now depends on the single row

[\,0\ \ 0\ \ a-3\,|\,b-14\,],

which as an equation reads

(a-3)z=b-14

. Whether this row carries a pivot, carries a contradiction, or vanishes decides the whole problem. Do not divide by

a-3

yet.

Step 5 · Case (i): $a\neq 3$ . Then

a-3\neq 0

, so row 3 has a pivot in column 3. There are three pivots in the first three columns, giving

\operatorname{rank}(A)=3

, and none in the

\mathbf{b}

column, so

\operatorname{rank}([A\,|\,\mathbf{b}])=3

as well. With

3=3=n

, the solution is unique, whatever

b

is. Now division is legal:

z=\frac{b-14}{a-3}.

Back substitute into row 2,

y+2z=8

:

y = 8-2z = 8-\frac{2(b-14)}{a-3}.

Back substitute into row 1,

x+y+z=6

:

\begin{aligned} x &= 6-y-z\\ &= 6-\Bigl(8-\frac{2(b-14)}{a-3}\Bigr)-\frac{b-14}{a-3}\\ &= -2+\frac{2(b-14)}{a-3}-\frac{b-14}{a-3}\\ &= -2+\frac{b-14}{a-3}. \end{aligned}

Step 6 · Spot-check case (i) with numbers. Take

a=4

,

b=15

. Then

z=1/1=1

,

y=8-2=6

,

x=-2+1=-1

. Test all three originals:

-1+6+1=6\ \checkmark,\qquad -1+12+3=14\ \checkmark,\qquad -1+12+4=15\ \checkmark

Step 7 · Case (ii): $a=3$ and $b\neq 14$ . Row 3 becomes

[\,0\ 0\ 0\,|\,b-14\,]

with

b-14\neq 0

. That is the equation

0=b-14

, which is false. Column 3 has lost its pivot, so

\operatorname{rank}(A)=2

, while the

\mathbf{b}

column has gained one, so

\operatorname{rank}([A\,|\,\mathbf{b}])=3

. Since

2 < 3

, there is no solution.

Step 8 · Case (iii): $a=3$ and $b=14$ . Row 3 becomes

[\,0\ 0\ 0\,|\,0\,]

, which is harmless. Now

\operatorname{rank}(A)=\operatorname{rank}([A\,|\,\mathbf{b}])=2

, and

2 < 3=n

, so there are infinitely many solutions with

3-2=1

free variable.

Step 9 · Solve case (iii). The matrix is

\left[\begin{array}{ccc|c}1&1&1&6\\0&1&2&8\\0&0&0&0\end{array}\right] \xrightarrow{\;R_1\to R_1-R_2\;} \left[\begin{array}{ccc|c}1&0&-1&-2\\0&1&2&8\\0&0&0&0\end{array}\right]

Pivots in columns 1 and 2, so

z

is free. Put

z=t

. Row 1 gives

x-z=-2

, so

x=-2+t

. Row 2 gives

y+2z=8

, so

y=8-2t

.

\begin{bmatrix}x\\y\\z\end{bmatrix} = \begin{bmatrix}-2\\8\\0\end{bmatrix} + t\begin{bmatrix}1\\-2\\1\end{bmatrix}

Step 10 · Check case (iii) symbolically. With

a=3

,

b=14

the third equation is identical to the second, so only two need checking:

\begin{aligned} (-2+t)+(8-2t)+t &= 6+(t-2t+t) = 6\ \checkmark\\ (-2+t)+2(8-2t)+3t &= 14+(t-4t+3t) = 14\ \checkmark \end{aligned}

Answer. Unique when

a\neq 3

(any

b

), with

z=\dfrac{b-14}{a-3}

,

y=8-\dfrac{2(b-14)}{a-3}

,

x=-2+\dfrac{b-14}{a-3}

. No solution when

a=3

,

b\neq 14

. Infinitely many when

a=3

,

b=14

, namely

(x,y,z)=(-2,8,0)+t(1,-2,1)

.

Watch out

Note which parameter controls which switch. Here $a$ alone decides $\operatorname{rank}(A)$ , because $a$ lives on the left of the bar. Then $b$ decides, only in the knife-edge case $a=3$ , whether the system is consistent. That division of labour is typical: parameters inside $A$ control the rank, and parameters inside $\mathbf{b}$ control consistency.

Wait — really?

At $a=3$ the answer does not change gradually, it changes character. For $a=3.001$ you get one point. At $a=3$ exactly you get either nothing at all or an entire line, depending on $b$ . And look at the unique solution as $a\to 3$ with $b\neq 14$ : the fraction $\tfrac{b-14}{a-3}$ blows up. The point runs off to infinity, and in the limit there is nowhere for it to be.

Problem 9Hard

Solve

\begin{aligned} x_1 + 2x_2 - x_3 + 3x_4 + x_5 &= 3\\ 2x_1 + 4x_2 - x_3 + 4x_4 + 3x_5 &= 5\\ 3x_1 + 6x_2 - 2x_3 + 7x_4 + 4x_5 &= 8\\ x_1 + 2x_2 + x_3 + 5x_5 &= 2 \end{aligned}

Give $\operatorname{rank}(A)$ , $\operatorname{rank}([A\,|\,\mathbf{b}])$ , the number of free variables, the general solution in vector form, and a basis for the null space of $A$ .

What this tests. A full-size elimination where one equation is redundant, the pivots skip columns, and the answer is a two-parameter family. Note the missing $x_4$ term in the last equation: it enters the matrix as a zero.

Show the worked solution — every step

Part 1 · Step 1 · Augmented matrix. Here

m=4

and

n=5

.

\left[\begin{array}{ccccc|c}1&2&-1&3&1&3\\2&4&-1&4&3&5\\3&6&-2&7&4&8\\1&2&1&0&5&2\end{array}\right]

Part 1 · Step 2 · Clear column 1. Three operations, all against

R_1

:

\begin{aligned} R_2\to R_2-2R_1&:\ [\,0,\ 0,\ -1+2,\ 4-6,\ 3-2\,|\,5-6\,]\\ &\ \ =[\,0,\ 0,\ 1,\ -2,\ 1\,|\,-1\,]\\ R_3\to R_3-3R_1&:\ [\,0,\ 0,\ -2+3,\ 7-9,\ 4-3\,|\,8-9\,]\\ &\ \ =[\,0,\ 0,\ 1,\ -2,\ 1\,|\,-1\,]\\ R_4\to R_4-R_1&:\ [\,0,\ 0,\ 1+1,\ 0-3,\ 5-1\,|\,2-3\,]\\ &\ \ =[\,0,\ 0,\ 2,\ -3,\ 4\,|\,-1\,] \end{aligned}

\left[\begin{array}{ccccc|c}1&2&-1&3&1&3\\0&0&1&-2&1&-1\\0&0&1&-2&1&-1\\0&0&2&-3&4&-1\end{array}\right]

Column 2 is now entirely zero below row 1. So no pivot can sit in column 2, and the next pivot is in column 3.

Part 1 · Step 3 · Clear below the second pivot, in column 3. Two operations against

R_2

:

\begin{aligned} R_3\to R_3-R_2&:\ [\,0,\ 0,\ 0,\ 0,\ 0\,|\,0\,]\\ R_4\to R_4-2R_2&:\ [\,0,\ 0,\ 0,\ -3+4,\ 4-2\,|\,-1+2\,]=[\,0,\ 0,\ 0,\ 1,\ 2\,|\,1\,] \end{aligned}

\left[\begin{array}{ccccc|c}1&2&-1&3&1&3\\0&0&1&-2&1&-1\\0&0&0&0&0&0\\0&0&0&1&2&1\end{array}\right]

Part 1 · Step 4 · Move the zero row to the bottom. REF demands it, so

R_3\leftrightarrow R_4

:

\left[\begin{array}{ccccc|c}1&2&-1&3&1&3\\0&0&1&-2&1&-1\\0&0&0&1&2&1\\0&0&0&0&0&0\end{array}\right]

This is REF. Pivots sit in columns 1, 3, 4.

Part 1 · Step 5 · Read the ranks and classify. Three pivots inside the first five columns, so

\operatorname{rank}(A)=3

. No pivot in the

\mathbf{b}

column, so

\operatorname{rank}([A\,|\,\mathbf{b}])=3

too. They match, so the system is consistent. And

n-r=5-3=2,

so there are two free variables and infinitely many solutions.

Part 1 · Step 6 · Where the redundancy was. Row 3 of the original equals row 1 plus row 2, on both sides:

3+5=8

matches. So the third equation carried no new information, which is why a zero row appeared.

Part 2 · Step 1 · Clear column 4, the rightmost pivot column, first. The pivot is the

1

in row 3. Rows 1 and 2 have entries

3

and

-2

above it.

\begin{aligned} R_2\to R_2+2R_3&:\ [\,0,\ 0,\ 1,\ -2+2,\ 1+4\,|\,-1+2\,]=[\,0,\ 0,\ 1,\ 0,\ 5\,|\,1\,]\\ R_1\to R_1-3R_3&:\ [\,1,\ 2,\ -1,\ 3-3,\ 1-6\,|\,3-3\,]=[\,1,\ 2,\ -1,\ 0,\ -5\,|\,0\,] \end{aligned}

\left[\begin{array}{ccccc|c}1&2&-1&0&-5&0\\0&0&1&0&5&1\\0&0&0&1&2&1\\0&0&0&0&0&0\end{array}\right]

Part 2 · Step 2 · Now clear column 3. Row 1 has

-1

above that pivot, so

R_1\to R_1+R_2

:

[\,1,\ 2,\ -1+1,\ 0,\ -5+5\,|\,0+1\,]=[\,1,\ 2,\ 0,\ 0,\ 0\,|\,1\,]

\left[\begin{array}{ccccc|c}1&2&0&0&0&1\\0&0&1&0&5&1\\0&0&0&1&2&1\\0&0&0&0&0&0\end{array}\right]

This is the RREF. Column 1 needs nothing, since its pivot already sits alone.

Part 2 · Step 3 · Name the variables. Pivot columns are 1, 3, 4, so

x_1, x_3, x_4

are basic. Columns 2 and 5 have no pivot, so

x_2

and

x_5

are free. Put

x_2=s

and

x_5=t

.

Part 2 · Step 4 · Read one row at a time.

\begin{aligned} \text{Row 1}&:\ x_1+2x_2=1 &\Rightarrow\quad x_1&=1-2s\\ \text{Row 2}&:\ x_3+5x_5=1 &\Rightarrow\quad x_3&=1-5t\\ \text{Row 3}&:\ x_4+2x_5=1 &\Rightarrow\quad x_4&=1-2t \end{aligned}

Part 2 · Step 5 · Assemble the vector form.

\begin{bmatrix}x_1\\x_2\\x_3\\x_4\\x_5\end{bmatrix} = \begin{bmatrix}1\\0\\1\\1\\0\end{bmatrix} + s\begin{bmatrix}-2\\1\\0\\0\\0\end{bmatrix} + t\begin{bmatrix}0\\0\\-5\\-2\\1\end{bmatrix}

Part 2 · Step 6 · Identify the three pieces. Setting

s=t=0

gives the particular solution

\mathbf{x}_p=(1,0,1,1,0)

. The two vectors multiplying

s

and

t

solve

A\mathbf{x}=\mathbf{0}

, so they form a basis of the null space.

Part 2 · Step 7 · Check the particular solution in all four originals.

\begin{aligned} 1+0-1+3+0 &= 3\ \checkmark & 2+0-1+4+0 &= 5\ \checkmark\\ 3+0-2+7+0 &= 8\ \checkmark & 1+0+1+0+0 &= 2\ \checkmark \end{aligned}

Part 2 · Step 8 · Check a general point. Take

s=t=1

, giving

\mathbf{x}=(-1,1,-4,-1,1)

:

\begin{aligned} \text{eq}_1&:\ -1+2+4-3+1 = 3\ \checkmark\\ \text{eq}_2&:\ -2+4+4-4+3 = 5\ \checkmark\\ \text{eq}_3&:\ -3+6+8-7+4 = 8\ \checkmark\\ \text{eq}_4&:\ -1+2-4+0+5 = 2\ \checkmark \end{aligned}

Part 2 · Step 9 · Rank–nullity.

\operatorname{rank}(A)+\operatorname{nullity}(A)=3+2=5=n\ \checkmark

.

Answer.

\operatorname{rank}(A)=\operatorname{rank}([A\,|\,\mathbf{b}])=3

, two free variables, and

\mathbf{x}=(1,0,1,1,0)+s(-2,1,0,0,0)+t(0,0,-5,-2,1)

. Null-space basis:

\{(-2,1,0,0,0),\ (0,0,-5,-2,1)\}

.

Watch out

Four equations did not give four pivots. Do not assume $\operatorname{rank}(A)=m$ . Here one equation was the sum of two others, so the rank was 3, not 4. Equally, do not assume the free variables are the last ones: they are $x_2$ and $x_5$ .

Problem 10Hard

Let

A=\begin{bmatrix}1&2&3\\2&5&8\\1&3&5\end{bmatrix},\qquad \mathbf{b}=\begin{bmatrix}b_1\\b_2\\b_3\end{bmatrix}.

Find $\operatorname{rank}(A)$ . Then find the exact condition on $b_1, b_2, b_3$ under which $A\mathbf{x}=\mathbf{b}$ is consistent. When the condition holds, give the general solution in terms of $b_1, b_2, b_3$ . Finally, verify your condition on $\mathbf{b}=(1,3,2)$ .

What this tests. Running the elimination with unknown symbols on the right-hand side. This is the reverse of Problem 8: there the left side carried the parameters, here the right side does.

Show the worked solution — every step

Step 1 · Augmented matrix, with symbols on the right.

\left[\begin{array}{ccc|c}1&2&3&b_1\\2&5&8&b_2\\1&3&5&b_3\end{array}\right]

Treat

b_1, b_2, b_3

as ordinary numbers whose values you do not know. Every row operation acts on them just like on any other entry.

Step 2 · Clear column 1.

R_2\to R_2-2R_1

:

2-2=0,\quad 5-4=1,\quad 8-6=2,\quad b_2-2b_1.

R_3\to R_3-R_1

:

1-1=0,\quad 3-2=1,\quad 5-3=2,\quad b_3-b_1.

\left[\begin{array}{ccc|c}1&2&3&b_1\\0&1&2&b_2-2b_1\\0&1&2&b_3-b_1\end{array}\right]

Step 3 · Clear below the second pivot.

R_3\to R_3-R_2

. The left side vanishes completely, and the right side becomes

(b_3-b_1)-(b_2-2b_1) = b_3-b_1-b_2+2b_1 = b_1-b_2+b_3.

\left[\begin{array}{ccc|c}1&2&3&b_1\\0&1&2&b_2-2b_1\\0&0&0&b_1-b_2+b_3\end{array}\right]

Step 4 · Read the rank of $A$ . Only two pivots survive on the left, in columns 1 and 2, so

\operatorname{rank}(A)=2

, whatever

\mathbf{b}

is. Column 3 has no pivot, so

z

will be free.

Step 5 · Impose consistency. Row 3 reads

0=b_1-b_2+b_3

. If the right side is nonzero, that row carries a pivot in the

\mathbf{b}

column, making

\operatorname{rank}([A\,|\,\mathbf{b}])=3 > 2=\operatorname{rank}(A)

, and the system is dead. So the system is consistent exactly when

b_1-b_2+b_3=0

Step 6 · Note there is no third case. When the condition holds,

\operatorname{rank}(A)=\operatorname{rank}([A\,|\,\mathbf{b}])=2 < 3=n

, so the answer is infinitely many solutions with one free variable. A unique solution can never occur, for any

\mathbf{b}

, because

\operatorname{rank}(A)

is fixed at 2 and can never reach

n=3

.

Step 7 · Reduce to RREF, assuming the condition holds.

R_1\to R_1-2R_2

:

1,\quad 2-2=0,\quad 3-4=-1,\quad b_1-2(b_2-2b_1)=5b_1-2b_2.

\left[\begin{array}{ccc|c}1&0&-1&5b_1-2b_2\\0&1&2&b_2-2b_1\\0&0&0&0\end{array}\right]

Step 8 · Write the general solution. Set

z=t

. Row 1 gives

x-z=5b_1-2b_2

, and row 2 gives

y+2z=b_2-2b_1

:

\begin{bmatrix}x\\y\\z\end{bmatrix} = \begin{bmatrix}5b_1-2b_2\\b_2-2b_1\\0\end{bmatrix} + t\begin{bmatrix}1\\-2\\1\end{bmatrix}

The direction vector

(1,-2,1)

carries no

b

at all. That is expected: the null space belongs to

A

alone, so changing

\mathbf{b}

slides the solution line around, but never turns it.

Step 9 · Verify with $\mathbf{b}=(1,3,2)$ . Check the condition first:

1-3+2=0\ \checkmark

, so it should be consistent. The formula gives

x=5(1)-2(3)+t=-1+t,\qquad y=3-2(1)-2t=1-2t,\qquad z=t.

Test at

t=0

, so

(x,y,z)=(-1,1,0)

:

-1+2+0=1\ \checkmark,\qquad -2+5+0=3\ \checkmark,\qquad -1+3+0=2\ \checkmark

Test at

t=1

, so

(x,y,z)=(0,-1,1)

:

0-2+3=1\ \checkmark,\qquad 0-5+8=3\ \checkmark,\qquad 0-3+5=2\ \checkmark

Step 10 · Verify a failing case too. Take

\mathbf{b}=(1,3,5)

, where

1-3+5=3\neq 0

. Row 3 becomes

[\,0\ 0\ 0\,|\,3\,]

, so the system has no solution, as the condition predicted.

The picture behind the condition. Since

\operatorname{rank}(A)=2

, the three columns of

A

span only a plane inside

\mathbb{R}^3

, not the whole space. And

A\mathbf{x}

is a combination of those columns, so

A\mathbf{x}

can only ever land on that plane. The system is solvable exactly when

\mathbf{b}

already lies there. The vector

(1,-1,1)

is the normal to the plane, which is why the condition reads

b_1-b_2+b_3=0

. You can see it directly: each column of

A

dotted with

(1,-1,1)

gives

1-2+1=0

,

2-5+3=0

and

3-8+5=0

.

Answer.

\operatorname{rank}(A)=2

. Consistent exactly when

b_1-b_2+b_3=0

, and then

(x,y,z)=(5b_1-2b_2,\ b_2-2b_1,\ 0)+t(1,-2,1)

. Never unique, for any

\mathbf{b}

.

Key takeaway

A parameter on the left of the bar controls $\operatorname{rank}(A)$ and so the number of solutions. A parameter on the right of the bar controls only whether any solution exists.

Next up

Unit 2 · Vector Spaces →

You can now solve $A\mathbf{x}=\mathbf{b}$ and read off its three fates. Unit 2 asks the question underneath: what kind of space are these solutions living in? Groups, subspaces, span, independence, basis, dimension — the architecture every later unit is built on.

← All units