In the Foundations track you saw gradient descent nudge a single weight downhill. A real network has millions or billions of weights across many layers, and gradient descent needs the slope of the loss for every one of them. Computing those slopes naïvely — perturb a weight, re-run the whole network, measure the change, repeat — would take one full forward pass per weight. That is hopeless.
Backpropagation computes them all at once. It is the algorithm that makes deep learning practical.
Blame, flowing backwards
The intuition is assigning blame. The loss at the end tells you how wrong the final output was. Backprop pushes that blame backwards: the output layer's weights are most directly responsible, so it works out their share first; then it passes the remaining blame to the layer before, and so on, back to the first layer. Each layer answers one local question: given how much my output was to blame, how much was each of my weights to blame?
The chain rule, reused
Formally, backprop is the chain rule of calculus applied layer by layer. The loss depends on the output, which depends on layer 2, which depends on layer 1. The chain rule says you can multiply these local sensitivities together to get the sensitivity of the loss to any early weight.
The efficiency trick is that the forward pass already computed and stored each layer's outputs. Backprop walks the layers in reverse, and at each one it combines the gradient arriving from the layer ahead with the values it saved on the way forward. Nothing is recomputed from scratch. The result: all the weight gradients for roughly the cost of one extra pass through the network — not one pass per weight.
Why it matters
Backpropagation is exact (it is real calculus, not an approximation) and cheap (one backward sweep). Those two properties together are why we can train models with hundreds of billions of weights at all. Every network in this track is trained by the same loop you now understand end to end:
- Forward pass — run the input through to a prediction.
- Loss — score how wrong it was.
- Backward pass (backprop) — get the gradient for every weight.
- Update — nudge each weight against its gradient (gradient descent).
- Repeat on the next batch of examples.
With the engine of learning understood, the rest of the track turns to a different question: how do you feed messy real-world things — especially language — into a network in the first place? That starts with turning words into vectors.
