A network with random weights is useless — it outputs noise. Learning is the process of finding weights that make its outputs right. With billions of weights and no human in the loop, how could that possibly work? The answer is one of the most important ideas in all of machine learning, and it is genuinely simple.
Step 1: measure how wrong you are
Show the network an example whose correct answer you already know. It produces a
prediction. Compare the prediction to the truth and score the gap with a loss
function — a single number that is large when the network is badly wrong and
zero when it is perfect. For a network trying to predict a value, a common loss
is the squared error, (prediction − truth)².
Now training has a crisp goal: make the loss as small as possible by changing the weights.
Step 2: roll downhill
Think of the loss as a landscape. Every possible setting of the weights is a point on that landscape, and its height is the loss there. Training is just walking downhill to a valley.
But you cannot see the whole landscape — it lives in a billion dimensions. You can only feel the slope under your feet: for each weight, which way, and how steeply, does the loss change if you nudge it? That slope is the gradient, and calculus gives it to you exactly. The recipe — gradient descent — is:
for each weight:
weight ← weight − learningRate × (slope of loss w.r.t. that weight)
Subtracting the slope moves you against the incline, i.e. downhill. Repeat over many examples and the loss falls, step by step, until it flattens out. That flat spot is a set of weights that works.
The learning rate is the whole trick
The learningRate controls how big each step is. Too small and training crawls;
too large and you leap clean over the valley and bounce out — the loss grows
instead of shrinking. Getting it right is most of the art of training.
Below is gradient descent on a simple one-weight loss, f(x) = x². The ball
starts up the slope; each Step applies the update above. Try a small learning
rate and watch it inch down. Then crank the rate up past 1.0 and watch it
overshoot and diverge — exactly the failure that wrecks real training runs.
That's the engine
A real network runs this same loop over many layers at once — the gradient is propagated backwards through the network (an algorithm called backpropagation, a topic for the Core level). But the heart of it is what you just saw: score the error, follow the slope down, repeat. Every model in this track, up to the largest language models, is trained by this one idea at staggering scale.