Training changes the model.
θ ← θ − η∇θL
Gradient descent is literal during training: optimization changes model parameters θ to reduce a training loss.
01 / LET’S WORK THIS OUT
More thinking.
Better answer… right?
Sometimes! Let’s follow a little dot through the fog
and see when thinking harder actually helps.
YOUR TURN AT THE BOARD
ESTIMATED LOSS SURFACE
Descent pathFog = what remains unobserved
CURRENT ESTIMATE0.246 loss
LOCAL SLOPE0.267 ‖∇L̂‖
IMPROVEMENT FROM START1.104 loss reduced
The local slope still gives a direction to follow.
SAME TERRAIN / A DIFFERENT VIEW
Let’s cut through the hill.
Same dot. A simpler picture.
A slice through the surface above, at y = 0. The dot shares the explorer’s x coordinate. A flat slope says “no local improvement.” It does not certify the best answer.
GRADIENT DESCENT
xt+1 = xt − η ∇L̂(xt)
- xt
- Your current candidate. Here, a point (x, y).
- ∇L̂
- The slope of your estimated loss. The minus sign points downhill.
- η
- Step size, fixed here at 0.15. It is separate from the number of steps.
NEXT x: -0.894 − 0.15 × -0.051 = -0.886
The hat in L̂ means “estimate.” The world may grade your answer with a different function.
02 / WHERE THE ANALOGY HOLDS
Okay, but is this how
an AI actually thinks?
Reasoning extends the context.
Pθ(a | p, r)
During ordinary inference, θ stays fixed. Prompt p and intermediate reasoning r condition the answer a. More compute may support checking, revision, or search.
Evidence tests the map.
P(h | e) ∝ P(e | h) P(h)
A measurement, a test, or a reliable source can discriminate between hypotheses h. Repeating a belief is not an independent observation e.
Here’s where I’d put a big circle on the board: the terrain is a teaching model of search, not a picture of an LLM’s internal computation. “High reasoning” is neither a thinner pen nor a guaranteed wider search. More reasoning need not sharpen confidence; it can also reveal uncertainty.
03 / CONFIDENCE IS NOT CORRECTNESS
Being sure feels good.
Being able to update is better.
A prompt can favor one interpretation. That preference is not proof. Suppose hypothesis A initially gets more weight than B. A test is eight times more likely under B than A.
PRIOR / BEFORE EVIDENCE
P(A | e) = 0.1 × P(A)0.1 × P(A) + 0.8 × P(B)
Toy likelihoods: P(e | A) = 0.1; P(e | B) = 0.8. These bars are explicit beliefs over two hypotheses, not token probabilities or measured model confidence.
04 / SO, WHY EVER USE LESS REASONING?
My rule of thumb:
make the next step count.
Clear task, easy check
Start with less. If a short answer passes a reliable check, further reasoning may add latency and cost with little benefit.
Hard task, useful feedback
Spend more. Compare approaches, test intermediate results, and revise. Extra computation has something useful to work with.
Missing fact, shaky premise
Observe first. Get the missing evidence or clarify the goal. More local work cannot guarantee that your assumptions are right.
VALUE OF ANOTHER STEP
Expected improvement > cost of time + compute
A decision rule, not a universal numerical threshold. Higher stakes raise the value of reliable checks.