A model can only vary what it has seen varied. Give it one example of a good answer and it copies that example's shape, because one example is the only evidence it has about what the shape should be. Give it eight and the range becomes the pattern. Reasoning does not fix this and cannot: the model is not confused about the task, it is copying the single sample it was shown. Fix the prompt, not the model.
By Pawan Nayar · from the LLOS AI Learning record
A model can only vary what it has seen varied.
Give it one example of a good answer and it copies that example's shape, because one example is the only evidence it has about what the shape should be. Give it eight and the range becomes the pattern. Reasoning does not fix this and cannot: the model is not confused about the task, it is copying the single sample it was shown. Fix the prompt, not the model.
Because the prompt showed one shape. A model reads an example as a specification. One example is a specification with no room in it, so every answer lands in the same place - and the more capable the model, the more faithfully it copies. Adding reasoning makes this worse rather than better. Reasoning with nothing to check against does one of two things: it over-reads the single sample and spends its whole output budget deciding, or it strips the task down to the one pattern it can see and repeats that. Neither is a model problem. Both are a prompt with one data point in it. The fix costs nothing. Put several real examples in the prompt, drawn from work already approved, and name the failure you do not want. Input grows; output does not. Input is priced far below output on every model worth using, so a bigger prompt is the cheapest change available - and a single re-run costs more than enriching the prompt eight times over. Two boundaries. Good and bad examples are not symmetric: a bare bad example is sometimes copied, because the model sees a shape and reproduces it whatever the label says. Show the weak version beside the version that replaced it, and state the difference in one line. And reasoning is not always waste - it earns its cost wherever the work can be checked, by a test that passes, a number that reconciles, a bug that reproduces. Where nothing can be checked, reasoning is expensive guessing.
Twelve clusters of lessons, two models, asked for a title and opening paragraph each. Round one carried one approved example in the prompt. The cheaper model opened ten of twelve titles with How, What or Why. The stronger model opened eleven of twelve paragraphs with the same three words. Twenty-one near-identical openings out of twenty-four. Round two changed the prompt only - eight real openings taken from already-published articles, plus one line naming the repetition to avoid. Same models, same material, same day. Formula openings dropped to zero out of twenty-four. The cost moved the other way from expectation. Input rose 45 per cent, output rose 9, and the run went from 4.4 cents to 5.3. But round one's output was unusable, so its true price was two runs at 8.9 cents. The richer prompt was 40 per cent cheaper. The same run showed the reasoning half. With reasoning left on, the cheaper model spent 12,801 output tokens and returned eight empty replies out of twelve - reasoning consumed the output allowance before a single word was written, and every failure came back as a success with an empty message attached. Reasoning switched off: zero reasoning tokens, twelve replies out of twelve, half the cost.
Resolved from this lesson's own record — real definitions, not invented for this page.
More from the same record, pulled by shared topic — not written new for this page.
A developer should list all known problems. This includes bugs, which are errors in the code.
An AI model should first decide on the correct answer. After that, the model should write the rules that lead to this answer.
Adding more rules to a prompt does not create variety. The AI model will follow the new rules, making the output rigid in a different way.
A developer can improve an AI model's output by changing the instructions. Instead of general commands, the developer should provide the full structure of all possible output types.
A model can only vary what it has seen varied. Give it one example of a good answer and it copies that example's shape, because one example is the only evidence it has about what the shape should be. Give it eight and the range becomes the pattern. Reasoning does not fix this and cannot: the model is not confused about the task, it is copying the single sample it was shown. Fix the prompt, not the model.