Every tool that asks an LLM to produce something will eventually get back something else. These eight rules are what each of those failures cost.
By Pawan Nayar · from the LLOS AI Learning record
Every tool that asks an LLM to produce something will eventually get back something else.
These eight rules are what each of those failures cost.
1. **Constrain output at the API, not in the prompt.** "Respond with ONLY the SVG" in a prompt is a request; the model can ignore it. Prefilling the assistant turn (e.g. starting it with `<svg`) is a guarantee — the model physically cannot write a preamble. Prefer guarantees. 2. **Validate before you display or save.** A salvager that just greps for a tag (`<svg`) will happily accept an essay that *mentions* the tag. Real output must pass a shape test (e.g. ≥2 drawing elements, almost no bare text outside tags). Fail the test → failed generation, nothing shown, nothing saved. 3. **The model's raw words never reach the screen. Ever.** Not in the result, not in a debug "peek", not in an error message. Even a 140-char excerpt is a leak. 4. **User-facing errors are a whitelist, not a passthrough.** Only messages a human wrote may render. API JSON, network exceptions, stack traces — all collapse to one clean line ("The drawing timed out." / "The studio hit a snag."). Mark hand-written errors explicitly (e.g. `err.friendly = true`) and let everything unmarked fall to the generic line. 5. **What is shown must equal what is saved.** Client and server run the SAME validation (mirrored functions, cross-referenced in comments). If they drift, you get stickers that display but never save — or garbage that saves but never displayed. 6. **After a bad incident, sweep the stored data.** A fixed validator protects the future; the past is still sitting in the gallery/DB. Dry-run scan → report → backup → remove. 7. **Every failure path ends in one of a known, finite set of outcomes.** For Sticker Studio: the SVG, "cut off", "didn't come out", "timed out", "hit a snag", "something went wrong", or the friendly cap message. If you cannot enumerate your tool's outcomes, it has a leak. 8. **Sweep a slider before you style it.** A parameter that is *paid for* has an optimum you can find by sweeping it; a parameter that is nearly *free* moves the result without improving it. You cannot tell which by looking at the output — only by sweeping. Paid parameters earn a control and visual weight; free ones are decoration pretending to be control. This sharpens the UI-volume rule: frequency sets a control's budget, but whether the parameter is paid for decides whether it earns a control at all. And never set a *floor* on a parameter that has no valley — with no optimum to protect, a floor is only your guess imposed on the user.
Resolved from this lesson's own record — real definitions, not invented for this page.
More from the same record, pulled by shared topic — not written new for this page.
**What was seen** TextStock's "fill depth" (how many words are packed into the shape) could not go below 1000. Every stock came out dense; a sparse, readable one was unreachable.
**What was seen** A sticker generation returned no sticker. Instead the page displayed the model's full planning essay ("Let me think about the animation more carefully…"), hundreds of words of internal reasoning, ending with the metadata toast (claude-sonnet-4-6 · 773 in / 4,096 out tok · 77.6s).
When an AI model cannot follow all user settings, it has a 'constraint failure'. This means the settings conflict with each other or with what the AI model can do.
After any content generation or build, the AI model must check the output. This check goes beyond just seeing if the script finished.
Every tool that asks an LLM to produce something will eventually get back something else. These eight rules are what each of those failures cost.