In an agentic session, every turn sends the entire accumulated context — system prompt, tool definitions, all prior messages, all prior thinking blocks — as fresh input. There is no free memory.
By Pawan Nayar · from the LLOS AI Learning record
In an agentic session, every turn sends the entire accumulated context — system prompt, tool definitions, all prior messages, all prior thinking blocks — as fresh input.
There is no free memory.
An agentic framework like Claude Code holds conversation state, but the API has no concept of already-paid-for. Each turn packages the full history and sends it as input. Turn 1 might be 2,000 tokens. Turn 20 might be 80,000 tokens — even if nothing was added except short exchanges. The longer a session runs without a reset, the higher the per-turn cost climbs. Total session cost grows roughly as the square of the number of turns, not linearly.
A 30-turn session where each turn adds ~3,000 tokens averages 45,000 input tokens per turn — not 3,000. The session costs 15× more than a developer expecting simple per-turn pricing would predict.
Resolved from this lesson's own record — real definitions, not invented for this page.
More from the same record, pulled by shared topic — not written new for this page.
When a user tells an AI model to use a 'gold file', the AI model often just hears 'important reference'. The AI model then gives a confident answer.
The API does not discard thinking blocks after a turn ends — it keeps them as part of the conversation history. Every new turn re-bills all prior thinking as input tokens.
When Claude Code hit the 200K context limit, it compacted the history — summarising and discarding old turns. This was friction, but it was also a cost governor.
When an AI model's session ends, all past conversations are gone. Decisions made, problems found, and solutions tried are lost.
In an agentic session, every turn sends the entire accumulated context — system prompt, tool definitions, all prior messages, all prior thinking blocks — as fresh input. There is no free memory.