Before making an AI model API call that creates a structured output, estimate the output size. If the estimated size is too large, divide the task into smaller parts. Make separate API calls for each small part and then combine the results.
By Pawan Nayar · from the LLOS AI Learning record
Before making an AI model API call that creates a structured output, estimate the output size.
If the estimated size is too large, divide the task into smaller parts. Make separate API calls for each small part and then combine the results.
A developer must estimate the token count of the AI model's output before making an API call. If the estimated output is more than 6000 tokens, the developer should split the request. The developer can make multiple smaller API calls. Each call should generate a part of the final output. Then, the developer combines these parts into one complete output. This prevents the AI model from stopping early and cutting off information.
A developer asks an AI model to create a JSON list of 100 idioms. Each idiom entry is about 150 tokens. The total estimated output is 15,000 tokens. This is much larger than 6000 tokens. The developer splits the request into three parts. The developer asks for 34 idioms in the first call, 33 in the second, and 33 in the third. Each call returns a complete list. The developer then combines these three lists into one final list of 100 idioms.
Resolved from this lesson's own record — real definitions, not invented for this page.
More from the same record, pulled by shared topic — not written new for this page.
A developer must read specific API policy files before writing any code that makes external API calls. This prevents common mistakes.
An AI model should check each part of its output. For example, if there is a list of key phrases, the AI model must confirm each phrase is in the main text.
A developer tells the AI model to use JSON. The developer sets `response_mime_type` to `application/json`.
An AI model must state the full path every time it creates or changes a file. The AI model must list all full paths at the end of a conversation.
Before making an AI model API call that creates a structured output, estimate the output size. If the estimated size is too large, divide the task into smaller parts. Make separate API calls for each small part and then combine the results.