A developer must make only one call to an external API. The developer must not retry the call if it fails. Retrying can cause an IP address to be banned.
By Pawan Nayar · from the LLOS AI Learning record
A developer must make only one call to an external API.
The developer must not retry the call if it fails. Retrying can cause an IP address to be banned.
When a developer's code calls an external API, it should try only one time. The developer must not use loops to retry the call. The developer must not use exponential backoff. The developer must not retry even for small errors. If the call fails, the developer's code must return the error. It should not try again in the background. It should not put the failed call back in a queue to try later. If many parts of a system call the same API, they need one central system to manage calls. Each part trying on its own can send too many requests. This can cause the system's IP address to be blocked by the API provider.
A developer creates a new feature that uses an external AI model. The developer adds code to retry the API call three times if it fails. Another developer also adds a script that calls the same API and includes retries. When the external API has a small problem, both the feature and the script start retrying many times. This sends a large burst of requests to the AI model. The AI model's security system blocks the project's IP address temporarily. No one can use the AI model for a while.
Resolved from this lesson's own record — real definitions, not invented for this page.
More from the same record, pulled by shared topic — not written new for this page.
An AI model should try to call an external API only one time. This rule applies to APIs like Gemini or OpenAI.
A developer should test specific error scenarios. First, turn off the server and try to access the application.
Sending many requests to an API server all at once can cause problems. This is like a 'thundering herd' of requests.
A developer should start with 8-10 workers and a 500ms delay between calls. The developer watches the failure rate.
A developer must make only one call to an external API. The developer must not retry the call if it fails. Retrying can cause an IP address to be banned.