The Batch API is for very large tasks where waiting is acceptable. A developer must confirm the file is linked correctly after submission. A real-time script should be ready as a backup.
By Pawan Nayar · from the LLOS AI Learning record
The Batch API is for very large tasks where waiting is acceptable.
A developer must confirm the file is linked correctly after submission. A real-time script should be ready as a backup.
The Batch API is best for tasks with more than 50,000 calls. The cost savings must be worth a 24 to 48-hour wait time. There should be no strict deadline for the task. A developer must check the job status for `src: None`. This means the file is not linked correctly. A developer should always have a real-time script ready. This script can be used if the Batch API fails or is too slow.
A developer uses the Batch API for 100,000 requests. The job is stuck in PENDING for 36 hours. The status shows `src: None`. This means the input file was not linked properly. The developer must fix the link or use a real-time script to complete the task.
More from the same record, pulled by shared topic — not written new for this page.
An AI model should try to call an external API only one time. This rule applies to APIs like Gemini or OpenAI.
A developer should design batch scripts to resume by default. This prevents lost work if the script stops.
A developer should start with 8-10 workers and a 500ms delay between calls. The developer watches the failure rate.
When a developer's code calls an external API, it should try only one time. The developer must not use loops to retry the call.
The Batch API is for very large tasks where waiting is acceptable. A developer must confirm the file is linked correctly after submission. A real-time script should be ready as a backup.