When concurrency is reached
Generation requests above your concurrency limit are still accepted. The API returns arequest_id with status queued, and the request starts automatically when one of your running requests reaches a terminal status.
When the queue is full
The number of generations waiting to start is limited too. The queue limit is a multiple of your account’s concurrency limit, and each output of a batch request counts separately. When the queue is full, a new generation request is rejected with429 Too Many Requests and a Retry-After header, and nothing is created or charged:
limit is the queue limit for your account. Wait for Retry-After seconds and submit again with the same Idempotency-Key; queued requests leave the queue as soon as they start.
If queue admission is briefly unavailable, the API returns 503 Service Unavailable with "code": "admission_control_unavailable" and "retryable": true. Retry with backoff.
Endpoint-specific limits
Some non-generation endpoints reject requests above their limit instead of queuing them. They return429 Too Many Requests with a Retry-After header:
Client recommendations
- On
429, wait for the number of seconds inRetry-Afterbefore submitting again. - Limit generation submissions with a worker pool or semaphore so the queue stays short and never reaches its limit.
- Track each accepted
request_iduntil it becomes terminal. - Use backoff and jitter instead of retrying in a tight loop.
- Keep polling traffic separate from generation submission concurrency.
- Let us know in Discord before a planned traffic increase if you need higher limits.
Successful responses do not yet include rate-limit headers with your remaining capacity. Treat the limits shown in your dashboard as the authoritative values for your account.