What this usually means
The 429 error means you've exceeded one of OpenAI's rate limits: requests per minute (RPM), tokens per minute (TPM), requests per day, or IP-based limits for the free tier. The response headers tell you which limit you hit — check 'x-ratelimit-limit-requests', 'x-ratelimit-limit-tokens', and 'x-ratelimit-remaining'. If you see 'x-ratelimit-remaining: 0', you are at the cap. The underlying cause is almost always a missing or broken retry-with-backoff strategy, or a misconfigured parallel request pattern that doesn't respect the tier limits.
The first ten minutes — establish facts before touching code.
- 1curl -v https://api.openai.com/v1/chat/completions -H "Authorization: Bearer $OPENAI_API_KEY" -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"test"}]}' 2>&1 | grep -i '429\|ratelimit'
- 2Check the response headers: x-ratelimit-limit-requests, x-ratelimit-limit-tokens, x-ratelimit-remaining, x-ratelimit-reset-requests, x-ratelimit-reset-tokens
- 3Look at your OpenAI dashboard (https://platform.openai.com/account/usage) to see current usage and rate limit tier
- 4Review application logs for the exact timestamp and request count preceding the 429
- 5Check if you're using a single API key across multiple services or threads without coordination
The specific files, logs, configs, and dashboards that usually own this bug.
- searchApplication logs: grep for HTTP 429 or 'Rate limit'
- searchOpenAI usage dashboard: https://platform.openai.com/account/usage
- searchOpenAI rate limits page: https://platform.openai.com/account/rate-limits
- searchYour code: look for HTTP client configuration, retry logic, and parallel request patterns
- searchLoad balancer or proxy logs if requests go through a gateway
- searchMonitoring tool (Datadog, Grafana) for API call volume and latency metrics
Practical causes, not theory. These are the things you will actually find.
- warningNo retry logic or immediate retry without backoff (causing cascading failures)
- warningExcessive parallel requests from async/threading code that doesn't respect concurrency limits
- warningUsing a free-tier API key (which has strict IP-based limits like 3 RPM)
- warningHitting the tokens-per-minute limit on a model like GPT-4 (TPM is often lower than GPT-3.5)
- warningMisconfigured batch processing that fires all requests at once on a timer
- warningShared API key across multiple services or environments (dev/staging/prod) without rate limiting
Concrete fix directions. Pick the one that matches your root cause.
- buildImplement exponential backoff with jitter: start at 1s, double each retry up to 60s max, add random jitter
- buildUse a token bucket rate limiter in your application code (e.g., token-bucket or leaky-bucket libraries)
- buildUpgrade your OpenAI tier to increase RPM/TPM limits if you need sustained throughput
- buildSet up a queue (e.g., Redis or SQS) to serialize requests and control concurrency
- buildFor chat completions, reduce max_tokens or use a cheaper model to lower TPM consumption
- buildMonitor remaining limits via headers and preemptively throttle before hitting 0
A fix you cannot prove is a guess. Close the loop.
- verifiedRun a load test with your retry logic: send 100 requests in parallel, verify no 429 responses
- verifiedCheck that x-ratelimit-remaining never drops to 0 in a sustained test
- verifiedConfirm retry count in logs: after a 429, you should see a backoff delay and eventual success
- verifiedUse the OpenAI dashboard to confirm usage stays below 80% of your rate limit during peak
- verifiedDeploy a canary: gradually increase traffic and observe zero 429s before full rollout
Things that make this bug worse or harder to find.
- warningRetrying immediately on 429 — this guarantees another 429 and can get your key temporarily blocked
- warningIgnoring the Retry-After header in the response (OpenAI often sends it); use that as the delay
- warningUsing a single API key for batch jobs and real-time user requests without separation
- warningSetting max_tokens too high for the model's TPM limit (e.g., 4096 tokens per request with GPT-4 can exhaust TPM quickly)
- warningNot testing with realistic traffic patterns — a few requests in dev may not reveal the issue
- warningHardcoding delays instead of reading remaining/reset headers to dynamically adjust
Batch Processing Pipeline Flooding OpenAI API
Timeline
- 09:15Deploy new batch summarization job that uses GPT-3.5-turbo to summarize 5000 user documents
- 09:16First batch of 100 requests fires concurrently via Celery tasks
- 09:17All 100 tasks receive HTTP 429; no retry logic, tasks fail permanently
- 09:18User-facing API endpoints also start failing with 429 because they share the same API key
- 09:20On-call engineer paged: production OpenAI calls returning 429 for user requests
- 09:25Engineer checks OpenAI dashboard: RPM limit 3500/min, but TPM limit 60k/min for GPT-3.5
- 09:30Found that each summarization request uses ~2000 tokens, 100 parallel requests = 200k TPM, exceeding limit
- 09:35Hotfix: reduce concurrency to 20 tasks at a time, add exponential backoff retry
- 09:40Re-run batch: now succeeds with occasional retries, no impact on user traffic
We had a batch processing job that needed to summarize 5000 documents using GPT-3.5-turbo. The job used Celery to distribute tasks, and each task called the OpenAI API directly with no rate limiting. I deployed it thinking 100 parallel tasks would be fine — we had 3500 RPM, after all. Within seconds, all 100 tasks returned 429. The real culprit was the tokens-per-minute limit: each request used about 2000 tokens, so 100 requests consumed 200,000 tokens — way over the 60k TPM limit.
Worse, the same API key served our user-facing chatbot. User requests started failing too because the batch job had exhausted the TPM bucket. I panicked and killed the batch job, but the damage was done. Users saw 'Service Unavailable' errors for about two minutes. I checked the OpenAI dashboard and saw the TPM usage spike to 200k. That's when I realized I had been looking only at RPM, not TPM.
The fix was threefold: first, I reduced the batch concurrency to 20 tasks and added a token bucket limiter using the openai library's built-in rate limiter. Second, I implemented exponential backoff with jitter for retries, reading the Retry-After header. Third, I created a separate API key for batch jobs with its own rate limit tier. After the fix, the batch job completed in about 30 minutes with only a few retries, and user traffic was unaffected. Lesson learned: always check both RPM and TPM limits, and never share keys between batch and real-time systems.
Root cause
Batch job sent 100 parallel requests each using ~2000 tokens, exceeding the 60k TPM limit. No retry logic, and shared API key caused collateral damage to user traffic.
The fix
Reduced concurrency to 20, implemented token bucket rate limiter, exponential backoff with jitter, and separate API key for batch jobs.
The lesson
Always monitor and respect both RPM and TPM limits. Use separate keys for different workloads. Implement retry with backoff from day one.
Every OpenAI API response includes rate limit headers. The key ones are: x-ratelimit-limit-requests (max RPM), x-ratelimit-limit-tokens (max TPM), x-ratelimit-remaining (remaining requests or tokens), x-ratelimit-reset-requests (seconds until request limit resets), x-ratelimit-reset-tokens (seconds until token limit resets). For 429 responses, also look for Retry-After header which specifies seconds to wait.
To diagnose which limit you hit, check the response body: it often says 'Rate limit reached for tokens' or 'requests'. Also check x-ratelimit-remaining: if it's 0 for requests, you hit the RPM limit; if it's 0 for tokens but not requests, you hit TPM. The reset headers tell you when the bucket refills — use that to set your retry delay exactly.
A robust retry strategy is critical. The textbook exponential backoff doubles the delay each retry, but without jitter, multiple clients can synchronize and still overload the API. Add random jitter to spread out retries. Example in Python: delay = min(60, base * 2 ** attempt) + random.uniform(0, 1). Use the Retry-After header if present, but fallback to your own calculation if missing.
OpenAI's Python client library (v1.0+) has built-in retry support via the openai.Retry class. You can configure max_retries, backoff_factor, and status_codes to retry (e.g., 429, 500). Example: client = OpenAI(max_retries=3, default_headers={'Retry-After': '5'}). However, this only handles retries at the client level; for fine-grained control, implement your own with asyncio or a task queue.
To prevent hitting limits proactively, use a token bucket algorithm. The bucket holds a number of tokens representing allowed requests or tokens. Each request consumes tokens, and tokens replenish at a fixed rate. For example, if your tier allows 60k TPM, you can set a bucket with capacity 60000 and refill rate 1000 tokens/second (60k/60). Before each API call, wait until enough tokens are available.
In Python, use the `ratelimiter` or `token-bucket` library. For high-throughput async code, use `asyncio` semaphores with a timer. Alternatively, OpenAI's client has a built-in rate limiter if you set the `max_retries` and use the `openai.Retry` class. However, for batch jobs, a queue-based approach (e.g., Celery with task rate limiting) is more scalable.
Set up monitoring to alert when you approach rate limits. The OpenAI dashboard shows usage over time, but it's delayed. Better: instrument your application to log rate limit headers and push them to a metrics system (Datadog, Prometheus). Create alerts when x-ratelimit-remaining drops below 20% of the limit. Also track the number of 429 responses per minute — a sudden spike indicates a problem.
For advanced monitoring, use an API gateway or proxy that can aggregate rate limit usage across all services. Some teams use a Redis-based counter to track current request/token usage in real time and throttle before sending requests. This prevents 429s entirely if you preemptively wait.
OpenAI has different rate limits per tier: free tier (3 RPM, 40k TPM, 200 requests/day), Tier 1 (500 RPM, 40k TPM), Tier 2 (5000 RPM, 80k TPM), Tier 3 (up to 500k RPM, 160k TPM). The limits also vary by model: GPT-4 has lower TPM than GPT-3.5. If you upgrade your tier, you must update your client configuration accordingly.
A common mistake is assuming all models have the same limits. For example, GPT-4-32k has a TPM limit of 40k even at Tier 1, while GPT-3.5-turbo has 60k TPM at Tier 1. Always check the specific model's rate limits in the dashboard. Also note that the free tier has an IP-based limit: if multiple instances run on the same IP, they share the 3 RPM limit.
Frequently asked questions
What does the 429 error response body look like?
It's a JSON object like: {'error': {'message': 'Rate limit reached for tokens. Limit: 60000. Current: 200000.', 'type': 'rate_limit_error', 'param': None, 'code': 'rate_limit'}}. The message tells you which limit (requests or tokens) and the current usage. Also check the headers for x-ratelimit-remaining and Retry-After.
Should I retry immediately on 429?
No. Immediate retry will almost certainly result in another 429 because the bucket hasn't refilled. Always wait at least the Retry-After header value, or implement exponential backoff with jitter. OpenAI's documentation recommends waiting at least 1 second before retrying.
Can I avoid 429 errors by upgrading my tier?
Upgrading increases your RPM and TPM limits, but you can still hit them if you don't implement rate limiting in your application. Always design your system to respect the limits regardless of tier. Upgrading gives you more headroom but doesn't eliminate the need for proper retry logic.
How do I calculate the token usage of a request?
Tokens depend on the model and the input/output length. Use OpenAI's tiktoken library to count tokens before sending. For GPT-3.5-turbo, roughly 1 token = 0.75 words. The response also includes 'usage' field with prompt_tokens and completion_tokens. Sum them to know total tokens consumed.
Why do I get 429 even though I'm under the RPM limit?
You likely hit the TPM limit. Each request consumes tokens, and if your requests are large (high max_tokens or long prompts), you can exhaust the token bucket before the request bucket. Check x-ratelimit-limit-tokens and x-ratelimit-remaining-tokens headers to confirm.