
Every AI model API enforces rate limits—caps on how many requests you can send in a rolling window. Hit the ceiling and you get a 429 status code, your script stalls, and work stops until the bucket refills. For solo operators running content workflows, customer-support triage, or bulk transcription, understanding how these limits work saves you from silent failures and wasted time debugging middleware that isn’t broken.
Rate limits aren’t the same as usage quotas or token caps. A quota governs total spend or tokens per month; a rate limit governs velocity—requests per minute (RPM) or tokens per minute (TPM)—and resets on a short cycle. Both can block you, but rate limits reset faster and trip more often when you parallelize work.
What counts as a request and when the counter increments
One API call equals one request, regardless of prompt length or response size. If you send ten questions in a loop without pausing, you consume ten requests in a few seconds. Most providers count the increment the moment your HTTP request hits their edge, before the model runs; a rejected call still burns a slot if it got past authentication.
Token-per-minute limits work differently: the counter adds your prompt tokens plus generated completion tokens. A short question that yields a long answer can cost more TPM than a long question with a terse reply. Some tiers count only completion tokens; others count both. Check your provider’s documentation for which counter applies to your plan tier.
Anthropic’s Claude API rate-limits page (checked November 2026) lists separate RPM and TPM caps for each model and tier. OpenAI publishes similar tables in its rate-limits guide (checked November 2026), and both reset limits using a token-bucket algorithm rather than a fixed window.
Token bucket vs. fixed window: why your script can send more than the published limit
Fixed-window rate limits reset every sixty seconds at a wall-clock boundary. If your cap is 60 RPM and you send 60 requests at 12:00:59, the counter zeros at 12:01:00 and you can immediately send another 60. That creates a burst allowance of 120 requests in one second, which isn’t what the provider intended.
Token-bucket systems refill continuously. Picture a bucket that holds 60 tokens; every second one token drips in, and each request removes one token. If the bucket is full you can send 60 requests in a burst, but after that you get one new request per second as tokens refill. The published “60 RPM” describes average sustained throughput, not peak burst—though the initial bucket depth lets you front-load work if you’ve been idle.
Google’s Gemini API uses per-minute quotas with a similar refill pattern, documented in its quota reference (checked November 2026). Mistral and Cohere apply comparable logic. The practical upshot: if your script spaces requests one second apart, you’ll never hit a rate limit even when the published cap seems tight.
Handling 429 errors without crashing your workflow
When you exceed the limit, the API returns HTTP 429 and a Retry-After header telling you how many seconds to wait. Naïve scripts crash or skip the request; production code should catch the exception, sleep for the specified duration (plus a small jitter to avoid thundering-herd retries), then resubmit.
Exponential backoff works when you don’t have Retry-After: wait one second, then two, then four, capping at something reasonable like thirty seconds. Most provider SDKs build this in—Anthropic’s Python client and OpenAI’s both retry automatically up to two attempts by default—but if you’re calling the REST endpoint directly or using a thin wrapper, you need to code it yourself.
For batch workflows—say, summarizing a hundred articles—split the list into chunks and throttle each chunk to stay under your RPM cap. If your limit is 60 RPM, send one request per second (3,600 per hour) and you’ll never trigger a retry. A simple time.sleep(1) between calls in Python, or await new Promise(resolve => setTimeout(resolve, 1000)) in Node, keeps you safe. Libraries like asyncio semaphores or p-limit in JavaScript let you run n requests concurrently while respecting the global rate.
When to request a limit increase and what providers ask for
Free tiers and new accounts start with low limits—often 5–20 RPM for frontier models—to prevent abuse. Paid tiers lift caps automatically; Anthropic’s paid Claude tiers start at 50 RPM and scale with usage history, while OpenAI offers higher limits on pay-as-you-go accounts that demonstrate consistent spend.
If your workload outgrows the automatic cap, most providers let you request an increase via a web form. You’ll describe your use case, estimate peak RPM, and sometimes link to your app or site. Approval is faster if you’ve been a customer for weeks, have billing history, and can show you’re not reselling API access. Turnaround ranges from a few hours to a few days; don’t wait until launch day to ask.
Rate limits exist to protect infrastructure and ensure fair access, but they’re not set in stone. Build retry logic into every integration, monitor your request velocity, and throttle proactively when you know a batch job is coming. The five minutes you spend adding a sleep timer will save you an hour of debugging 429 errors at three in the morning.
Have a question about running your online business? Reply to any One Two Three Send email—we read every message and turn the best into future articles.
Heads up — some links in this article are affiliate links. If you sign up through them, we may earn a small commission at no extra cost to you. We only recommend tools we use ourselves.
