Rate limits are the invisible fence around every AI API. Cross it and your request gets delayed, rejected, or silently queued. The problem: every provider structures limits differently, and most documentation buries the details three clicks deep.
If you’re building workflows that lean on AI—content generation, summarisation, customer-reply drafting—you need to know where the throttle sits, how hard it clamps, and what your fallback looks like when the queue fills.
How AI rate limits are measured
Most providers set three separate limits: requests per minute (RPM), tokens per minute (TPM), and tokens per day (TPD). Hit any one and you’re blocked until the window resets.
OpenAI’s GPT-4 models, for example, enforce both RPM and TPM caps that vary by subscription tier. On a free-tier account, you might be capped at 3 RPM and 40,000 TPM (checked November 2026, OpenAI rate limits documentation). Pay-as-you-go accounts on higher tiers get significantly more headroom, but the limits still exist.
Anthropic’s Claude models use a similar structure but label them differently: message batches, input tokens, and output tokens each have separate ceilings (checked November 2026, Anthropic rate limits guide). If you’re chaining prompts or running batch jobs, you’ll hit the output token limit before the request count.
Google’s Gemini API publishes per-minute request caps that reset on a rolling window, not a fixed clock boundary—so a burst at 10:59 and another at 11:01 can still trigger a block (checked November 2026, Gemini API quotas).
What happens when you exceed the limit
Behavior varies. OpenAI returns a 429 status code and expects your application to retry with exponential backoff. If you don’t handle it, the request just fails.
Claude queues some requests when you’re near the limit but not over it, then processes them as capacity frees. That sounds helpful until you realise your API call now takes 45 seconds instead of 5, and your user is staring at a spinner.
Some platforms—particularly those wrapping multiple models under one API—will silently downgrade your request to a cheaper, faster model when the preferred one is rate-limited. You won’t get an error, but you also won’t get the output quality you expected.
How to structure requests around limits
If you’re running a workflow that makes more than a handful of AI calls per hour, you need request pacing. The simplest approach: add a fixed delay between calls. Say you’re allowed 60 requests per minute—schedule one every 1.1 seconds and you’ll never hit the wall.
For batch jobs, split the work into smaller chunks and process them serially instead of firing everything at once. A backlog of 200 summarisation tasks can run overnight without a single retry if you queue them properly.
If you’re using a no-code automation tool like Zapier or Make, check whether it respects rate-limit headers. Some do; others will hammer the endpoint until it returns an error, then mark the entire task as failed. In that case, you’ll need to add a manual delay action between steps or route requests through a middleware function that handles retries.
The non-obvious tip: monitor your token budget, not just request count
Most operators watch request volume and assume they’re safe. Then a single prompt with a 15,000-token context window eats half the daily allowance in one call.
Track both input and output tokens separately. If you’re generating long-form content, output tokens climb fast—and on tiered pricing, they’re often the expensive half. Set a per-request token ceiling in your application logic so a runaway generation doesn’t blow your quota before lunch.
Anthropic and OpenAI both return token counts in the API response metadata. Log them. Build a running tally. When you approach 80% of your daily cap, switch to a smaller model or pause non-critical jobs until the window resets.
Rate limits aren’t advertised as a feature, but they shape how you build. Know the ceiling before you hit your head on it.
Want more breakdowns like this? Subscribe to One Two Three Send—we explain the mechanics behind the tools you use every week.
Heads up — some links in this article are affiliate links. If you sign up through them, we may earn a small commission at no extra cost to you. We only recommend tools we use ourselves.