AI rate limits queue, fail, or charge you—here's which
The kettle clicks off in a quiet kitchen; outside, a delivery van idles while the driver taps their phone. Somewhere in Virginia, your API request sits in a queue behind eleven thousand others, waiting for a slot that may never come.
AI rate limits queue, fail, or charge you—here’s which provider does what
OpenAI, Anthropic, and Google all throttle requests differently—and handle overages in three incompatible ways.
Rate limits govern how many API requests you can make in a given window. Some providers count requests per minute, others per day, and a few track tokens consumed rather than calls made. When you exceed the cap, one of three things happens: your request queues and waits, it fails immediately with an HTTP 429 error, or it succeeds but triggers a higher billing tier. The outcome depends entirely on which vendor you’re calling and which pricing plan you selected.
Per-minute limits are common for real-time use cases—chatbots, live search augmentation, on-the-fly content generation. Cross that threshold and you’ll see either a retry-after header telling your code to wait, or an instant rejection. Per-day caps tend to apply to batch workloads: bulk summarisation, nightly digest generation, archive processing. Exceed the daily quota and some platforms let requests stack in a queue until the window resets at midnight UTC; others return an error and force you to wait. Token-based limits count input and output tokens separately or together, depending on the model, and a single long prompt can consume your entire budget in one call.
The distinction matters most when you’re chaining AI calls inside automation workflows, generating content at scale, or offering AI features to your own subscribers. A queued request might arrive two minutes late—acceptable for a weekly summary email, fatal for a live chat widget. A failed request requires retry logic, error messaging, and fallback content. A billable overage can double your monthly cost without warning if you didn’t set a hard spending cap in the dashboard.
RELATED TACTIC
AI output length caps strand you mid-sentence
Every model has a maximum output token limit, and when you hit it the response simply stops—mid-word, mid-paragraph, sometimes mid-JSON object. OpenAI’s GPT-4 models cap output differently from Anthropic’s Claude family, and the limit isn’t always visible in the API error response. If you’re generating long-form articles, email sequences, or structured data, you need to know the ceiling before you send the prompt. Otherwise you’ll parse incomplete JSON, publish truncated posts, or send subscribers an email that ends without a call to action.
ADJACENT RISK
Token pricing varies wildly—calculate before you scale
Rate limits tell you how fast you can call an API; token pricing tells you what it costs when you do. Input tokens, output tokens, and cached-prompt tokens all carry different price tags, and those rates shift between models even within the same provider. A workflow that costs three cents per run on one model might cost thirty cents on another. If you’re processing subscriber questions, generating social posts, or summarising reader feedback at scale, a back-of-the-envelope token calculation prevents budget surprises. The maths is straightforward once you know where to find the rates and how your prompts break down.
WORKFLOW WATCH
Automation chaining bottlenecks hide in the slowest step
When you chain an AI call inside a Zapier or Make workflow—trigger on new subscriber, generate welcome email, send via ESP—the entire sequence waits for the slowest connector. If your AI request queues because you’ve hit a per-minute limit, every downstream step stalls. The welcome email arrives five minutes late, the tag doesn’t apply, and the follow-up sequence misfires. Rate limits and automation delays compound. Knowing which step throttles lets you add a delay action, split high-volume workflows across multiple API keys, or route urgent requests to a faster model with higher throughput.
Know someone who would like this? Forward today’s email — every operator we reach is one closer to running an online business with a little less friction.