Handling Rate Limits and Retries in a Marketing Automation

Why does my automation fail only when it gets busy?

Because you are hitting a rate limit. Every API you depend on caps how many requests it will accept in a window, and your automation only discovers that cap when volume rises. The logic is fine. The pace is not, and pace only becomes visible at scale.

This is the most common way a working automation turns into a broken one without anybody changing a line of it. A bulk import, a busy campaign day, or simply a growing CMS pushes you over a number you never looked up.

Here is what the limits actually say on the tools I use most, and the handling that makes the difference between a delay and a data loss.

What is a rate limit and how is it usually measured?

A rate limit is a cap on requests within a time window, and it is usually measured per minute. Anthropic's documentation describes limits for the Messages API measured in requests per minute, input tokens per minute, and output tokens per minute for each model class, which shows how many dimensions a single limit can have.

The important subtlety is that a per minute limit is rarely enforced as a flat minute. Anthropic's documentation is explicit about this, noting that a rate of sixty requests per minute might be enforced as one request per second, so short bursts can exceed the limit and trigger errors even when your minute total is fine.

That single sentence explains a lot of confused debugging. People check their hourly volume, conclude they are nowhere near the cap, and keep firing requests in tight loops that breach a per second enforcement they did not know existed.

What do the real limits look like on the tools I use?

They are lower than most people assume. Webflow's developer documentation states that your site plan determines your Data API rate limit, listing sixty requests per minute on Starter and Basic, one hundred and twenty on CMS, eCommerce and Business, and custom limits on Enterprise.

Some endpoints carry their own constraints on top of that. Webflow's documentation notes that site publish operations are limited to one successful publish per minute. If your automation publishes after every item it creates, you have designed a queue that cannot drain, regardless of your plan.

There is also a useful exemption worth knowing. Webflow's documentation says that for cached requests to the content delivery API there are effectively no rate limits, while uncached requests to the origin server do count against your plan's limit. Reading and writing are not the same problem.

I keep these numbers written down next to each automation I run, because a limit you have to look up during an incident is a limit you will guess at instead.

What should happen the moment you get a 429?

Wait for the period the API tells you to wait, then retry with increasing gaps. Webflow's documentation says that exceeding the limit returns an HTTP 429 Too Many Requests error along with a Retry-After header telling you how long to wait, and that the reset time is typically sixty seconds.

Respect that header rather than inventing your own interval. Webflow's guidance is direct about it, recommending that anyone not using the SDK implement retry logic that respects Retry-After, and noting that the Webflow SDK includes built in exponential backoff that adjusts the wait between retries.

Exponential backoff matters because a fixed retry interval turns one failure into a synchronised stampede. Every stalled request comes back at the same moment, breaches the limit again, and you have built a loop that makes the problem worse each time round.

Also decide what happens to the record while you wait. A retry that holds the data in memory loses it if the process dies. A retry that parks the record in a queue survives, and surviving is the entire point.

Which 429s should you never retry?

The ones that are not really rate limits. Anthropic's documentation describes a case where reaching your tier's spend cap pauses API usage and returns an HTTP 429 with the error type rate_limit_error, the same type as an ordinary rate limit, but with no retry-after header on the response.

The documentation is blunt about what happens next: retrying, including the SDKs' automatic retries, fails until access resumes. It also gives you the way to tell the two apart, noting that on the Messages API the error details carry an error code of enforced_spend_limit_reached.

This is the kind of detail that separates an automation that degrades gracefully from one that burns an afternoon. A retry loop against a spend cap will run happily for hours, log nothing alarming, and deliver nothing at all.

So branch on the reason, not on the status code. Any handler that treats every 429 identically will eventually meet one it cannot solve by waiting.

How do you stop a bulk job from hitting the wall at all?

Pace it deliberately instead of running as fast as the code allows. Work out your permitted rate, divide it by two for safety, and space requests evenly rather than in bursts. A job that takes twenty minutes and finishes beats one that takes four and fails at minute three.

For large imports, batch the work and checkpoint after every batch. If the run stops, you resume from the last confirmed record rather than starting again and duplicating everything you already wrote. I covered the mechanics of that for large CMS imports in running bulk imports as background tasks.

Separate reads from writes where the platform prices them differently. If cached reads are effectively unlimited and writes are capped, then design the job so that the expensive half is as small as possible. Fetch first, decide in memory, write once.

What breaks when several automations share one key?

Your limit is shared and your failures become other people's failures. Two automations on the same API token draw from the same allowance, so a bulk import can starve a form handler that has nothing to do with it and would otherwise never come close to a limit.

This is a genuinely nasty class of bug, because the automation that breaks is not the one that misbehaved. Somebody investigates the innocent process for an hour while the guilty one finishes quietly and leaves no trace.

Give scheduled bulk work its own credentials wherever the platform allows it, and run it outside business hours. Interactive automations, the ones a customer is waiting on, should never share an allowance with a batch job. The failure modes of that arrangement are covered in how an Airtable to Webflow sync actually breaks.

How do you know this is happening at all?

Read the headers and record them. Webflow's documentation describes three response headers on every request: X-RateLimit-Remaining for the requests left in the current minute, X-RateLimit-Limit for your current limit, and Retry-After for the wait time. Logging the remaining count turns an invisible ceiling into a visible one.

Alert on the trend rather than on the failure. If your remaining count is regularly dropping into single digits, you are one busy day from an incident, and that is a far better moment to act than the morning after.

Count your retries too. A rising retry rate with no change in failure rate looks healthy on a dashboard and means your automation is quietly taking longer and costing more. That same principle of watching for silent drift is why I keep dependency notes for every automation, as I described in keeping an automation alive when tools change.

What should you do next?

Look up the actual rate limit for every API your automations call and write the number down next to the automation. Then check one thing in your code: does it read the Retry-After header, and does it distinguish a retryable 429 from one that will never succeed?

If the answer to either question is no, fix that before you optimise anything else. Rate limit handling is not glamorous work, and it is the difference between an automation that slows down under load and one that loses records under load.

If you have a job that keeps falling over at volume and you cannot see why, describe the shape of it to me. Nine times out of ten the answer is pacing, and it is usually a small fix.