Transloadit

Rate limiting

We enforce rate limits to guarantee that no customers are adversely affected by the usage of any given customer. Buggy integration code can result in too many requests being sent in a short period of time, leading to excessively high bills or increased queue times.

We have three Assembly limits in place:

  1. Customers can create up to 250 Assemblies per minute.
  2. By default, each Workspace can have 2,500 Assemblies running at the same time in each region.
  3. By default, an Assembly can spend up to 8 hours uploading and then up to another 8 hours executing. These phase limits are independent, so its end-to-end lifetime can approach 16 hours.

In our experience, this satisfies even top-tier usage, but enterprise accounts are free to contact us if they require higher limits.

The limits are in place mostly to shield customers from other customers’ “infinite loop”-type of programming bugs.

Customers that hit a creation or concurrency limit receive a RATE_LIMIT_REACHED error and HTTP status code 429. The JSON payload for this error contains an info.retryIn property that indicates the number of seconds in the recommended backoff delay before retrying the request. It does not guarantee that concurrent capacity will be available after that delay. If you use one of our official SDKs, proper back-offs and retries based on this are already included.

The following is an example of what the JSON payload may look like:

{
  "error": "RATE_LIMIT_REACHED",
  "message": "Request limit reached",
  "info": {
    "retryIn": 41
  }
}

Exceeding an Assembly’s upload or execution time limit is different: it records ASSEMBLY_EXPIRED in the Assembly Status. The status response can use HTTP 200, so inspect its error field instead of treating HTTP success as processing success. Backoff does not resume an expired Assembly. If its input files can still be recovered, replay the Assembly to start a new one.

High frequency rate limit

In addition to the above rate limiting, there also exists another rate limiting mechanism in order to deter too rapid requests. This rate limit will trigger in any of these cases:

  • If there are too many simultaneously open TCP connections from one client
  • If too many TCP connections are opened in the last 10 seconds from one client
  • If too many HTTP error codes have occurred in the last 10 seconds from one client
  • If too many HTTP requests have been sent in the last 10 seconds from one client

In these cases the client will receive a 429 HTTP status code, and the response will contain the following JSON payload:

{
  "error": "429 Too Many Requests",
  "hostname": "morven.transloadit.com",
  "message": "You have sent too many requests in a given amount of time. "
}