Best practices

Get the most out of Transloadit in terms of speed and security

Use Templates

Templates allow you to store Assembly Instructions on our servers so that they are encrypted at rest. You then only reference the Template by its ID in your API request to create an Assembly.

By using Templates, you do not need to worry about other people seeing your Assembly Instructions, which may include credentials for import or storage services such as Amazon S3. You should make a habit of using Templates to avoid the risk of leaking sensitive information such as these credentials.

In addition, this also allows Assembly Replays. This is because if credentials end up directly in your requests instead of in Templates, we'll have to obfuscate them in the Assembly Status JSON. This is so that, should you share Assembly URLs with your customers, they cannot see the credentials.

It's possible to overrule any parameter in a Template if you want to tweak things like an image width based on user interaction.

You can set up Templates directly within your account at transloadit.com/templates after you log in. Once you save a Template, its ID will be printed to you to be copy/pasted into your integration code.

» Learn about Templates

Go async if possible

We try our best to avoid having any encoding queues. Nevertheless, they sometimes (if rarely) happen and when that does, your users will have a negative experience if, in your integration, they have to wait on processing.

Users can also be sent on their way as soon as the upload completes. When using Uppy, that means setting waitForEncoding to false, and making use of Assembly Notifications. With those, your visitors will only need to wait for the file uploading to finish. They do not have to wait for the subsequent file conversions to finish as well. This way, your visitors will bypass any queues, allowing them to move on as soon as the upload has finished.

When the processing is done, we ping your notify_url, and you ping your user.

Notifications are a good way to obtain file results, because they allow you to specify a URL that Transloadit will use to contact you whenever an Assembly has finished all of its encoding. You then save the results in your database and inform the user that the results are available. We retry these notifications if your backend is unavailable, and support Signature Authentication for Notifications to secure them.

By employing Assembly Notifications, your users will never waste time staring at progress bars.

» Learn about Assembly Notifications

Leverage the Transloadit CDN

If you need to deliver or present files to users, we recommend utilizing a Content Delivery Network (CDN) to significantly enhance the performance and reliability of your content delivery. Transloadit offers a SmartCDN that allows you to distribute your content across the globe, ensuring fast and efficient delivery to your users regardless of their geographical location.

Benefits of Using the Transloadit SmartCDN

1. Improved Speed: By caching your files on servers distributed across various geographic locations, the Transloadit SmartCDN reduces latency and delivers content faster to end-users than serving your content from a single server.

2. Enhanced Scalability: The CDN infrastructure is designed to handle large volumes of traffic without compromising performance, ensuring a seamless experience for your users even during periods of high demand.

3. Increased Reliability: The distributed nature of the CDN minimizes the risk of downtime and network congestion, providing a reliable platform for serving your content to users worldwide.

4. Lower Data Usage: CDN caching helps optimize your monthly usage by serving cached content from edge servers closer to the user, reducing the load on origin servers and preventing unnecessary file transformations, that would otherwise incur further costs.

Best Practices for TLCDN Usage

• Cache Optimization: Configure caching headers and cache-control directives to control how content is cached and invalidated on the CDN edge servers, balancing between freshness and efficiency.

• On Demand File Transformations: Rather than having to configure independent Templates for the different content you need to serve, use a single Template and override the specific parameters in the URL, using URL-based transformations .

By leveraging the Transloadit SmartCDN, you can optimize the delivery of your files, improve the end user experience, and let us worry about delivering files where they're needed.

» Learn more about TLCDN

Set a bill limit

When you hit the limit of a paid plan, Transloadit will continue to serve you — albeit at a higher price — and sends emails that more economic options are available. While integrating, "infinite loop" type of bugs can, of course, happen. To stop the service in such an event, you can set a Bill Limit. Set it high enough so that any natural growth will not be hindered by it, and low enough that you won't ramp up the bill to 100x your normal usage just because of a simple bug. This is especially useful for back-end integrations.

Check your Robot's documentation

It's easy to take inspiration from a library of demos, and copy-paste them to get going quickly. Sometimes, however, Robots come with edge cases or limitations. All the ones we know have been documented carefully. Reading this documentation can save you headaches or surprises later on.

When saving encoding results, do not rely on their order!

When your encoding results are finished and Transloadit POSTs to your result URL or Assembly Notification URL, please do not rely on the proper order of results.

Imagine you have two 🤖/image/resize Steps in your Template that use the :original step. Like so:

{
  "steps": {
    "high": {
      "use": ":original",
      "robot": "/image/resize",
      "width": 500,
      "height": 500
    },
    "low": {
      "use": ":original",
      "robot": "/image/resize",
      "width": 200,
      "height": 200
    }
  }
}

When you upload 30 photos with this Template, the resulting Assembly JSON contains two result arrays: "results" -> "high" and "results" -> "low". The order of images within these arrays is not the same, though! The first "high" result could be a derivative of the 15th uploaded photo, whereas the first result in the "low" array might be a derivative of the 10th uploaded photo.

If you need to match encoding results to their corresponding uploads (you likely need to), please match them by the id property from the "uploads" array entries and the original_id property of the entries in your various result arrays.

Use the latest encoding stack

Our customers would get upset if we upgraded our encoding software without telling them. There are often backward-incompatible changes, so we keep old stacks around for a long time, giving our customers the chance to upgrade when they are ready for it.

This does, however, put a burden on the customer to every now and again try out a newer stack. We recommend checking available encoding stacks at least once a year.

We currently recommend:

• "ffmpeg_stack": "v6.0.0"
• "imagemagick_stack": "v3.0.0"

» Learn about Encoding Engines and our upgrade policies

Secure your requests

When, for example, you are integrating with us in your browser using Uppy, your auth key is publicly visible. Other people can read your Auth Key just by looking at the HTML source of your website. They could then upload files to your account from their websites and you would be billed for it.

To prevent this, please consider enabling Signature Authentication if you integrate with us through browsers or other public channels.

In addition to this, use our HTTPS endpoints (https://api2.transloadit.com) so instructions are encrypted in transit.

» Learn more about Security and encryption in transit

Don't give us the keys to your everything

Taking a few security paradigms to heart, we'd like as little privileges as possible to do our work. If you export files to your Amazon S3 bucket, for instance, it's perfectly possible to create an IAM user that only has write access for that. In the unlikely case that Transloadit gets compromised, we would like the worst-case scenario to be that a few extra files are stored in your bucket by a hacker.

» Learn more about Security

Encode more files in fewer Assemblies

Assemblies are expensive on our platform, and to ensure performance for all, we have a Rate Limiter in place.

If possible, we recommend our customers to utilize the fact that Assemblies can process many files. If you have a large library to import, we typically recommend processing up to 1000 files per Assembly. Chunking things like this also allows you to retry a smaller chunk if you're unhappy with the results for some reason. Please make sure to not import files sized at more than 10GB (all file sizes combined) in the same import Step.

Processing more files in fewer Assemblies allows you to maximize processing speeds as our Rate Limiter will happily accept a lower Assemblies, regardless of the number of files inside.

Please also consider that one ZIP file could result in tens of thousands of files when we extract it.

Store the originally uploaded files

Storing not only the encoded result files, but also the files that your users originally uploaded gives you the chance to re-encode them later on into different formats as your requirements change. If you do not store them, they are lost forever, because asking your users to upload them again is likely not an option.

Store any metadata

In addition to the above, keep in mind that Assemblies are ephemeral. Transloadit purges records after three months, so if you want to hang on to any metadata inside of them, you should import it to your datastore.

Use the "use" parameter

Every Robot (except import Robots) provides a use parameter, which determines which Assembly Step should be used by this Robot as its input. The order of Steps in JSON format cannot be predicted. Hence, this parameter should be used to prevent weird side effects from Robots using the wrong Steps as their input.

It is best practice to always use the "use" parameter in every Step, because it can be a source of confusion when left out.

Design for failure

We strive to provide a top-notch service with no downtime and zero crashed Assemblies. While we have many years of experience in this field, it remains technically challenging to run a distributed service such as Transloadit. In other words, problems cannot be ruled out completely. A server might crash or perform badly, resulting in some of your Assemblies crashing.

If you keep the following things in mind, crashed Assemblies can be replayed automatically by our service and neither you nor your customers would have to even notice that something went wrong. To design for failure:

• Enable automatic Assembly Replays in your account. You can do so here.
• Always use Templates, or automatic Assembly Replays will not always work properly. This is because we obfuscate S3 credentials, etc., that come from request parameters, which even our own system can't decipher anymore when it wants to replay an Assembly at a later stage.
• Design with the Rate Limiter in mind. If you send us too many requests, we will rate limit them or even block your traffic entirely. You can learn about our Rate Limiters here.
• Implement retries on your end.

Check for Assembly warnings

Assembly warnings are a source of optimization and compatibility advice. We'll warn about using deprecated stacks, being explicit about your bucket regions to avoid AWS doing an extra lookup, and much more.

Use an SDK

We recommend using official Software Development Kits. SDKs come with many of these Best Practices out of the box. They usually support Signature Authentication, handling rate limits, and retries. This often means higher quality integration, and less time to market.