At Transloadit, we like to focus on things that need to be improved. After being in operation for thirteen years, we continue to find those each day. While it leads to a lot of good, this mindset can also make it easy to underappreciate the things that are going well. We tend to quickly move onto the next issue and forget about whatever we just accomplished. The things that we forced into existence, often through hard work and dedication, all too quickly get mistaken for the new status quo.

I recently stumbled upon a few large, completed projects. Projects that I remember felt insurmountable when we stood before them, but were now denoted with a little green checkmark — hidden under an inconspicuous tab marked "Completed". These projects are in use today as if that never was any different. So here's an attempt to celebrate that "Completed" tab, that "Archived" Kanban column, the "Done" in Getting Things Done.

Some of my teammates have struggled with COVID and its direct and indirect hardships, such as depression or simply a general feeling of frustration and despair. On top of all that, my co-founder Tim had to deal with so much after his wife Cindy was diagnosed with cancer last year. Things like these can make you feel powerless and work was often a welcome distraction. So, especially in these challenging times, I feel it is good to reflect on and celebrate the progress that we have made.

Here are our company's highlights for the past year.

Vector art lilac background with rockets, planets and stars. The text '21 Highlights is in the foreground with the Transloadit logo just above.

API

New features

  • /azure/import — import files from Azure.
  • /image/ocr — recognize text in images.
  • /script/run — your own code on our systems to build very wild things; things that we haven't imagined yet, but you will! Powered by Deno.
  • /speech/transcribe — turn voices into text.
  • /text/speak — turn text into voices.
  • /text/translate — translate text from and to many different human languages.
  • /vimeo/store — upload videos to Vimeo.
  • Our Smart CDN (beta) is being used in production, first customers have been charged.

Improvements

  • /audio/artwork — you can opt into different Exiftool stacks.
  • /document/convert — can also convert MS Office documents and Markdown to HTML.
  • /image/facedetect — better accuracy by leveraging big-cloud-vendor AI.
  • /file/filter — conditions work more intuitively.
  • /http/import — more easily import Google Docs and Dropbox URLs, and add a fail_fast option to let you opt out of retrying failures.
  • /image/describe — now you can scan images for inappropriate content via the explicit_descriptions parameter.
  • /s3/store now checks the integrity of files after saving to S3.

Performance

  • Reduced Assembly overhead, making small Assemblies 5x faster.
  • Reduced queue times thanks to rewriting large parts of our autoscaler, and reducing machine launch-to-encode times 5x.
  • Reduced transcoding times by leveraging c6 machines.
  • Replaced ImageMagick with libvips when possible, reducing image resize times 10x.

Company

Team & expertise

  • Partnered up with authors of software that is of vital importance for us, like Nix, FFmpeg, Prometheus, Jekyll — as well as consultants from AWS readily available in our Slack.
  • Established seniority across the team. As founders, we've had to step out more, and we noticed others were stepping up to cover for us. We installed an official lead who calls the shots in our absence.
  • Hired Yukesh, Joseph, Merlijn, Antoine, and Florian. That means by end of 2021, we counted 21 Transloadians among our ranks. Welcome folks!
  • Improved our chatbot to help us get into new habits, such as more efficient weekly calls and theme months during which we focus on one specific topic.

Accounting & finance

  • Automated much of the manual accounting labor.
  • Created a Dutch branch.
  • Dissolved Transloadit Ltd into Transloadit-II GmbH.
  • Established payrolling for German employees.
  • Replaced Worldpay with Stripe for charging customers and storing their credit card details.
  • Switched to a new accountant, a large firm with offices around the globe, to help us better navigate international waters.
  • Switched to a new lawyer, who revised our contracts for freelancers and enterprises.

Developer experience

As with any craft: a dirty, unsafe workshop with blunt, half broken tools does not inspire great results. It's possible, yes, but you'll see a lot less of them. Programming is no different. Besides reduced friction being a boost on creativity & producticity, it also helps to attract and retain talent. So we invest a lot in a clean, safe workspace, and sharpening our tools.

CI

  • Added browsable website previews to every Pull Request.
  • Created Git Tags during deploy, so we can more easily track changes between releases of the API.
  • Cut the API test suite time in half.
  • Reduced the time it takes to build operating system images (AMIs) by a factor of three.
  • Replaced Jenkins with GitHub Actions, adding three Hetzner Runners, and created a script to easily add those. We also switched from Build Numbers to Build URLs, making it easy to deploy from different branches, CI environments, or sources.
  • Switched all SDKs from Travis to GitHub Actions.
  • Switched from custom deploys to full CI/CD.
  • The website's Lighthouse score is now shown in every Pull Request.

Code quality

  • Dropped support for Internet Explorer.
  • Added Sentry to get a grip on any client-side error that our website could be throwing.
  • Introduced React + Hooks, TypeScript, Jest, Playwright, and Async/await to many codebases.
  • Rolled out our own linting and applied it in all big JS projects.
  • Sunset unmaintained modules like knox, required-cache, and unmaintained versions of async, aws-sdk, lodash, etc.
  • Upgraded our API from Node 10 to Node 14, so we can use optional chaining and more.

Iteration times

  • Made our dev environment bootstrap without manual intervention, and improved documentation for it.
  • Made starting up and tearing down staging clusters so easy that it became fun. We also split log handling for it, allowing us to see issues without having to wake up the Ops team.
  • Removed variables from any require or import path to have better code autocompletion in editors.
  • Sped up static content build fourteen times.
  • Upgraded Webpack from 2 to 5, and switched to Dart Sass and CSS+code splitting for a better asset building & downloading experience. We ditched Docker-based Jekyll as it is too slow on macOS.

Project management

  • Replaced Asana with GitHub Projects and added automation to ease project management.
  • Switched to free GitHub, adding actions to compensate for some shortcomings.

Growth

  • Designed T-shirts that you can be seen in at a party.
  • Nearly doubled our revenue.
  • Introduced Slack Connect for our most valued customers.
  • Sent out our 50th Dev Times.
  • Signed our first OSS support contract.
  • Signed up WeTransfer, Blackboard, and many more, as new customers.

Infra & stack

  • Added support to reloads for HAProxy and Nginx so deploys are hitless.
  • Changed to per-build nix profiles, solving much — if not all — of our stack issues in production.
  • Hardened production so that it can sustain a 20 minute database outage.
  • Improved handling of Spot Termination to more gracefully hand jobs to other machines.
  • Improved rate limiting and responses so clients have a better time dealing with them.
  • Moved all legacy AWS infra into a VPC.
  • Ported raidmaker from Bash to Python to more robustly and correctly bootstrap whatever drives we attach to a machine when launching one on AWS.
  • Upgraded our base operating system to Ubuntu Focal, got rid of snaps, and scheduled automated apt security patches. We also added tooling to inspect coredumps.
  • Upgraded Terraform to 1.0, FFmpeg to 4.3, Nix to 2.3, and HAProxy to 2.0.

Open source

Uppy

  • More than doubled revenue allocation for Uppy.
  • Added screen capture capability to Uppy.
  • Allowed Uppy Companion to stream uploads, making it much faster and resource efficient.
  • Attacked Uppy DX issues (corepack, ditched Lerna for Yarn 3, GitHub Actions, Vite, ESBuild, Stale Bot, auto-releases).
  • Brought the Uppy open issue count down from 120 to 40.
  • Made Uppy work fluently with thousands of files thanks to a virtual file list.
  • Reduced Uppy bundle size, in part by removing IE polyfills.
  • Released Uppy 2.0.
  • Upgraded Preact to the latest version in Uppy.

Tus

  • Sped up tusd, made it handle RST, and implemented HashiCorp's Go plugin system.
  • Started work on tus-v2. The aim is standardization in HTTP, with help of Apple, Cloudflare, Vimeo.

SDKs & Node.js

Ops

  • Added observability through Prometheus & Grafana. Included snapshots of graphs in Pagers for more efficient troubleshooting.
  • Added SSH tooling to let us quickly open sessions to machines in many different tabs or panes.
  • Created team policies for security and on-call rotation.
  • Increased the size of the on-call team, relieving strain on the existing crew.
  • Introduced the concept of Low Severity pagers that we can safely sleep through and check in the morning.

Security

  • Fixed more than twenty ethical hacker security reports.
  • Introduced nsjail to sandbox encoding tools further.
  • Reduced IAM permissions of different machine roles by analyzing precisely what they need and removing anything else.
  • Simplified secret rotation, and rotated all secrets.

Web

  • Achieved a Lighthouse score of 100 points for accessibility on our homepage, and instructed CI to fail if we drop below 100 to avoid ever regressing on this.
  • Created more than ten new demos.
  • Created a brand new Content Delivery page with an interactive demo showcasing our near real-time Smart CDN resize times.
  • Created a brand new Security page and answered many questions around this. It also includes a Hall of Fame for ethical hackers who find and report vulnerabilities.
  • Improved the UX of the Template Wizard.
  • Introduced feature pages for all of our Robots, their demos, and pricing.
  • Moved large parts of our website to S3 and CloudFront, making pages load much faster (US: 2x, EU: 6x, AP: 100x).
  • Moved many parts of the console to React (gradual process, but shipped ~80% already), giving our customers a faster and more pleasant UI, and our developers a more pleasant experience working on it.
  • Re-designed our homepage.
  • Rewrote and re-designed all documentation.
  • Rewrote and re-designed the status page and moved it off AWS completely. All the way up to the domain registrar, it is now completely separate from production.

Concluding

On top of this, we added more than six hundred smaller fixes and improvements that generally made things run smoother.

As a remote and digital company we got lucky in this pandemic. We saw other companies face existential crises, while we could often continue to work how we were used to. We didn't take that for granted and also actively decided to continue to create value where we could.

I can take very little personal credit for all the work that was done subsequently, so I am all the more proud of my teammates and the countless mountains they have moved. They not only kept our ship afloat, but managed to propel it into uncharted waters. Our team persevered when it needed to, and enjoyed the good times when it was possible.

My dear Transloadians, I am grateful for all of your contributions. I'm not sure how we succeeded in finding such a talented bunch of caring individuals, but I wouldn't want to be on this adventure with any other crew.

(although there is room to expand the crew, so if you are interested in working for Transloadit, check /jobs/)