Will there ever be an encoding queue?
In 2013, it happened three times that we had a few minutes of queue time for our live queue. We were receiving big volumes and it took a few minutes for new machines to come online and help drain the queue more quickly.
We have already:
- separated batch/import jobs from live/upload jobs
- increased our base capacity to 540 cores
- parallelized upscaling
- implemented analysis of uploads before they enter the encoding queue, so we can launch additional machines ahead of time
- brought down launch times to five minutes
- raised our instance ceiling at Amazon, so we can have 1500 heavy machines online (raised over the years, as our requirements grew, from the default of 20).
We are planning to do even more to give a real-time feeling to our live queue. The truth, however, is that it is still a queue and we cannot 100% guarantee that a few minutes wait time won't ever happen. We strive for this, but having a base capacity big enough to handle any spike instantly would not be economical. For us, or anyone else.
We realize this could mean a bad user experience if a visitor of yours is just trying to upload e.g. an avatar. This is why we recommend setting
wait: false in your jQuery integration (if that is what you use of course).
wait: false makes sure media processing is handled in an asynchronous way, so 2 minutes queue-time won't block the user experience. As soon as the files are uploaded to us, the user is on their way. We notify you when the files are ready, you notify your user.
Of course, we are not sure if this workflow is feasible for your use-case. At any rate, you can be assured that live queues are very rare, and we are always working on making them even rarer.
One thing we're doing to prevent live queues, is keeping dozens of machines on stand-by. This provides reserved capacity for all of our customers, allowing them to burst with many simultaneous live jobs, before being trickled down into the batch queue. It's possible to purchase more of this reserved capacity through higher plans, allowing you to have more real-time jobs.
More on this topic in reserved capacity.View more FAQs