Easily remove green screens with Transloadit & FFmpeg

Using, and then removing, a green screen can be a tedious process. However, if conditions are favorable, leanings towards some automation can help save a lot of time. Naturally, at Transloadit we love automating the minutiae, so let's take a look at how we can use FFmpeg and Transloadit to automatically chroma key out a green screen from a video – leaving it with a transparent background.

How does a green screen work?

Before we can remove a green screen, we need to understand what a green screen really is. A green screen is placed behind a subject, to create a uniform background that can be more easily removed. Normally, the color chosen is a neon green, since it's the furthest from skin-tones, although blue is also commonly used.

When you want to remove the background, you need to use a chroma-key to selectively remove the green colour from each frame of the video, leaving you with a transparent background behind your subject.

FFmpeg command

Before we get into the nuts and bolts of our Assembly Instructions, let's pop open the hood and take a look at what is underneath 🔎

Transloadit uses a selection of CLI tools to help us perform transcoding operations on your files. One of these tools is the highly regarded FFmpeg, which we use in many of our Robots for video, audio and image transcoding.

Before we draft up our Template, we can first attempt the operation locally using an FFmpeg command. Then, we convert the FFmpeg command into a set of Assembly Instructions, allowing us to capitalize on the convenience and speed that Transloadit offers.

To remove a green screen from a video, the command is as follows:

ffmpeg -i input.mp4 -vf 'colorkey=0x00FF00:similarity=0.2:blend=0.3' -c copy -c:v vp9 -f webm output.webm

Let's take it apart.

First, we use the i flag to specify our input file. From this, we can pass a series of video filters to the vf flag. In our case, we only need to use the colorkey filter, setting the color key to 0x00FF00.

We can then tweak the colorkey using the similarity and blend parameters. These specify how close a color needs to be to the key color to be made transparent, and also how harsh the key is. Feel free to experiment with their values to see how the result changes.

Next, we use c flag to copy the original video and audio streams to the output file and c:v is used to encode the video stream with the VP9 encoder. For more information on the VP9 encoder, check out our video compression blog.

Finally, we set the output format to webm, since this format supports video transparency.

Template

Now that we've seen the underlying command, let's see how it looks inside our Template.

{
  "steps": {
    ":original": {
      "robot": "/upload/handle"
    },
    "remove_green_screen": {
      "use": ":original",
      "robot": "/video/encode",
      "result": true,
      "ffmpeg_stack": "v6.0.0",
      "preset": "empty",
      "ffmpeg": {
        "vf": "colorkey=0x00FF00:similarity=0.2:blend=0.3",
        "c": "copy",
        "c:v": "vp9",
        "f": "webm"
      }
    },
    "exported": {
      "use": ["remove_green_screen", ":original"],
      "robot": "/s3/store",
      "credentials": "YOUR_AWS_CREDENTIALS",
      "url_prefix": "https://demos.transloadit.com/"
    }
  }
}

Notice how within the ffmpeg parameter, we can specify the exact same flags as in the original command, letting us manipulate the file in the same way, whilst gaining the extra functionality that Transloadit offers.

An important detail to note here is that we are using the empty preset, to make sure that no other encoding takes place on the file apart from what we specified. Check out our presets page to see the specifics on what each preset offers.

Final results

Below is the result from our Template, when passing in a video of a dinosaur in front of a bright green background.

Looks pretty good! All of this happened automatically too, allowing us to process hundreds of files with ease, and meaning we can export our results to a storage bucket to be retrieved later.

That's all for this blog! Hopefully, it has given a look behind the curtain on how to use FFmpeg within your Templates. To keep up to date with future blogs, and to see articles that our developers find interesting, why not sign up for our newsletter below?