What’s new in video: transitions, chapters, CMAF, and more

Kevin van Zonneveld

Co-founder · Amsterdam, The Netherlands ·

Over the past months, we added a handful of video features, several of them because customers asked. We never wrote them up, so here they are in one place, each with Assembly Instructions that we ran in production before publishing.

Crossfade between clips

🤖 /video/concat joins videos end to end. Set transition to crossfade or fade_to_black, and the clips blend into each other instead of cutting hard:

{
  "steps": {
    "first": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/big-buck-bunny-10s.mp4"
    },
    "second": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/waves10.mp4"
    },
    "concatenated": {
      "robot": "/video/concat",
      "use": {
        "steps": [
          { "name": "first", "as": "video_1" },
          { "name": "second", "as": "video_2" }
        ]
      },
      "transition": "crossfade",
      "transition_duration": 1,
      "ffmpeg_stack": "v7"
    }
  }
}

The audio crossfades along with the picture. Here are frames from just before, during, and just after the transition:

Three video frames side by side: a grassy scene from Big Buck Bunny, the same scene blended with an ocean wave halfway through the crossfade, and the wave on its own

A few things to know:

transition_duration is in seconds and defaults to 1. It is capped at half the length of the shorter clip in each pair.
Clips overlap during the transition, so the result is shorter: our 10-second and 13-second clips made a 22-second video. That goes for fade_to_black too.
A transition means we re-encode the video, so the job takes longer than a plain join, which can copy the streams as they are.
🤖 /video/merge accepts the same two parameters, but /video/concat is the better fit for joining clips. It handles clips of different sizes and frame rates by converting them to matching specs first, and it keeps your MP4 output.

Chapter markers for joined videos

When you join an intro, the main content, and an outro, set chapter_markers: true, and players that support chapters can jump straight to each part:

{
  "steps": {
    "intro": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/big-buck-bunny-10s.mp4"
    },
    "main": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/waves10.mp4"
    },
    "outro": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/fake-ad.mp4"
    },
    "concatenated": {
      "robot": "/video/concat",
      "use": {
        "steps": [
          { "name": "intro", "as": "video_1" },
          { "name": "main", "as": "video_2" },
          { "name": "outro", "as": "video_3" }
        ]
      },
      "chapter_markers": true,
      "transition": "crossfade",
      "transition_duration": 1,
      "ffmpeg_stack": "v7"
    }
  }
}

Each chapter is named after its input file, without the extension. ffprobe shows them in the result:

$ ffprobe -v error -show_entries chapter=start_time,end_time:chapter_tags=title -of compact big-buck-bunny-10s_waves10_fake-ad.mp4
chapter|start_time=0.000000|end_time=9.000000|tag:title=big-buck-bunny-10s
chapter|start_time=9.000000|end_time=21.000000|tag:title=waves10
chapter|start_time=21.000000|end_time=26.000000|tag:title=fake-ad

To give chapters readable titles, name your input files accordingly. Chapter times account for the overlap when you add a transition, so each marker still lines up with its clip.

Bold, italic, and outlined subtitles

🤖 /video/subtitle burns subtitles into the picture, and you can now style them with bold, italic, and outline_width (0 to 10 pixels). This Assembly takes an uploaded .srt file and an imported video:

{
  "steps": {
    ":original": {
      "robot": "/upload/handle"
    },
    "video": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/fake-ad.mp4"
    },
    "subtitled": {
      "robot": "/video/subtitle",
      "use": {
        "steps": [
          { "name": "video", "as": "video" },
          { "name": ":original", "as": "subtitles" }
        ]
      },
      "subtitles_type": "burned",
      "preset": "empty",
      "font_size": 22,
      "bold": true,
      "outline_width": 4,
      "ffmpeg_stack": "v7"
    }
  }
}

The top half shows the default style; the bottom half shows bold: true with outline_width: 4, which stays readable on a light background:

The same subtitle line twice on a light video frame: thin regular text with a faint outline at the top, and bold text with a thick black outline at the bottom

The styling only applies to burned-in subtitles, not to subtitle tracks. "preset": "empty" keeps the input’s MP4 format. Without a preset, the Robot falls back to an old FLV preset.

Multi-language audio in HLS

To offer one video in several languages, you no longer need a separate video ladder per language. 🤖 /video/adaptive with audio_group: true packages audio-only files as alternate audio tracks, and every video rendition refers to them. Viewers then switch languages in the player without reloading the video:

{
  "steps": {
    "video": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/big-buck-bunny-10s.mp4"
    },
    "audio_en": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/joakim_karud-rock_angel.mp3"
    },
    "audio_de": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/german-anthem.mp3"
    },
    "video_480p": {
      "use": "video",
      "robot": "/video/encode",
      "preset": "hls/480p",
      "ffmpeg_stack": "v7"
    },
    "video_720p": {
      "use": "video",
      "robot": "/video/encode",
      "preset": "hls/720p",
      "ffmpeg_stack": "v7"
    },
    "aac_en": {
      "use": "audio_en",
      "robot": "/audio/encode",
      "preset": "aac",
      "ffmpeg_stack": "v7",
      "ffmpeg": {
        "t": 10,
        "metadata:s:a:0": "language=eng"
      }
    },
    "aac_de": {
      "use": "audio_de",
      "robot": "/audio/encode",
      "preset": "aac",
      "ffmpeg_stack": "v7",
      "ffmpeg": {
        "t": 10,
        "metadata:s:a:0": "language=deu"
      }
    },
    "hls": {
      "use": {
        "steps": ["aac_en", "aac_de", "video_480p", "video_720p"],
        "bundle_steps": true
      },
      "robot": "/video/adaptive",
      "technique": "hls",
      "audio_group": true
    }
  }
}

Our demo files are music, so picture them as an English and a German soundtrack. t: 10 trims them to the length of the video. Each track’s language comes from its language tag, which the metadata:s:a:0 option sets. Files that already carry a language tag don’t need it. The resulting master playlist gives the tracks readable names, and the first audio file in use becomes the default. If a track has a title tag, that title becomes its name; otherwise, the Robot uses the language name. AUTOSELECT=YES lets players select a track for the viewer’s language, and each rendition’s BANDWIDTH includes its audio:

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio",NAME="English",DEFAULT=YES,AUTOSELECT=YES,LANGUAGE="eng",CHANNELS="2",URI="128515_44100/128515_44100.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="group_audio",NAME="German",DEFAULT=NO,AUTOSELECT=YES,LANGUAGE="deu",CHANNELS="2",URI="128539_44100/128539_44100.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=797670,AVERAGE-BANDWIDTH=797670,RESOLUTION=852x480,CODECS="avc1.4d401f,mp4a.40.2",AUDIO="group_audio",FRAME-RATE=25
852x480_616860_25/852x480_616860_25.m3u8

With audio_group, the video renditions carry no audio of their own, so supply every language as a separate audio file. Store the results with ${file.meta.relative_path} in the path, as described in the Robot docs, so the playlists find their segments.

CMAF: one set of segments for HLS and DASH

Apple devices play HLS, and many other players prefer MPEG-DASH. With technique: "cmaf", /video/adaptive writes fragmented MP4 segments once, plus an HLS playlist and a DASH manifest that both point to them. You store and cache one copy of your video instead of two:

{
  "steps": {
    "imported": {
      "robot": "/http/import",
      "url": "https://demos.transloadit.com/inputs/big-buck-bunny-10s.mp4"
    },
    "encoded_480p": {
      "use": "imported",
      "robot": "/video/encode",
      "preset": "hls/480p",
      "ffmpeg_stack": "v7"
    },
    "encoded_720p": {
      "use": "imported",
      "robot": "/video/encode",
      "preset": "hls/720p",
      "ffmpeg_stack": "v7"
    },
    "cmaf": {
      "use": {
        "steps": ["encoded_480p", "encoded_720p"],
        "bundle_steps": true
      },
      "robot": "/video/adaptive",
      "technique": "cmaf"
    }
  }
}

The result contains playlist.m3u8 for HLS, playlist.mpd for DASH, and the .m4s segments they share. FFmpeg decoded both manifests without errors in our tests. hls_playlist_name and playlist_name rename the two manifests.

The stock hls/480p and hls/720p presets work together in a CMAF ladder, including the small aspect-ratio difference caused by rounding the 480p width to an even number.

While we were at it, /video/adaptive now applies the ffmpeg options you pass when it packages HLS and CMAF, such as hls_flags or start_number. Before, it silently ignored them.

iPhone videos with spatial audio

Videos recorded with spatial audio on recent iPhones carry two audio tracks: regular stereo AAC, and an Apple Positional Audio track that our encoding stack can’t decode. Until recently, that second track made the whole encode fail. /video/encode, 🤖 /audio/encode, and 🤖 /speech/transcribe now skip audio tracks they can’t decode and use the stereo track. You can encode the video, extract its audio, and transcribe an uploaded iPhone recording in one Assembly:

{
  "steps": {
    ":original": {
      "robot": "/upload/handle"
    },
    "video": {
      "robot": "/video/encode",
      "use": ":original",
      "preset": "web/mp4/240p",
      "ffmpeg_stack": "v7"
    },
    "audio": {
      "robot": "/audio/encode",
      "use": ":original",
      "preset": "aac",
      "ffmpeg_stack": "v7"
    },
    "transcript": {
      "robot": "/speech/transcribe",
      "use": ":original",
      "provider": "aws",
      "format": "text",
      "source_language": "en-US"
    }
  }
}

The spatial audio itself isn’t kept: the encoded result has the stereo mix. If the only audio track in a file is spatial audio, audio encoding and transcription still fail, because there is nothing we can decode.

Get started

All of this works with any plan. On the free Community plan, audio and video results are trimmed, so a transition or a later chapter may fall outside a trial result. The Robot docs for 🤖 /video/concat, 🤖 /video/subtitle, and 🤖 /video/adaptive list every parameter, and you can create a free account to try them.