CLI subtitle workflow: generate, convert, and burn

Tim Koschützki

Co-founder · Berlin, Germany · Show bio

Give the script below a directory of MP4 videos with English speech. It produces timed SRT captions, a WebVTT copy, an MKV with a separate subtitle track, and an MP4 with captions painted into its frames. Whisper.cpp handles transcription on the CPU; FFmpeg handles the subtitle conversion and video outputs. Treat the results as drafts to review before publishing.

Choose soft or burned subtitles

Use the MKV for a selectable track and the burned MP4 when the text must remain visible without a subtitle switch. Neither choice guarantees that every player supports the resulting video. File size depends on the source and encoding settings; burning text does not always increase it.

Install the CPU transcription tools

Use a Linux system with Bash and permission to install packages. These commands target Ubuntu 24.04 and use Whisper.cpp’s v1.9.4 release⁠. The English-only tiny.en model keeps this example small; it is not an accuracy guarantee. You need internet access for installation and the model download, then transcription runs locally.

Install FFmpeg, a C++ compiler, CMake, and a font for rendering:

(
  sudo apt-get update &&
  sudo apt-get install -y build-essential cmake curl ca-certificates tar ffmpeg fonts-liberation
)

From a writable working directory, create subtitle-tools. The subshell keeps your current directory and shell options unchanged. If that directory already exists, setup stops before installing or writing inside it; choose another working directory instead of deleting existing work.

(
  mkdir subtitle-tools &&
  cd subtitle-tools &&
  curl -fsSLo whisper.tar.gz https://github.com/ggml-org/whisper.cpp/archive/refs/tags/v1.9.4.tar.gz &&
  mkdir source &&
  tar -xzf whisper.tar.gz -C source --strip-components=1 &&
  cmake -S source -B build -DCMAKE_BUILD_TYPE=Release \
    -DGGML_CUDA=OFF -DGGML_VULKAN=OFF -DGGML_NATIVE=OFF \
    -DBUILD_SHARED_LIBS=OFF -DWHISPER_BUILD_TESTS=OFF &&
  cmake --build build --target whisper-cli -j2 &&
  curl -fsSLo ggml-tiny.en.bin \
    https://huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-tiny.en.bin &&
  printf '%s  %s\n' \
    921e4cf8686fdd993dcd081a5da5b6c365bfde1162e72b08d75ac75289920b1f \
    ggml-tiny.en.bin | sha256sum -c -
)

This uses Whisper.cpp’s CMake build and audio preparation workflow⁠. Its CLI accepts the mono, 16 kHz, 16-bit PCM WAV that the script extracts from each video. Other Whisper wrappers, including SubsAI, have their own command syntax; they are not needed here.

To try the workflow without supplying a recording, make a video from the release’s bundled JFK speech sample. Run this from the same working directory as setup:

(
  mkdir demo-videos &&
  ffmpeg -nostdin -n -v error -f lavfi -i color=c=black:s=640x360:r=24 \
    -i ./subtitle-tools/source/samples/jfk.wav \
    -map 0:v:0 -map 1:a:0 -c:v libx264 -threads 2 -c:a aac -shortest \
    ./demo-videos/example.mp4
)

Automate everything with a shell script

Save the following as subtitle-tools/subtitles.sh. It processes only top-level regular files with the lowercase .mp4 suffix and skips symlink entries. Each input gets a new adjacent directory, such as example.mp4.subtitles. Spaces, quotes, and leading hyphens in filenames are supported; paths containing newline characters are rejected. Run one batch at a time.

The fixed filenames inside each result directory keep the original path out of FFmpeg’s separate filter syntax. The script keeps a source copy and a PCM audio file, so allow disk space for those as well as both video outputs.

#!/usr/bin/env bash
set -euo pipefail
export LC_ALL=C

fail() { printf '%s\n' "$*" >&2; exit 1; }
[[ $# -eq 1 ]] || fail 'Usage: bash subtitles.sh VIDEO_DIRECTORY'
[[ $PWD != *$'\n'* && $1 != *$'\n'* && ${BASH_SOURCE[0]} != *$'\n'* ]] ||
  fail 'Newline characters in paths are not supported'

input_directory=$1
[[ $input_directory == /* ]] || input_directory=$PWD/$input_directory
case ${BASH_SOURCE[0]} in
  */*) tool_directory=${BASH_SOURCE[0]%/*} ;;
  *) tool_directory=. ;;
esac
cd -P -- "$tool_directory"
tool_directory=$PWD
[[ $tool_directory != *$'\n'* ]] || fail 'Newline characters in paths are not supported'
whisper=$tool_directory/build/bin/whisper-cli
model=$tool_directory/ggml-tiny.en.bin
[[ -x $whisper && -s $model ]] || fail 'Complete the subtitle-tools setup first'
cd -P -- "$input_directory"
[[ $PWD != *$'\n'* ]] || fail 'Newline characters in paths are not supported'
shopt -s nullglob

process_video() (
  local video=$1
  local output=$video.subtitles
  mkdir -- "$output"
  cp -- "$video" "$output/source.mp4"
  cd -- "$output"
  ffmpeg -nostdin -n -v error -xerror -i source.mp4 -map 0:a:0 \
    -ar 16000 -ac 1 -c:a pcm_s16le audio.wav
  "$whisper" --no-gpu -t 2 -p 1 -l en -m "$model" -f audio.wav -osrt -of captions
  [[ -s captions.srt ]] || fail 'No subtitle file was produced'
  grep -q -- '-->' captions.srt || fail 'No timed captions were produced; check for speech'
  ffmpeg -nostdin -n -v error -i captions.srt -map 0:s:0 -c:s webvtt captions.vtt
  ffmpeg -nostdin -n -v error -i source.mp4 -i captions.srt \
    -map 0:v:0 -map 0:a:0 -map 1:s:0 -c copy -metadata:s:s:0 language=eng soft.mkv
  ffmpeg -nostdin -n -v error -xerror -i source.mp4 \
    -vf "subtitles=captions.srt:force_style='Fontname=Liberation Sans,FontSize=24'" \
    -map 0:v:0 -map 0:a:0 -c:v libx264 -threads 2 -crf 20 -preset veryfast \
    -c:a copy burned.mp4
)

count=0
current_video='setup'
trap 'status=$?; if [[ $status -ne 0 ]]; then printf "Failed while processing: %s\n" "$current_video" >&2; fi' EXIT
for video in "$PWD"/*.mp4; do
  [[ -f $video && ! -L $video ]] || continue
  [[ $video != *$'\n'* ]] || fail 'Newline characters in paths are not supported'
  current_video=$video
  printf 'Processing: %s\n' "$video"
  process_video "$video"
  count=$((count + 1))
  printf 'Created: %s.subtitles\n' "$video"
done
[[ $count -gt 0 ]] || fail 'No regular .mp4 files found'

Run it from the working directory where you installed the tools, which is different from the input directory:

bash ./subtitle-tools/subtitles.sh ./demo-videos

Replace ./demo-videos with your directory of recordings. The script resolves that directory before selecting files; it does not process MP4s from your caller’s directory. You can also pass absolute paths to both the saved script and the input directory.

An empty directory returns an error. A video without an audio stream, a decoding error, or a failed later conversion stops the batch and names the current input. A failure can leave partial files in its result directory. Existing result directories cause an error before that clip is copied or transcribed, so repeat runs preserve previous outputs. This is not a resume script: to retry one corrected clip, put it in a separate directory with no existing results.

Silent audio is not the same as a missing audio stream. The model can return no cues, non-speech labels, or invented text. For example, tiny.en can produce a [BLANK_AUDIO] cue that extends beyond the recording. The script checks for timed cues; it does not validate their words or timing. Listen to the audio and check cue endings against the video’s duration before publishing.

FFmpeg can also recover some damaged media rather than report an error. Successful commands do not certify an intact input or an accurate transcript.

Check the SRT and WebVTT captions

In demo-videos/example.mp4.subtitles, open captions.srt and captions.vtt. The SRT has numbered cues with comma-separated milliseconds; the VTT begins with WEBVTT and uses decimal points in its timestamps. Both should contain the sample’s “ask not what your country can do for you” and “ask what you can do for your country” phrases. Check the words against the audio, especially names, punctuation, and the start and end of each cue.

The conversion in the script uses FFmpeg’s webvtt subtitle encoder. You do not need Subtitle Edit CLI for this SRT-to-WebVTT path. A WebVTT sidecar is ready for a compatible web player’s caption track; generating the file alone does not wire it into a website.

Check the selectable subtitle track

Open soft.mkv in a player with Matroska and SRT subtitle support, then select its English subtitle track. The script explicitly maps the first video stream, first audio stream, and generated captions. FFmpeg’s -c copy streamcopy mode⁠ copies those streams into the MKV without encoding the video. This performs the muxing job without an additional MKVToolNix installation.

Burn (hard-code) subtitles with FFmpeg

Open burned.mp4 without enabling an external subtitle file. Text should already appear in the video image. FFmpeg’s libass-backed subtitles filter⁠ renders the SRT, and libx264 encodes the changed frames. That is why burning requires video encoding while adding a soft track does not. The audio is copied in both outputs.

For the black-background sample, you can also decode a frame at two seconds. The new preview.png should contain visible white text near the bottom; the source video at that time is plain black:

(
  cd ./demo-videos/example.mp4.subtitles || exit
  [[ ! -e preview.png && ! -L preview.png ]] || { printf 'preview.png already exists\n' >&2; exit 1; }
  ffmpeg -nostdin -n -v error -ss 2 -i burned.mp4 -frames:v 1 -threads 2 preview.png
)

The -n option⁠ refuses to replace an existing preview. Editing captions.srt afterward does not change either video or the VTT you already generated. Keep corrected captions and render new outputs before publishing them.

Performance and scaling tips

Measure transcription and burn-in separately on your own recordings. This example deliberately uses two CPU threads and no GPU; it does not predict processing speed on other hardware. If the draft misses speech, compare a larger English model on a representative clip and review its output too. A larger model is not a substitute for checking the transcript.

Keep the reviewed SRT with the recording so you can change the delivery format without transcribing again. The batch script always transcribes new result directories; it does not automatically reuse your corrections.

Wrap-up

If you already have reviewed captions and want managed subtitle burn-in, our 🤖 /video/subtitle Robot accepts existing SRT or WebVTT subtitles. It does not generate the transcript. Upload the video in the input_video form field and the subtitle file in input_srt; these Assembly Instructions keep those inputs separate:

{
  "steps": {
    "subtitle_video": {
      "robot": "/video/subtitle",
      "use": {
        "steps": [
          { "name": ":original", "fields": "input_video", "as": "video" },
          { "name": ":original", "fields": "input_srt", "as": "subtitles" }
        ]
      },
      "subtitles_type": "burned",
      "font_size": 24,
      "position": "bottom",
      "font_color": "FFFFFF",
      "border_style": "outline"
    }
  }
}