Automatic spoken language detection with cURL & open source

Tim Koschützki

Co-founder · Berlin, Germany ·

Detect spoken language directly from audio with a multilingual Whisper model, served locally by whisper.cpp. cURL sends the recording to its HTTP endpoint, which returns the detected language and a transcript. Once the software and model are downloaded, inference runs on your computer without an account or cloud API.

System requirements

Use macOS or Linux with Bash, a C++17 compiler, Make, CMake 3.16 or newer, cURL 7.83 or newer, FFmpeg, and Python 3.9 or newer. Python only reads the JSON response; it needs no extra packages. The multilingual tiny model occupies about 75 MiB on disk. Allow additional disk space for the source and build, and several hundred megabytes of free memory for inference. This walkthrough uses a CPU build of whisper.cpp v1.8.7.

Installation

From a fresh working directory, download the pinned release, build its server, and download the multilingual model. Choose tiny, not the English-only tiny.en model:

set -e
curl --fail --location --output whisper-v1.8.7.tar.gz \
  https://github.com/ggml-org/whisper.cpp/archive/refs/tags/v1.8.7.tar.gz
tar -xzf whisper-v1.8.7.tar.gz
cd whisper.cpp-1.8.7
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DGGML_METAL=OFF -DWHISPER_BUILD_SERVER=ON
cmake --build build --config Release --target whisper-server -j 4
sh ./models/download-ggml-model.sh tiny

Setting up the transcription server

From the extracted directory, start the HTTP server:

./build/bin/whisper-server --model models/ggml-tiny.bin \
  --host 127.0.0.1 --port 8080 --language auto --no-gpu

Wait for the listening message, then leave this terminal open. The example server has no authentication, so keep it bound to loopback and use trusted local recordings. Stop it with Ctrl+C when you finish.

Transcribing audio with cURL

In a second terminal, work in a directory containing a spoken recording named input.mp3. Convert it to mono, 16 kHz, 16-bit PCM WAV, then upload it. -n prevents FFmpeg from overwriting an existing audio.wav; choose a fresh directory for each recording.

set -e
ffmpeg -nostdin -n -i input.mp3 -vn -ar 16000 -ac 1 -c:a pcm_s16le audio.wav
curl --fail --silent --show-error --remove-on-error --max-time 300 \
  http://127.0.0.1:8080/inference \
  --form 'file=@audio.wav;type=audio/wav' \
  --form 'language=auto' \
  --form 'response_format=verbose_json' \
  --form 'no_language_probabilities=true' \
  --output transcript.json

language=auto asks the speech model to identify the input language. verbose_json includes language, text, and timestamped segments; plain json does not include the language field. Disabling the optional language-probability report avoids a second detection pass. Translation is off, so the transcript remains in the detected language.

Language detection

Save this as read_language.py beside transcript.json. It validates and prints the language already inferred from the audio, rather than guessing a language from the transcript:

import json
import sys

def read_language(path):
    with open(path, encoding="utf-8") as handle:
        data = json.load(handle)
    if not isinstance(data, dict):
        raise ValueError("Expected a JSON object")
    language = data.get("language")
    text = data.get("text")
    if not isinstance(language, str) or not language.strip():
        raise ValueError("Response has no detected language")
    if not isinstance(text, str) or not text.strip():
        raise ValueError("No speech transcript; language result is inconclusive")
    return language

def main():
    if len(sys.argv) != 2:
        raise ValueError("Usage: python3 read_language.py transcript.json")
    print(f"Detected language: {read_language(sys.argv[1])}")

if __name__ == "__main__":
    try:
        main()
    except (OSError, ValueError) as error:
        print(f"Cannot read language result: {error}", file=sys.stderr)
        sys.exit(1)

After a successful upload, run:

python3 read_language.py transcript.json

For English speech, the language value is english; for Spanish, it is spanish. These are language names from the server, not ISO language codes. A nonempty transcript is a useful sanity check, but it does not prove that speech was present or that the language is correct.

Performance optimization

Keep the server running across requests so it can reuse the loaded model. Start with tiny for short experiments; the multilingual base or small models trade more memory and processing time for potential accuracy improvements. Download the chosen model with the same download script and restart the server with its matching model path.

Use a clear excerpt containing several sentences in one language. For a recording with language changes, assess excerpts separately: one file-level language result is not a timeline of every language spoken. Increase the cURL timeout deliberately if longer inputs need more time.

Error handling

The request block stops if conversion or upload fails. cURL exits nonzero for HTTP errors, connection failures, or timeouts, and --remove-on-error removes an incomplete response file. Only run the reader after that block succeeds. Invalid JSON, missing fields, an empty transcript, and unreadable files make the reader exit with status 1.

If startup fails, check the model path and whether port 8080 is already occupied. If an upload fails, inspect the local server terminal and confirm the WAV conversion succeeded. Do not treat a guessed language on silence, music, or very short speech as a reliable classification.

Tips and best practices

Check results against recordings whose spoken language you know, including different accents.
Evaluate the model on your own audio quality before using its output to route other work.
Keep uncertain results available for review instead of assuming every recording has one language.

For hosted transcription in a file-processing workflow, see our Speech Robot.