Optimizing image processing in Rust with parallelism and Rayon

Tim Koschützki

Co-founder · Berlin, Germany · Show bio

Use Rayon to process separate images concurrently, with one decode–resize–encode job per worker. This walkthrough builds a Rust CLI that converts a directory of PNG and JPEG images into smaller JPEGs. You choose the worker count, and the command reports every file failure before returning its final status.

Choose the image and output policies

The example is for local batches of still images, with these choices:

• Select regular files ending in .png, .jpg, or .jpeg, ignoring extension case. Skip subdirectories, symlinks, and other extensions. Do not modify the input directory during a run.
• Accept images decoded to 8-bit RGB, grayscale, or either with alpha. Use sRGB inputs: the program does not convert embedded color profiles. Animation and color-managed print workflows are outside this example.
• Apply the decoder’s EXIF orientation, flatten transparency onto white, then fit within 800 × 600 pixels without cropping or enlarging smaller images. Encode JPEG at quality 85.
• Append .jpg to the entire input filename: photo.png becomes photo.png.jpg, and photo.jpg becomes photo.jpg.jpg. This keeps same-stem inputs distinct.
• Require a new output directory. An existing directory, even an empty one, is an error. Successful files remain when another file fails; a rerun needs a different output directory.

The encoder receives pixels only, so source EXIF, GPS, ICC, and text metadata are not copied. The orientation is applied to the pixels before that metadata is discarded. Alpha compositing and resizing operate on encoded channel values here, not linear-light values. That is a deliberate simplification for thumbnails, not a color-management pipeline.

Create the Cargo project

The commands below use Bash on Linux, tested with Rust and Cargo 1.98.1. Install the current stable toolchain using the official Rust installation guide⁠. Use a working directory outside an existing Cargo project and its .cargo configuration. All shell blocks below are pasted from that same working directory.

rustc --version && cargo --version

Create a new directory; this refuses to reuse an existing image-batch:

(mkdir image-batch && mkdir image-batch/src)

Save this complete manifest as image-batch/Cargo.toml:

[package]
name = "image-batch"
version = "0.1.0"
edition = "2024"

[dependencies]
anyhow = "=1.0.104"
image = { version = "=0.25.10", default-features = false, features = ["jpeg", "png"] }
rayon = "=1.12.0"

[workspace]

Only PNG and JPEG codecs are enabled. Rayon supplies the batch parallelism; the image crate’s optional Rayon feature is disabled. The image release notes⁠ describe the current decoding and orientation APIs used here.

Implement the complete batch command

Save the following as image-batch/src/main.rs. Directory discovery finishes before the output directory is created. Rust’s read_dir⁠ can fail both when opening a directory and while advancing its iterator, so both errors propagate instead of silently shrinking the batch.

use std::{env, fs, path::Path, process::ExitCode};

use anyhow::{Context, Result, ensure};
use image::{ColorType, DynamicImage, ImageDecoder, ImageReader, Limits, Rgb, RgbImage};
use image::codecs::jpeg::JpegEncoder;
use image::imageops::FilterType;
use rayon::prelude::*;

fn convert(input: &Path, output_dir: &Path) -> Result<()> {
    let mut reader = ImageReader::open(input)?.with_guessed_format()?;
    let mut limits = Limits::default();
    limits.max_image_width = Some(4096);
    limits.max_image_height = Some(4096);
    limits.max_alloc = Some(128 * 1024 * 1024);
    reader.limits(limits);

    let mut decoder = reader.into_decoder().context("Read image header")?;
    ensure!(
        matches!(decoder.color_type(), ColorType::L8 | ColorType::La8 | ColorType::Rgb8 | ColorType::Rgba8),
        "Expected an 8-bit image"
    );
    let orientation = decoder.orientation()?;
    let mut decoded = DynamicImage::from_decoder(decoder).context("Decode pixels")?;
    decoded.apply_orientation(orientation);

    let rgba = decoded.into_rgba8();
    let rgb = RgbImage::from_fn(rgba.width(), rgba.height(), |x, y| {
        let pixel = rgba.get_pixel(x, y);
        let alpha = u16::from(pixel[3]);
        let blend = |channel: u8| {
            ((u16::from(channel) * alpha + 255 * (255 - alpha) + 127) / 255) as u8
        };
        Rgb([blend(pixel[0]), blend(pixel[1]), blend(pixel[2])])
    });
    drop(rgba);
    let image = DynamicImage::ImageRgb8(rgb);
    let resized = if image.width() > 800 || image.height() > 600 {
        image.resize(800, 600, FilterType::Lanczos3)
    } else {
        image
    };

    let mut bytes = Vec::new();
    JpegEncoder::new_with_quality(&mut bytes, 85).encode_image(&resized)?;
    let mut name = input.file_name().context("Missing filename")?.to_os_string();
    name.push(".jpg");
    let output = output_dir.join(&name);
    name.push(".part");
    let staging = output_dir.join(name);
    fs::write(&staging, bytes).context("Write staged JPEG")?;
    fs::rename(&staging, &output).context("Publish JPEG")?;
    Ok(())
}

fn run() -> Result<()> {
    let args: Vec<_> = env::args_os().skip(1).collect();
    ensure!(args.len() == 3, "Usage: image-batch INPUT_DIR NEW_OUTPUT_DIR THREADS");
    let threads: usize = args[2].to_str().context("Invalid thread count")?.parse()?;
    ensure!((1..=16).contains(&threads), "THREADS must be between 1 and 16");

    let input_dir = fs::canonicalize(&args[0]).context("Resolve input directory")?;
    let mut inputs = Vec::new();
    for entry in fs::read_dir(&input_dir).context("Open input directory")? {
        let entry = entry.context("Read directory entry")?;
        if !entry.file_type().context("Read entry type")?.is_file() {
            continue;
        }
        let path = entry.path();
        let supported = path.extension().is_some_and(|ext| {
            ext.eq_ignore_ascii_case("png")
                || ext.eq_ignore_ascii_case("jpg")
                || ext.eq_ignore_ascii_case("jpeg")
        });
        if supported {
            inputs.push(path);
        }
    }
    ensure!(!inputs.is_empty(), "No PNG or JPEG files found");
    inputs.sort();

    let pool = rayon::ThreadPoolBuilder::new().num_threads(threads).build()?;
    let output_dir = Path::new(&args[1]);
    fs::create_dir(output_dir).context("Create a new output directory; parent must exist")?;
    let failed: usize = pool.install(|| {
        inputs.par_iter().map(|input| {
            match convert(input, output_dir) {
                Ok(()) => 0,
                Err(error) => {
                    eprintln!("FAILED {}: {error:#}", input.display());
                    1
                }
            }
        }).sum()
    });
    println!("{} succeeded; {failed} failed", inputs.len() - failed);
    ensure!(failed == 0, "Batch incomplete; successful outputs were kept");
    Ok(())
}

fn main() -> ExitCode {
    match run() {
        Ok(()) => ExitCode::SUCCESS,
        Err(error) => {
            eprintln!("{error:#}");
            ExitCode::FAILURE
        }
    }
}

Each job encodes before writing a .part file, then renames that file to its final JPEG name. Only a completed rename counts as success. A write failure or interruption may leave .part files; they are incomplete work, not published images. Keep this output directory exclusive to the run. The example does not synchronize files to durable storage or roll back successful files.

The parallel iterator sums failures instead of stopping at the first error. This attempts every discovered image on an ordinary per-file error, while retaining a nonzero process status for an incomplete batch. Error messages can appear in any order.

Build, run, and inspect an output

Generate image-batch/Cargo.lock once, and keep it with the project. Exact direct dependency pins alone do not freeze transitive dependencies; subsequent builds use the retained lock.

(cd image-batch && cargo generate-lockfile)

For a reproducible input, install ImageMagick⁠, tested here with 7.1.2-31. This creates a 1,600 × 800 PNG with an opaque red left half and a transparent right half. Use new directory names for the example:

(
  mkdir input-images &&
  magick -size 1600x800 xc:none -fill red -draw 'rectangle 0,0 799,799' \
    PNG32:input-images/sample.png
)

Build in release mode and run with two workers. The command runs the executable only after the build succeeds:

(
  cd image-batch &&
  cargo build --release --locked &&
  ./target/release/image-batch ../input-images ../output-images 2
)

For this single input, the program reports 1 succeeded; 0 failed. Inspect the actual produced file with an independent decoder:

magick ./output-images/sample.png.jpg -format '%m %wx%h\n' info:

JPEG 800x400

Open that JPEG too: the left half should be red and the right half white, including the bottom edge. resize⁠ preserves aspect ratio within the box; it does not force this image to 800 × 600. The explicit size check keeps a smaller input at its original dimensions. Flattening before resizing prevents hidden RGB values in transparent pixels from bleeding into the white background.

To process your own images, replace the two directory arguments and keep the output directory new. Quote paths containing spaces. A corrupt selected file is reported if the decoder rejects it; unsupported extensions are skipped. A successful decode is not an integrity check: with this decoder, a truncated JPEG can return success with gray blocks replacing missing pixels. Inspect image content across the full canvas, and retain your originals.

Measure worker count against memory and elapsed time

num_threads⁠ bounds this pool’s workers. The CLI accepts one through 16; begin with one or two for large inputs. It stores the directory’s paths, then holds decoded pixels only for active jobs. Each job can need several pixel buffers, so doubling workers can substantially increase memory use.

The 4,096-pixel width and height limits apply to the source before orientation and resizing. The 128 MiB decoder allocation limit is best effort⁠, and does not cover the later RGBA/RGB buffers, resizing, JPEG encoding, path list, or total process memory. These controls do not make the program a sandbox for untrusted uploads.

After a successful build, compare the same representative multi-image input with fresh destinations:

time ./image-batch/target/release/image-batch ./input-images ./timed-one 1 &&
time ./image-batch/target/release/image-batch ./input-images ./timed-two 2

Compare Bash’s real elapsed time, repeat with new output directory names, and account for warm filesystem caches. The one-image smoke test cannot demonstrate batch speedup. More workers may help when independent decoding and resizing keep CPU cores busy; storage, memory pressure, and scheduling overhead can erase the gain. Keep Lanczos3 if its result suits your images; compare a different filter with the same inputs before trading image quality for elapsed time.