[Research](/research.md)/Image optimization

Benchmark report · Dataset published August 3, 2022

# Image optimization: speed, size, and quality

We re-analyzed the public measurements behind Transloadit’s image optimization experiment. The data covers 136 recorded Assembly runs and makes the tradeoffs between compression, speed, metadata, format, and JPEG quality inspectable.

Researcher: Joseph GrabskiAnalysis: TransloaditStatus: Exploratory

## Results at a glance

Measurements

136

Recorded rows across the three public CSV files.

WebP size reduction

81.2%

Average reduction for two WebP inputs across eight test combinations.

Quality 25 reduction

77.6%

Average reduction across ten source files encoded at JPEG quality 25.

Priority difference

18.9 points

More average reduction from compression-ratio when metadata was retained.

Finding 1

## WebP inputs had the largest observed mean reduction

Across priority and metadata settings, the two WebP inputs averaged a 81.2% reduction. The JPEG inputs averaged 12.3%. These are averages for this small corpus, not expected savings for every file of that type.

InputMean size reductionMean execution

WebP

81.2%

2.62 sec

GIF

27.1%

1.55 sec

SVG

18.2%

9.34 sec

PNG

17.1%

14.17 sec

JPEG

12.3%

4.14 sec

Finding 2

## The priority tradeoff appeared when metadata was retained

With metadata retention enabled, compression-ratio averaged a 44.6% reduction and 6.64 sec of execution. Conversion-speed averaged a 25.6% reduction and 5.72 sec. With metadata removed, the average size-reduction difference narrowed to 0.3 percentage points.

Compression-ratio

44.6%

mean reduction · metadata kept

Conversion-speed

25.6%

mean reduction · metadata kept

Finding 3

## Output size varied sharply across JPEG quality settings

Quality 25 reduced size by 77.6% on average, quality 50 by 60.6%, and quality 75 by 31.5%. At quality 100, outputs were 126.4% larger than inputs on average. Mean execution ranged from 2.02 sec to 3.14 sec, but the single-run timings are too noisy to establish a clean speed trend.

Quality 25

77.6%

Quality 50

60.6%

Quality 75

31.5%

Quality 100

-126.4%

Method

## How the measurements were collected and recalculated

The source repository contains 12 test images and 3 Python scripts. Each script creates a Transloadit Assembly, waits for completion, and records input size, output size, and the Assembly execution\_duration value.

The format dataset contains 48 runs: 12 files multiplied by 2 priorities and 2 metadata settings. The quality dataset contains 40 runs: 10 files multiplied by 4 JPEG quality values. The factorial dataset contains 48 runs across 4 images, 3 quality values, 2 priorities, and 2 metadata settings.

For this report, mean size reduction is calculated as 100 × (1 − output size ÷ input size). Positive values mean a smaller output; negative values mean the output grew. Means are unweighted across rows, so every run contributes equally regardless of input size.

Read before generalizing

## This is an exploratory historical benchmark

* The corpus contains twelve files, and the source does not document a sampling method for representing production image traffic.
* Each configuration appears once per source file, so timing variance and confidence intervals cannot be estimated.
* The CSV files do not record run date, region, Robot stack version, hardware, or Assembly identifiers.
* The results describe the 2022 experiment. They should not be treated as a benchmark of current Transloadit stacks.
* No perceptual metric such as SSIM or VMAF was recorded, so smaller files cannot be ranked by visual fidelity from this dataset alone.

## Open materials

* [Source repository⁠](https://github.com/joegrabski/OptimizeTester)
* [Format and priority CSV⁠](https://github.com/joegrabski/OptimizeTester/raw/master/no-quality.csv)
* [Quality CSV⁠](https://github.com/joegrabski/OptimizeTester/raw/master/quality.csv)
* [Factorial CSV⁠](https://github.com/joegrabski/OptimizeTester/raw/master/data.csv)
* [Original experiment notes](/blog/2022/08/benchmark-image-optimize.md)

## Suggested citation

Grabski, Joseph. “Image optimization: speed, size, and quality.” Transloadit Research, 2022.

[https://transloadit.com/research/image-optimization/](https://transloadit.com/research/image-optimization.md)

## Run your own workload

Use the same Robots with inputs that represent your product and quality bar.

[Image Optimize Robot](/docs/robots/image-optimize.md)[Image Resize Robot](/docs/robots/image-resize.md)