How to read the numbers
Four measurements per Assembly
- Size reduction
- 1 − output bytes ÷ input bytes. 0% means the file did not change, 50% means it is half the size, and a negative value would mean the output grew.
- SSIM
- Structural similarity between the result and its source, from 0 to 1. A score of 1.000 means the pixels are identical, so the step was lossless. Lower scores mean more visible change.
- Robot time
- Seconds the Robot spent executing the Step, as reported by the API for each result. It excludes the upload and time spent waiting in a queue.
- Assembly duration
- The execution duration the API reports for the whole Assembly. On top of Robot time it covers handling the upload, reading the result’s metadata, and storing the result.
Finding 1
Optimizing a file in its own format depends on the parameters
The Image Optimize Robot keeps the file format and runs one or more optimizers, then returns the smallest result. For JPEG, the default settings produced a median reduction of 84.7% at a median SSIM of 0.973: the Robot re-encoded the image. With metadata removed, the same priority only applied lossless optimizers, so the pixels stayed identical and the median reduction was 5.5%. Choosing conversion-speed gave 4.6% in 1.26 sec of Robot time instead of 2.37 sec.
PNG behaved the other way around. Lossless optimization saved a median of 8.9%, and PNG was the slowest format in the benchmark at 10.70 sec of Robot time. Allowing lossy optimization and removing metadata saved 60.3% at a median SSIM of 0.997, in 1.96 sec of Robot time. WebP inputs were re-encoded, for a median reduction of 32.9% at a median SSIM of 0.964.
| Source | Parameters | Median size reduction | Median SSIM | Median Robot time | 90th percentile Robot time | Median Assembly duration |
|---|---|---|---|---|---|---|
| JPEG | priority: compression-ratiopreserve_meta_data: true | 84.7% | 0.973 | 2.33 sec | 2.65 sec | 3.35 sec |
| JPEG | priority: compression-ratiopreserve_meta_data: false | 5.5% | 1.000 | 2.37 sec | 2.82 sec | 3.64 sec |
| JPEG | priority: conversion-speedpreserve_meta_data: true | 4.2% | 1.000 | 1.21 sec | 1.54 sec | 2.36 sec |
| JPEG | priority: conversion-speedpreserve_meta_data: false | 4.6% | 1.000 | 1.26 sec | 1.57 sec | 2.18 sec |
| PNG | lossy: falsepreserve_meta_data: true | 8.9% | 1.000 | 10.70 sec | 26.17 sec | 11.64 sec |
| PNG | lossy: falsepreserve_meta_data: false | 8.9% | 1.000 | 10.70 sec | 25.90 sec | 11.48 sec |
| PNG | lossy: truepreserve_meta_data: true | 8.9% | 1.000 | 10.74 sec | 26.11 sec | 11.45 sec |
| PNG | lossy: truepreserve_meta_data: false | 60.3% | 0.997 | 1.96 sec | 2.45 sec | 2.91 sec |
| GIF | preserve_meta_data: true | 37.4% | Not scored | 2.08 sec | 3.94 sec | 2.89 sec |
| WebP | preserve_meta_data: true | 32.9% | 0.964 | 0.93 sec | 2.54 sec | 1.81 sec |
| WebP | preserve_meta_data: false | 33.1% | 0.964 | 1.01 sec | 2.71 sec | 1.72 sec |
| SVG | preserve_meta_data: true | 47.3% | Not scored | 1.32 sec | 2.06 sec | 2.17 sec |
| SVG | preserve_meta_data: false | 47.3% | Not scored | 1.60 sec | 2.03 sec | 2.55 sec |
Six source files per format. Animated GIF and SVG results are not scored because they have no single raster frame to compare.
Finding 2
WebP and AVIF reach the same similarity as JPEG in fewer bytes
Starting from lossless PNG photographs, the Image Resize Robot wrote JPEG, WebP, and AVIF at three quality settings without changing the dimensions. At quality 75, WebP needed 25.7% fewer bytes than JPEG while scoring a median SSIM of 0.951 against 0.951. AVIF at quality 75 needed 31.0% fewer bytes than JPEG at quality 90, at a median SSIM of 0.978 against 0.976. Quality settings are not comparable across formats, so compare rows by SSIM and size together.
| Output | Quality | Median output size | Reduction from PNG source | Median SSIM | Lowest SSIM | Median Robot time | Median Assembly duration |
|---|---|---|---|---|---|---|---|
| JPEG | 50 | 38 kB | 93.8% | 0.923 | 0.899 | 0.48 sec | 1.17 sec |
| JPEG | 75 | 60 kB | 90.5% | 0.951 | 0.938 | 0.48 sec | 1.14 sec |
| JPEG | 90 | 107 kB | 83.2% | 0.976 | 0.967 | 0.49 sec | 1.14 sec |
| WebP | 50 | 30 kB | 95.1% | 0.930 | 0.898 | 0.60 sec | 1.24 sec |
| WebP | 75 | 43 kB | 93.0% | 0.951 | 0.932 | 0.48 sec | 1.21 sec |
| WebP | 90 | 94 kB | 84.9% | 0.980 | 0.967 | 0.58 sec | 1.24 sec |
| AVIF | 50 | 28 kB | 95.4% | 0.942 | 0.919 | 0.99 sec | 1.71 sec |
| AVIF | 75 | 68 kB | 88.9% | 0.978 | 0.959 | 1.18 sec | 1.79 sec |
| AVIF | 90 | 148 kB | 76.4% | 0.992 | 0.987 | 1.42 sec | 2.14 sec |
24 source images of 768 × 512 pixels with a median size of 620 kB. Reductions are relative to a lossless source, so they are larger than what re-compressing an existing JPEG would yield.
Method
How the benchmark was run
Source files
The optimization study uses 30 files from Wikimedia Commons: for each of JPEG, PNG, GIF, WebP, and SVG, the first six search results inside a fixed file-size band that Commons labels CC0 or public domain. The re-encoding study uses the 24 lossless PNG photographs of the Kodak image set, a common reference set for codec comparisons. The corpus manifest lists every file with its URL, license, dimensions, byte size, and SHA-256 checksum, together with the exact search queries, so the same bytes can be fetched again.
Assemblies
Every measurement is one Assembly that uploads one file and runs one Step. The optimization study varies priority and metadata handling for JPEG, lossy optimization and metadata handling for PNG, and metadata handling for WebP and SVG. Parameters that the Robot ignores for a format are left out, because they would repeat an identical job. The upload Step sets output_meta to false: no Step here reads the metadata of the uploaded file, and extracting it would add to every Assembly duration without changing any result. These are the Assembly Instructions, with the varied parameters shown at one of their values:
{
"steps": {
":original": {
"robot": "/upload/handle",
"output_meta": false
},
"measured": {
"robot": "/image/optimize",
"use": ":original",
"priority": "compression-ratio",
"preserve_meta_data": true
}
}
}{
"steps": {
":original": {
"robot": "/upload/handle",
"output_meta": false
},
"measured": {
"robot": "/image/resize",
"use": ":original",
"format": "webp",
"quality": 75,
"strip": true
}
}
}Runs and environment
All Assemblies ran on October 17, 2026 against the production API at api2.transloadit.com, on the same shared machines that serve customer traffic. The full matrix of 294 file and parameter combinations was executed 3 times in sequence, so repeats of one combination are spread over the session, with 3 Assemblies in flight at a time.
Measurements and aggregation
Sizes and timings come from the Assembly Status: the upload size, the result size, the result’s Robot execution and queue time, and the Assembly execution duration. For SSIM, the result of the first run is downloaded, both images are flattened onto white and converted to 8-bit luma, and structural similarity is averaged over 8 × 8 pixel windows at a stride of 4 pixels. Output bytes were identical across the repeated runs of every combination, so only timings vary between repeats. Each table row first takes the median of the repeated runs per file, then the median across files. The 90th percentile is taken over all runs in the row.
Read before generalizing
What this benchmark cannot tell you
- Six files per format is a small sample, and search results from Wikimedia Commons are not a sample of production traffic. The JPEGs are large, carefully exported photographs and the GIFs are mostly diagrams.
- Three runs per combination show the spread of timings but are too few for confidence intervals, and all runs happened on a single day.
- SSIM on luma is a structural metric, not a model of human perception, and it ignores color. Animated GIF and SVG results were not scored at all.
- The re-encoding study uses small images. Robot time grows with pixel count, so these timings do not describe multi-megapixel photographs.
- The Assembly Status does not expose machine type or optimizer versions, so they are not part of the dataset. Results describe the production stack on the run date.