Build a local image origin with Sharp and Redis

Tim Koschützki

Co-founder · Berlin, Germany ·

Serve a resized JPEG or WebP from the same image URL without mixing their cached bytes. This guide builds a local image origin, generates a transparent sample, and checks both cold and warm responses. The origin is one component of a custom image CDN; global edge delivery needs a separate CDN.

Prerequisites

Use a maintained Node.js 24 LTS release, at least 24.15.0, Corepack with Yarn 4, Docker, and cURL. The example uses Express 5.2.1, Sharp 0.35.4, the Redis client 6.2.1, and Redis 8.10.2. Node’s native TypeScript support runs the .mts files directly, without a compiler or tsx.

Only publish reviewed, immutable, single-frame JPEG or PNG sources. This origin has no upload or authentication route: every listed image and derivative is public. The byte and pixel limits below reduce accepted work; they do not sandbox Sharp’s native decoder.

Set up the project

Create a new, empty image-origin directory and open a terminal there. Save this package.json:

{
  "name": "image-origin",
  "private": true,
  "type": "module",
  "packageManager": "yarn@4.12.0",
  "scripts": { "fixture": "node create-fixture.mts", "start": "node server.mts" },
  "dependencies": { "express": "5.2.1", "redis": "6.2.1", "sharp": "0.35.4" },
  "devDependencies": { "@types/express": "5.0.6" }
}

Save .yarnrc.yml too. It selects a local node_modules installation and prevents an enclosing project’s Yarn executable from taking over:

nodeLinker: node-modules
enableGlobalCache: false
ignorePath: true

Create the project’s lockfile boundary before installing. Run every command below from this directory:

touch yarn.lock && corepack yarn install

Keep the generated yarn.lock; repeat installations can use corepack yarn install --immutable. Save create-fixture.mts to generate a 640 × 400 sample: an opaque red left half and a transparent right half. It refuses to replace an existing source file.

import { mkdir, writeFile } from 'node:fs/promises'
import sharp from 'sharp'

const width = 640
const height = 400
const pixels = Buffer.alloc(width * height * 4)
for (let y = 0; y < height; y += 1) {
  for (let x = 0; x < width / 2; x += 1) {
    const offset = (y * width + x) * 4
    pixels[offset] = 255
    pixels[offset + 3] = 255
  }
}
const png = await sharp(pixels, { raw: { width, height, channels: 4 } }).png().toBuffer()
await mkdir('images', { recursive: true })
await writeFile('images/photo-v1.png', png, { flag: 'wx' })

Generate it, then start a disposable Redis cache bound to loopback. If port 6379 is occupied, choose another host port and supply its URL through REDIS_URL when starting the origin.

corepack yarn fixture &&
docker run --detach --rm --name image-origin-redis \
  -p 127.0.0.1:6379:6379 redis:8.10.2 \
  redis-server --maxmemory 128mb --maxmemory-policy allkeys-lru --save '' --appendonly no

The cache is disposable: this configuration writes no Redis snapshot or append-only file. Originals remain in images/. Redis’s eviction policy removes cached entries when needed; a miss can regenerate them.

Build a minimal image optimizer

Save optimizer.mts. Its caller supplies a filename from the publication map, rather than a URL or user-selected path. It accepts at most 8 MiB of source-file bytes and 12 million input pixels. The directory must remain under the publisher’s control while requests run.

import { open, realpath } from 'node:fs/promises'
import { resolve, sep } from 'node:path'
import sharp from 'sharp'

export type Format = 'jpeg' | 'webp'
const MAX_BYTES = 8 * 1024 * 1024
const MAX_PIXELS = 12_000_000
sharp.concurrency(1)
sharp.cache(false)

export async function optimize(filename: string, size: number, format: Format): Promise<Buffer> {
  const root = await realpath(resolve('images'))
  const path = await realpath(resolve(root, filename))
  if (!path.startsWith(root + sep)) throw new Error('Source is outside the publishing directory.')
  const handle = await open(path, 'r')
  let data: Buffer
  try {
    const stat = await handle.stat()
    if (!stat.isFile() || stat.size === 0 || stat.size > MAX_BYTES) {
      throw new Error('Source size is unsupported.')
    }
    data = Buffer.alloc(MAX_BYTES + 1)
    let length = 0
    while (length < data.length) {
      const { bytesRead } = await handle.read(data, length, data.length - length, null)
      if (bytesRead === 0) break
      length += bytesRead
    }
    if (length === 0 || length > MAX_BYTES) throw new Error('Source size is unsupported.')
    data = data.subarray(0, length)
  } finally {
    await handle.close()
  }
  const pipeline = sharp(data, { limitInputPixels: MAX_PIXELS, failOn: 'warning' })
  const metadata = await pipeline.metadata()
  if (!metadata.format || !['jpeg', 'png'].includes(metadata.format) ||
      (metadata.pages ?? 1) !== 1) {
    throw new Error('Only single-frame JPEG and PNG sources are supported.')
  }
  pipeline.rotate().resize({ width: size, height: size, fit: 'inside', withoutEnlargement: true })
  return format === 'jpeg'
    ? pipeline.flatten({ background: 'white' }).jpeg({ quality: 80 }).toBuffer()
    : pipeline.webp({ quality: 80 }).toBuffer()
}

The image fits inside the requested square, preserving its aspect ratio without enlargement. JPEG composites transparency onto white; WebP preserves it. Sharp’s default output policy converts to sRGB and drops source metadata. Rotation applies EXIF orientation before that metadata is removed. Successful decoding is not an integrity check for an original: review source pixels before publishing.

Serve and cache negotiated variants

Save server.mts. Omit f to negotiate a format using Accept, or request f=jpeg or f=webp explicitly. Width w accepts 320, 640, or 1,280; the default is 640. Other or duplicate parameters are rejected.

import type { Response } from 'express'
import express from 'express'
import { createClient } from 'redis'
import { type Format, optimize } from './optimizer.mts'

const published = new Map([['photo-v1.png', 'photo-v1.png']])
const sizes = new Set(['320', '640', '1280'])
const types = { webp: 'image/webp', jpeg: 'image/jpeg' }
const CACHE_SECONDS = 3600
const app = express()
app.disable('x-powered-by')
const redis = createClient({
  url: process.env.REDIS_URL ?? 'redis://127.0.0.1:6379',
  disableOfflineQueue: true,
  socket: { connectTimeout: 2000, reconnectStrategy: false },
})
redis.on('error', () => console.error('Image cache connection failed.'))
let active = 0

async function cacheCommand<T>(command: () => Promise<T>): Promise<T> {
  // Closing the connection also rejects commands already sent to a stalled Redis server.
  const timer = setTimeout(() => {
    if (redis.isOpen) redis.destroy()
  }, 2000)
  try {
    return await command()
  } finally {
    clearTimeout(timer)
  }
}

function fail(res: Response, status: number, message: string): Response {
  return res.status(status).set('Cache-Control', 'no-store').type('text').send(message)
}

function sendImage(res: Response, format: Format, bytes: Buffer): Response {
  return res.type(types[format]).set('Cache-Control', `public, max-age=${CACHE_SECONDS}`)
    .set('X-Content-Type-Options', 'nosniff').send(bytes)
}

app.get('/images/:name', async (req, res) => {
  const filename = published.get(req.params.name)
  if (!filename) return fail(res, 404, 'Image not found.')
  const params = new URL(req.originalUrl, 'http://localhost').searchParams
  if ([...params.keys()].some((key) => key !== 'w' && key !== 'f') ||
      params.getAll('w').length > 1 || params.getAll('f').length > 1) {
    return fail(res, 400, 'Unsupported image parameters.')
  }
  const size = params.get('w') ?? '640'
  const requested = params.get('f')
  const accepted = requested === null ? req.accepts(['image/webp', 'image/jpeg']) : null
  const format = requested ?? (accepted === 'image/webp' ? 'webp' : 'jpeg')
  if (requested === null && !accepted) return fail(res, 406, 'No supported image format.')
  if (!sizes.has(size) || (format !== 'jpeg' && format !== 'webp')) {
    return fail(res, 400, 'Unsupported image variant.')
  }
  if (requested === null) res.vary('Accept')
  const key = JSON.stringify(['image-v1', filename, size, format])
  if (active >= 2) return fail(res, 503, 'Image processor is busy.')
  active += 1
  try {
    const cached = await cacheCommand(() => redis.get(key))
    if (cached !== null) return sendImage(res, format, Buffer.from(cached, 'base64'))
    const bytes = await optimize(filename, Number(size), format)
    await cacheCommand(() => redis.set(key, bytes.toString('base64'), { EX: CACHE_SECONDS }))
    return sendImage(res, format, bytes)
  } catch {
    console.error('Image request failed.')
    return fail(res, 503, 'Image is temporarily unavailable.')
  } finally {
    active -= 1
  }
})

app.use((_req, res) => fail(res, 404, 'Route not found.'))
app.use((_error: unknown, _req: express.Request, res: Response, _next: express.NextFunction) =>
  fail(res, 400, 'Request could not be processed.'))

async function main(): Promise<void> {
  const port = Number(process.env.PORT ?? '3000')
  if (!Number.isInteger(port) || port < 1 || port > 65535) throw new Error('Invalid PORT.')
  await redis.connect()
  const server = app.listen(port, '127.0.0.1')
  server.requestTimeout = 10_000
  server.headersTimeout = 10_000
  server.on('listening', () => console.log(`Image origin: http://127.0.0.1:${port}`))
  server.on('error', () => {
    console.error('Image server could not start.')
    if (redis.isOpen) redis.destroy()
    process.exitCode = 1
  })
  let stopping = false
  const stop = () => {
    if (stopping) return
    stopping = true
    const deadline = setTimeout(() => {
      server.closeAllConnections()
      if (redis.isOpen) redis.destroy()
      process.exit(1)
    }, 15_000)
    deadline.unref()
    server.close(() => {
      if (redis.isOpen) redis.destroy()
      clearTimeout(deadline)
    })
  }
  process.once('SIGTERM', stop)
  process.once('SIGINT', stop)
}

main().catch(() => {
  console.error('Image origin startup failed.')
  if (redis.isOpen) redis.destroy()
  process.exitCode = 1
})

Start the origin in this terminal:

corepack yarn start

Wait for Image origin: http://127.0.0.1:3000. An occupied port or unavailable Redis causes startup to fail. Choose another origin port with PORT=3002 corepack yarn start and adjust the URLs below. Keep Redis private and trusted: cached bytes are not revalidated as images.

Check cold and warm variants

In a second terminal in the project directory, request the same URL four times. These commands overwrite the four named output files; use a fresh directory if you want to keep earlier results.

curl -fsSLo photo.webp -D webp.headers -H 'Accept: image/webp' 'http://127.0.0.1:3000/images/photo-v1.png?w=320' &&
curl -fsSLo photo.jpg -D jpeg.headers -H 'Accept: image/jpeg' 'http://127.0.0.1:3000/images/photo-v1.png?w=320' &&
curl -fsSLo cached.webp -H 'Accept: image/webp' 'http://127.0.0.1:3000/images/photo-v1.png?w=320' &&
curl -fsSLo cached.jpg -H 'Accept: image/jpeg' 'http://127.0.0.1:3000/images/photo-v1.png?w=320'

Both header files should contain Vary: Accept, Cache-Control: public, max-age=3600, and the corresponding Content-Type. Vary tells an HTTP cache that the response also depends on Accept; it does not make the Redis key variant-aware. The key must include the resolved format too.

Save check-results.mts to decode the files, check their actual formats and dimensions, and compare cold and warm bytes:

import assert from 'node:assert/strict'
import { readFile } from 'node:fs/promises'
import sharp from 'sharp'

for (const [cold, warm, format] of [
  ['photo.webp', 'cached.webp', 'webp'],
  ['photo.jpg', 'cached.jpg', 'jpeg'],
]) {
  const bytes = await readFile(cold)
  assert.deepEqual(await readFile(warm), bytes)
  const decoder = sharp(bytes)
  const metadata = await decoder.metadata()
  assert.equal(metadata.format, format)
  assert.equal(metadata.width, 320)
  assert.equal(metadata.height, 200)
  const { data, info } = await decoder.ensureAlpha().raw().toBuffer({ resolveWithObject: true })
  const left = (100 * info.width + 80) * info.channels
  assert.ok(data[left] > 245 && data[left + 1] < 15 && data[left + 2] < 15)
  assert.equal(data[left + 3], 255)
  const right = (100 * info.width + 240) * info.channels
  assert.equal(data[right + 3], format === 'webp' ? 0 : 255)
  if (format === 'jpeg') assert.ok(data[right] > 245 && data[right + 1] > 245 && data[right + 2] > 245)
  console.log(`${cold}: ${format}, 320 × 200, warm bytes match`)
}

node check-results.mts

Open both cold files in a browser too: the red half should be on the left, with a transparent WebP right half and a white JPEG right half. Equal cold and warm bytes alone cannot prove a cache hit. While the origin runs, inspect the two Redis keys:

docker exec image-origin-redis redis-cli --scan --pattern '[[]"image-v1"*'

They should be ["image-v1","photo-v1.png","320","webp"] and ["image-v1","photo-v1.png","320","jpeg"]. Each stored value is the corresponding image in Base64, with a one-hour expiry. A new width creates another variant; requesting w=1280 still produces 640 × 400 because the optimizer does not enlarge the sample.

Publish a changed source

Add a reviewed photo-v2.png file and a new ['photo-v2.png', 'photo-v2.png'] publication-map entry. Use /images/photo-v2.png?w=320 in the consuming page. Keep the old file and route immutable: overwriting photo-v1.png or retargeting its existing URL leaves previously cached browser responses valid until expiry. The source filename in the Redis key separates versions at the origin.

Increment the image-v1 encoder-policy prefix when changing output settings. That replaces Redis keys, but existing HTTP caches still require new URLs or deliberate invalidation. Object storage can supply approved files through a publisher; arbitrary remote-URL fetching is outside this recipe.

Stop and recover the local origin

Press Ctrl+C in the origin terminal to stop accepting connections and let active requests finish. After 15 seconds the process forces connections closed and exits with failure; this is not immediate transform cancellation. Then stop the cache:

docker stop image-origin-redis

If Redis disappears or a command stalls for two seconds, the origin returns an uncached 503 rather than continuing without the cache. The deadline closes the shared Redis connection and rejects its pending commands. Automatic reconnection and the offline queue are disabled; restart this origin after restarting Redis. Rerun only the docker run command from setup, then corepack yarn start; do not rerun the source generator. The two-request cap includes cache access and remains occupied until each handler finishes, so an extra concurrent request receives 503. It is a per-process admission limit, not a total memory or CPU guarantee.

Connect an origin to delivery

Redis reuses a transformed response at this origin. It does not put copies near readers, terminate public TLS, or establish a performance improvement. A CDN in front of a transforming origin must key on width and explicit format, or correctly honor Vary: Accept for negotiated responses. Exclude errors from edge caching. Multiple origin processes also multiply the admission limit.

Our S3 and CloudFront guide covers a separate storage-backed delivery setup; it is not a deployment recipe for this Express server. For managed processing and delivery, see Transloadit’s image processing and Smart CDN.