Secure API file uploads with magic numbers

Tim Koschützki

Co-founder · Berlin, Germany ·

Validating file uploads is crucial for API security. Relying solely on file extensions or MIME types can leave your application vulnerable to spoofing attacks. A more secure method involves checking the file's magic numbers, also known as file signatures.

Why file extension validation isn't enough

File extensions and MIME types are easily spoofed. Attackers can rename malicious files to appear harmless, bypassing basic validation checks. For example, a malicious executable could be renamed document.pdf, fooling systems that only check the extension.

MIME type validation is equally unreliable because browsers and clients can set arbitrary Content-Type headers. This makes magic-number validation essential for robust file verification.

Understand magic numbers and file signatures

Magic numbers are recognizable byte sequences, often near the start of a file, that help identify its likely format. They are independent of the filename and client-provided MIME type, but an attacker can forge them too. A signature match neither proves that the complete file is valid nor that it is safe to process. Treat it as one layer of an upload policy.

Common magic numbers

Below is a short but frequently used list. For a comprehensive overview, see Wikipedia's “List of file signatures.”

Format	Hex (offset 0)	Notes
PNG	`89 50 4E 47 0D 0A 1A 0A`	Always 8 bytes
JPEG	`FF D8 FF DB` `FF D8 FF E0` `FF D8 FF E1`	Covers JFIF and EXIF variants
GIF	`47 49 46 38 37 61` (GIF87a) `47 49 46 38 39 61` (GIF89a)	Six bytes
PDF	`25 50 44 46 2D` (`%PDF-`)	Five bytes
ZIP	`50 4B 03 04` `50 4B 05 06`	DOCX, ODT, and APK are ZIP containers
MP4	`66 74 79 70` (offset 4)	Preceded by four-byte size field

Implement magic-number validation in Node.js

The file-type package used here is version 22 and requires Node.js 22 or newer. It detects signatures from a buffer or a file. The library is ESM-only, so make sure your project’s package.json contains "type": "module" or use .mjs files.

Install the example dependencies:

npm install file-type@22 express@4.22.2 multer@2.3.0

Inspect a file without loading it all into memory

import { fileTypeFromFile } from 'file-type'

/**
 * Validate a file by magic number.
 * @param {string} filePath Absolute or relative path to the file on disk
 * @param {string[]} allowList Array of allowed MIME types
 */
export async function validateFileType(filePath, allowList = []) {
  // Default allow-list: PNG, JPEG, and PDF
  const allowedTypes = allowList.length ? allowList : ['image/png', 'image/jpeg', 'application/pdf']

  const type = await fileTypeFromFile(filePath)
  if (!type) throw new Error('Unknown or unsupported file type')
  if (!allowedTypes.includes(type.mime)) {
    throw new Error(`Disallowed file type: ${type.mime}`)
  }
  return { ...type, valid: true }
}

// Example
// (async () => {
//   await validateFileType('uploads/avatar.png')
// })()

Validate size-limited uploads with Express.js

This endpoint buffers at most 5 MiB per file, then checks its signature. It deliberately does not persist the upload. It is not a streaming-to-disk implementation; limit concurrency and request rates as well when using memory storage.

import express from 'express'
import multer from 'multer'
import { fileTypeFromBuffer } from 'file-type'

const app = express()
const upload = multer({
  storage: multer.memoryStorage(),
  limits: { fileSize: 5 * 1024 * 1024, files: 1, fields: 0 },
})

app.post('/upload', upload.single('file'), async (req, res) => {
  try {
    if (!req.file) return res.status(400).json({ error: 'No file uploaded' })

    const type = await fileTypeFromBuffer(req.file.buffer)
    if (!type || !['image/png', 'image/jpeg'].includes(type.mime)) {
      return res.status(400).json({ error: 'Invalid file type' })
    }

    res.json({ message: 'File validated', mime: type.mime, size: req.file.size })
  } catch (err) {
    res.status(400).json({ error: 'Unable to recognize this file' })
  }
})

app.use((error, req, res, next) => {
  if (res.headersSent) return next(error)
  if (error instanceof multer.MulterError) {
    return res.status(error.code === 'LIMIT_FILE_SIZE' ? 413 : 400).json({ error: 'Upload rejected' })
  }
  console.error('Upload processing failed')
  res.status(500).json({ error: 'Unable to process the upload' })
})

app.listen(3000, () => console.log('API listening on :3000'))

Validate files in Python with `python-magic`

The example requires Python 3.10 or later. python-magic is a thin wrapper around the libmagic C library, so installation differs by platform:

# Debian/Ubuntu
sudo apt-get install python3-magic libmagic1

# macOS (homebrew)
brew install libmagic
pip install python-magic

# Windows (pre-built binaries)
pip install python-magic-bin

from pathlib import Path
import magic

class FileValidator:
    """Validate MIME type using libmagic signatures."""

    def __init__(self, allowed=None):
        self.allowed = set(allowed or {
            'image/png',
            'image/jpeg',
            'application/pdf',
        })
        self._mime = magic.Magic(mime=True)

    def validate(self, file_path: str | Path) -> dict:
        path = Path(file_path)
        if not path.is_file():
            raise FileNotFoundError(path)

        mime_type = self._mime.from_file(str(path))
        if mime_type not in self.allowed:
            raise ValueError(f'Blocked MIME: {mime_type}')

        return {
            'mime': mime_type,
            'size': path.stat().st_size,
            'valid': True,
        }

# Example
# validator = FileValidator()
# print(validator.validate('uploads/report.pdf'))

Handle edge cases and polyglot files

Some files can be interpreted as more than one format. Searching for short byte sequences such as MZ, <html, or ZIP headers inside a file is not a reliable polyglot detector: ordinary binary data can contain them, and valid ZIP-based documents contain repeated ZIP headers. Signature libraries are not malware scanners. Follow identification with a maintained, format-specific parser or decoder in an isolated process, enforce resource limits, and scan or reconstruct content according to your threat model. Do not execute uploads or serve active content from your app origin.

Performance tips for large files

Prefer the library's file or stream APIs. Some formats require more than a fixed-size prefix; there is no universal 4,100-byte detection guarantee.
Process uploads as streams to avoid buffering entire gigabyte-sized files in memory.
Cache allowed-type arrays and regular expressions, especially in serverless environments where cold starts are expensive.

Combine magic-number validation with other controls

Magic numbers suggest a file's format; they do not prove its validity or safety. Strengthen your upload pipeline by layering additional guards:

Enforced upload byte limits, including requests without a trustworthy Content-Length
Virus/malware scanning (ClamAV or a commercial API)
Rate limiting and authentication on upload endpoints
Content-Security-Policy headers when you serve user-supplied media

Test your implementation

A few unit tests (using Vitest, for example) help ensure future refactors don’t break validation. Test with valid files, malformed files, and edge cases like empty files or files with incorrect extensions.

import { describe, expect, it } from 'vitest'
import { fileTypeFromBuffer } from 'file-type'

const png = Buffer.from(
  'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+j3ioAAAAASUVORK5CYII=',
  'base64',
)
const empty = Buffer.alloc(0)
const jpeg = Buffer.from(
  'ffd8ffe000104a46494600010100000100010000ffdb0043000101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101ffc00011080001000103012200021101031101ffc4001f0000010501010101010100000000000000000102030405060708090a0bffda000c03010002110311003f00f2a900',
  'hex',
) // A JPEG-signature fixture; this test does not validate complete image decoding.

describe('magic-number validation', () => {
  it('detects PNG correctly', async () => {
    const t = await fileTypeFromBuffer(png)
    expect(t?.mime).toBe('image/png')
  })

  it('detects JPEG correctly', async () => {
    const t = await fileTypeFromBuffer(jpeg)
    expect(t?.mime).toBe('image/jpeg')
  })

  it('rejects empty buffers', async () => {
    const t = await fileTypeFromBuffer(empty)
    expect(t).toBeUndefined()
  })
})

Troubleshoot common issues

Symptom	Possible cause	Fix
`Error: Unknown or unsupported file type`	`file-type` cannot match any signature	Pass the complete file or buffer, check supported formats, and ensure the file is not encrypted or truncated
`Module not found: file-type`	Using CommonJS `require()`	Switch to ESM or use `import('file-type')` dynamic import
`ImportError: failed to find libmagic`	`libmagic` missing on OS	Install using package manager or use the `-bin` wheel on Windows

Wrap-up

Magic-number checks help reject obvious type mismatches before storage or processing. Combine them with enforced size limits, format-aware validation, malware scanning, and robust error handling; no signature check alone makes an upload safe.

Need an easier way to handle uploads at scale? Check out our handling uploads service at Transloadit.

Secure API file uploads with magic numbers

Why file extension validation isn't enough

Understand magic numbers and file signatures

Common magic numbers

Implement magic-number validation in Node.js

Inspect a file without loading it all into memory

Validate size-limited uploads with Express.js

Validate files in Python with `python-magic`

Handle edge cases and polyglot files

Performance tips for large files

Combine magic-number validation with other controls

Test your implementation

Troubleshoot common issues

Wrap-up

👩‍💻 Join 20k+ developers

File uploading and encoding. Made simple.

Secure API file uploads with magic numbers

Why file extension validation isn't enough

Understand magic numbers and file signatures

Common magic numbers

Implement magic-number validation in Node.js

Inspect a file without loading it all into memory

Validate size-limited uploads with Express.js

Validate files in Python with python-magic

Handle edge cases and polyglot files

Performance tips for large files

Combine magic-number validation with other controls

Test your implementation

Troubleshoot common issues

Wrap-up

👩‍💻 Join 20k+ developers

File uploading and encoding. Made simple.

Validate files in Python with `python-magic`