What are Magic Bytes?
Magic bytes are a short signature, usually the first few bytes of a file, that identifies its real format. A PNG always starts with the bytes 89 50 4E 47 and a PDF with %PDF, whatever the file is named.
How Magic Bytes work
Most file formats begin with a fixed signature so that software can recognize them without trusting the name. Tools such as the Unix file command compare the start of a file against a table of known signatures. A few formats put the signature further in: MP4 files have ftyp at byte 4, and tar archives have ustar at byte 257.
Key facts
- Common signatures in hexadecimal: JPEG
FF D8 FF, PNG89 50 4E 47, GIF47 49 46 38, PDF25 50 44 46, ZIP50 4B 03 04. - DOCX, XLSX, JAR and APK files are all ZIP archives, so they share the ZIP signature. Telling them apart means looking inside the archive.
- Plain text formats such as CSV, JSON and most source code have no magic bytes, so their type has to be guessed from the content.
When Magic Bytes matter
Check magic bytes when accepting uploads, because a file extension can be changed by anyone. They tell you what type a file is, not that it is safe: a real PDF can still carry something harmful.
How Transloadit helps with Magic Bytes
Transloadit inspects the content of every uploaded file on the server and reports the detected type as mime, so it does not rely on the type the browser claimed. The /file/filter Robot can use that value to accept or reject files, for example with the condition ["${file.mime}", "regex", "image"].