Transloadit

What is Entropy Coding?

Entropy coding is lossless compression that uses fewer bits on average for more probable symbols and more bits for less probable ones. Huffman and arithmetic coding are common methods.

How Entropy Coding works

A codec first converts source information into symbols whose statistical distribution can be modeled, then an entropy stage represents the likely sequences compactly. The decoder must know or reconstruct the same code tables, contexts, and state to reverse that representation. This stage differs from quantization: it can reduce the byte count but does not intentionally discard symbol values. It commonly appears near the end of an image, audio, or video encoder and near the start of its decoder.

Key facts

  1. A Huffman code gives each coded symbol its own variable-length codeword, so it cannot spend less than one bit on a symbol however predictable that symbol is. Arithmetic coding has no such floor. The VP8 specification, whose boolean coder is a variant of an arithmetic coder, works an example in which one outcome has probability 1/1024 and the lowest possible rate is roughly one hundredth of a bit per value. The gain is largest on heavily skewed data and disappears when both outcomes are equally likely, where the same text gives a lower bound of one bit.
  2. Entropy coding can be redone without touching quality. libjpeg-turbo’s jpegtran rearranges a JPEG’s compressed DCT coefficients without fully decoding the image, and its documentation describes switching to optimized entropy parameters, progressive scans, or arithmetic coding as lossless. For cjpeg, the same documentation says -optimize usually makes a file a little smaller and leaves image quality and decompression speed unaffected, while the encoder runs somewhat slower and needs much more memory; jpegtran accepts the same switch. jpegtran cannot change the quality setting, and it can remove metadata: its default, -copy comments, keeps comment markers and discards other metadata, so pass -copy all to keep Exif data, an ICC profile, and other extra markers.
  3. A more efficient entropy coder helps only if every target decoder implements it. In the JPEG standard the baseline sequential process uses Huffman coding, and arithmetic coding is available only in the extended and lossless processes. libjpeg-turbo’s documentation cautions that arithmetic-coded JPEG is not widely implemented and that many decoders cannot read such a file at all. Check the decoders you must support before enabling a non-default entropy mode.
  4. The decoder must reproduce the encoder’s tables and probability state exactly, so damage spreads beyond the corrupted bytes. libjpeg-turbo’s documentation says that without restart markers, damage to a JPEG usually ruins the image from the error to the end; with them it is usually confined to the portion up to the next marker, and the markers cost extra space. JPEG resets its DC prediction and its arithmetic probability estimates at each restart interval, which is what lets decoding resume there. jpegtran emits no restart markers by default, even when the source file has them, and it does not carry over the source’s interval: -restart N inserts new markers every N MCU rows, so choose N explicitly to keep that protection. Its documentation lists -restart among the switches that set the coded representation and calls its transformations lossless, so an undamaged file decodes to the same image with or without markers.

When Entropy Coding matters

Codec developers apply entropy coding to quantized coefficients, prediction residuals, and other structured values, and its efficiency depends on how closely the probability model matches the data. Everyone else meets it as an encoder setting: JPEG tools offer optimized Huffman tables or arithmetic coding, and H.264 encoders offer CAVLC or CABAC. The stage itself discards nothing, so at a given quantization these settings trade file size against encoding cost and decoder support.

Sources

Turn media knowledge into a working pipeline

Connect uploads, processing, AI, storage, and delivery through one declarative API — with the encoding stack, scaling, and format churn handled for you.

Try Transloadit for free