# Data compression and archive tools

Canonical URL: https://encode.wiki/reference/data-compression-tools/
Content version: ca42e61ede0a24f789d7287a0001244ad22f4abf
Status: current
License: CC BY-SA 4.0
Checksum (SHA-256): 3f44e869d5bdeafe07b90cbc868ee3b6d61ee0494195440e77cc8f1e976445af

This wiki is about media encoding, but you will still compress logs, project folders, caches, subtitles and source tarballs. These pages cover the general-purpose tools. None of them shrink video, audio or image files: those are already compressed, and re-compressing them saves almost nothing.

## Which one should I use?

| You want…                                          | Use                                                                    | Typical command                      |
| -------------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------ |
| A good default for anything on Linux/macOS         | [zstd](/codecs/data/zstd/)                                             | `zstd -T0 file`                      |
| Backups of a directory with Unix metadata          | [tar](/codecs/data/tar/) + zstd or xz                                  | `tar --zstd -cf backup.tar.zst dir/` |
| Smallest file you will compress once               | [xz](/codecs/data/xz/) or [7z](/codecs/data/7z/)                       | `xz -9e file`                        |
| A file anyone can open without installing software | [ZIP](/codecs/data/zip/)                                               | `zip -r archive.zip dir/`            |
| Static web assets                                  | [brotli](/codecs/data/brotli/) (+ [gzip](/codecs/data/gzip/) fallback) | `brotli -q 11 file`                  |
| Maximum compatibility, scripts, old systems        | [gzip](/codecs/data/gzip/)                                             | `gzip -9 -k file`                    |
| Incremental, deduplicated, versioned backups       | [zpaq](/codecs/data/zpaq/)                                             | `zpaq a backup.zpaq dir/`            |
| Opening old `.tar.bz2` files                       | [bzip2](/codecs/data/bzip2/)                                           | `bzip2 -d file.bz2`                  |

## The formats at a glance

| Format | Archive?                   | Strength                                                    | Weakness                                              |
| ------ | -------------------------- | ----------------------------------------------------------- | ----------------------------------------------------- |
| gzip   | No (single stream)         | Universal, fast, tiny memory                                | Weakest ratio                                         |
| bzip2  | No                         | Better than gzip on text                                    | Slow, superseded                                      |
| xz     | No                         | Very small output, cheap to decompress                      | Slow and memory-hungry to compress                    |
| zstd   | No                         | Fast at every level, close to xz at the top, multi-threaded | Less universal than gzip; HTTP window limit           |
| brotli | No                         | Best for web text (built-in dictionary)                     | Slow at high quality; web-oriented                    |
| tar    | Yes (no compression)       | Preserves Unix metadata; pairs with any compressor          | No index, no random access                            |
| ZIP    | Yes (per-file compression) | Opens everywhere, random access                             | Weaker ratio; mixed support for newer methods and AES |
| 7z     | Yes (solid)                | Strong ratio, AES-256, big dictionaries                     | Slow to extract single files; drops Unix owner/ACLs   |
| ZPAQ   | Yes (journaling)           | Deduplication, every version restorable                     | Slow, rare                                            |

A "single stream" format compresses one byte sequence. For directories, put them in tar or zip first, which is why you see `.tar.gz`, `.tar.xz` and `.tar.zst`.

## Speed against size

Levels trade time for size, with sharply diminishing returns. zstd `-3` and gzip `-6` are the fast defaults; zstd `-19` and xz `-6` are slow but noticeably smaller; xz `-9e`, 7z `-mx=9`, zstd `--ultra -22` and brotli `-q 11` are for files compressed once and read many times. If you compress frequently (logs, intermediate files, network transfers), stay at zstd `-1`…`-9`. zstd needs `-T0` to use all cores; xz has done so by default since 5.6.0.

## HTTP content coding

Servers and browsers negotiate a coding with `Accept-Encoding` and `Content-Encoding`. gzip, `br` (Brotli) and `zstd` are the three registered codings that matter. gzip works everywhere; `br` is accepted by browsers over HTTPS; zstd is advertised by Chrome and Edge 123+, Firefox 126+ and Safari 26.3+. Always keep a fallback and serve with `Vary: Accept-Encoding`. zstd for HTTP is limited to an 8 MiB window ([RFC 9659](https://www.rfc-editor.org/rfc/rfc9659)), so files made with `--long` or `--ultra` levels cannot be served that way. xz, 7z and bzip2 are not HTTP codings.

## Checking an archive

`zstd -t`, `xz -t`, `gzip -t`, `bzip2 -t`, `brotli -t`, `unzip -t`, `7z t` and `zpaq x -test` verify integrity without extracting. The built-in checks catch accidental corruption, but not tampering: publish a SHA-256 alongside downloads.