Data compression and archive tools
This wiki is about media encoding, but you will still compress logs, project folders, caches, subtitles and source tarballs. These pages cover the general-purpose tools. None of them shrink video, audio or image files: those are already compressed, and re-compressing them saves almost nothing.
Which one should I use?
Section titled “Which one should I use?”| You want… | Use | Typical command |
|---|---|---|
| A good default for anything on Linux/macOS | zstd | zstd -T0 file |
| Backups of a directory with Unix metadata | tar + zstd or xz | tar --zstd -cf backup.tar.zst dir/ |
| Smallest file you will compress once | xz or 7z | xz -9e file |
| A file anyone can open without installing software | ZIP | zip -r archive.zip dir/ |
| Static web assets | brotli (+ gzip fallback) | brotli -q 11 file |
| Maximum compatibility, scripts, old systems | gzip | gzip -9 -k file |
| Incremental, deduplicated, versioned backups | zpaq | zpaq a backup.zpaq dir/ |
Opening old .tar.bz2 files | bzip2 | bzip2 -d file.bz2 |
The formats at a glance
Section titled “The formats at a glance”| Format | Archive? | Strength | Weakness |
|---|---|---|---|
| gzip | No (single stream) | Universal, fast, tiny memory | Weakest ratio |
| bzip2 | No | Better than gzip on text | Slow, superseded |
| xz | No | Very small output, cheap to decompress | Slow and memory-hungry to compress |
| zstd | No | Fast at every level, close to xz at the top, multi-threaded | Less universal than gzip; HTTP window limit |
| brotli | No | Best for web text (built-in dictionary) | Slow at high quality; web-oriented |
| tar | Yes (no compression) | Preserves Unix metadata; pairs with any compressor | No index, no random access |
| ZIP | Yes (per-file compression) | Opens everywhere, random access | Weaker ratio; mixed support for newer methods and AES |
| 7z | Yes (solid) | Strong ratio, AES-256, big dictionaries | Slow to extract single files; drops Unix owner/ACLs |
| ZPAQ | Yes (journaling) | Deduplication, every version restorable | Slow, rare |
A “single stream” format compresses one byte sequence. For directories, put them in tar or zip first, which is why you see .tar.gz, .tar.xz and .tar.zst.
Speed against size
Section titled “Speed against size”Levels trade time for size, with sharply diminishing returns. zstd -3 and gzip -6 are the fast defaults; zstd -19 and xz -6 are slow but noticeably smaller; xz -9e, 7z -mx=9, zstd --ultra -22 and brotli -q 11 are for files compressed once and read many times. If you compress frequently (logs, intermediate files, network transfers), stay at zstd -1…-9. zstd needs -T0 to use all cores; xz has done so by default since 5.6.0.
HTTP content coding
Section titled “HTTP content coding”Servers and browsers negotiate a coding with Accept-Encoding and Content-Encoding. gzip, br (Brotli) and zstd are the three registered codings that matter. gzip works everywhere; br is accepted by browsers over HTTPS; zstd is advertised by Chrome and Edge 123+, Firefox 126+ and Safari 26.3+. Always keep a fallback and serve with Vary: Accept-Encoding. zstd for HTTP is limited to an 8 MiB window (RFC 9659), so files made with --long or --ultra levels cannot be served that way. xz, 7z and bzip2 are not HTTP codings.
Checking an archive
Section titled “Checking an archive”zstd -t, xz -t, gzip -t, bzip2 -t, brotli -t, unzip -t, 7z t and zpaq x -test verify integrity without extracting. The built-in checks catch accidental corruption, but not tampering: publish a SHA-256 alongside downloads.