LZ4 for Linux: Install, Use & Speed Up File Compression (Examples)

LZ4 is an extremely fast lossless compression algorithm that aims to provide very high speeds of compression and decompression (compression speeds of over 500 MB/s per CPU core) with reasonable compression ratios.

This makes it an excellent choice for applications that require high-speed compression, such as data transmission, real-time processing, network transmission, kernel decompression, and storage optimization.

Unlike some other compression algorithms, LZ4 also features a very fast decompression speed, often reaching the limits of RAM speed on multi-core systems.

This article will guide you through installing and using LZ4 on a Linux system with practical examples.

Installing LZ4 on Linux

The easiest way to install LZ4 is to use pre-built packages available in your Linux distribution’s repositories.

sudo apt install lz4         [On Debian, Ubuntu and Mint]
sudo yum install lz4         [On RHEL/CentOS/Fedora and Rocky/AlmaLinux]
sudo emerge -a sys-apps/lz4  [On Gentoo Linux]
sudo apk add lz4             [On Alpine Linux]
sudo pacman -S lz4           [On Arch Linux]
sudo zypper install lz4      [On OpenSUSE]    
sudo pkg install lz4         [On FreeBSD]

If you prefer to build LZ4 from the source, your distribution does not provide a precompiled package, or you need the latest version of LZ4, you can clone the repository and build the library and tools using the following commands.

git clone https://github.com/lz4/lz4.git
cd lz4
make
sudo make install

Using LZ4 to Compress and Decompress Files

LZ4 provides a command-line tool that can be used to compress and decompress files, as shown in the following practical examples.

To compress a file using LZ4, use the following command.

lz4 tecmint.txt tecmint_file.lz4

To decompress a file compressed with LZ4, use:

lz4 -d tecmint_file.lz4 decompressed_tecmint.txt

Using LZ4 to Compress and Decompress Directories

LZ4 does not directly support compressing directories, but you can use the tar command in conjunction with LZ4 to achieve this.

To compress a directory using LZ4, use the following command.

tar cf - tecmint | lz4 - compressed_tecmint.tar.lz4

To decompress a directory using LZ4, use the following command.

lz4 -d compressed_tecmint.tar.lz4 | tar -xvf -

Using LZ4 with Pipes

LZ4 can be used in a pipeline to compress or decompress data as it is being processed by other commands.

If you have a large log file and want to compress it while streaming.

cat large_log_file.log | lz4 - compressed_log.lz4

You can decompress a file and process its content on the fly.

lz4 -d compressed_log.lz4 | grep "ERROR"

Setting LZ4 Compression Levels

LZ4 offers several compression levels, ranging from 1 (fast) to 9 (high compression). You can specify the compression level using the -1 to -9 flags.

For example, to use the highest compression level:

lz4 -9 input_file.txt compressed_file.lz4

This command uses compression level 9 for a better compression ratio at the expense of speed.

To get more information about the compression process, use the -v flag.

lz4 -v input_file.txt compressed_file.lz4

For maximum compression, use the -HC option.

lz4 -HC input_file.txt compressed_file.lz4

This mode is significantly slower but provides better compression ratios.

Using LZ4 Library in Applications

If you are a developer, you can use the LZ4 library to integrate compression capabilities into your applications.

Here is a basic example using the LZ4 C API:

#include <lz4.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main() {
    const char* input = "This is a test string to compress using LZ4.";
    int inputSize = strlen(input) + 1;
    int maxCompressedSize = LZ4_compressBound(inputSize);
    char* compressed = malloc(maxCompressedSize);

    int compressedSize = LZ4_compress_default(input, compressed, inputSize, maxCompressedSize);

    if (compressedSize > 0) {
        printf("Compression successful: %d bytes compressed to %d bytes\n", inputSize, compressedSize);
    } else {
        printf("Compression failed\n");
    }

    free(compressed);
    return 0;
}

This simple program compresses a string using LZ4 and prints the compressed size.

Conclusion

LZ4 is an extremely fast compression algorithm suitable for scenarios where speed is crucial. Installing LZ4 on Linux is straightforward by using package managers or by building from source.

This article covered basic and advanced usage of LZ4 with practical examples. Additionally, we provided a brief example of integrating LZ4 into a C application.

With this knowledge, you can leverage LZ4 for efficient data compression in various applications and workflows on your Linux system.