You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
Go to file
Dave Vasilevsky 4ca208ccd8 Find blocksize dynamically based on dict_size 15 years ago
.gitignore ignore list 15 years ago
Makefile factor out common code 15 years ago
README docs 15 years ago
TODO docs 15 years ago
common.c multi-threaded encoding 15 years ago
list.c factor out common code 15 years ago
pixz.h Find blocksize dynamically based on dict_size 15 years ago
read.c factor out common code 15 years ago
tar.c create a compressed tar file with a file index 15 years ago
test.sh multi-threaded encoding 15 years ago
write.c Find blocksize dynamically based on dict_size 15 years ago

README

Pixz (pronounced 'pixie') is a parallel, indexing version of XZ.


The existing XZ Utils ( http://tukaani.org/xz/ ) provide great compression in the .xz file format, but they have two significant problems:

* They are single-threaded, while most users nowadays have multi-core computers.
* The .xz files they produce are just one big block of compressed data, rather than a collection of smaller blocks. This makes random access to the original data impossible.


With pixz, both these problems can eventually be solved. Currently these pixz tools are available:

* write INPUT.tar OUTPUT.tpxz: Compresses an uncompressed tarball. The compression uses two cores. An index of all the files in the tarball is stored within the file, yet it remains compatible with standard xz and tar.

* read INPUT.tpxz PATH: Efficiently extracts a single file from a tarball compressed by 'write'.

* list [-t] INPUT.xz: Lists the xz blocks present within any .xz file. Optionally also lists a file index as stored by 'write'.