This is the source code of XAD's bzip2 client, for the curious. extheader.c The 'file header' used to identify XAD slaves. bzip2.c The XAD client for bzip2. SDI_Compiler.h Macros to allow independence of compiler-specific features. bzlib.c The public interface to the bzip2 decompressor. bzlib.h The public interface definition. bzlib_private.h The private bits and bobs of libbzip2. crctable.c A table with CRC values in it, for checksumming. decompress.c The bzip2 block decompressor. huffman.c A huffman table generator. randtable.c A table with random numbers in it, for entropy matters. bzip2, (C) 1996-2000 Julian R. Seward, is a "block-sorting file compressor". It uses the Burrows-Wheeler block-sorting text compression algorithm, with some Huffman coding on the output, and, run length encoding and randomisation thrown in to improve the speed efficiency of the compressor. The run-length encoding "is entirely irrelevant" and the randomisation "doesn't really need to be there". However, "compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, and approaches the performance of the PPM family of statistical compressors." bzip2 is mainly used as a replacement for gzip in compressing UNIX tar archives, because it beats gzip (which uses the LZH-like deflate algorithim) in compression and is therefore useful for saving valuable bandwidth for Linux distributions and the like. The main 'problems' with bzip2 are: - it uses huge amounts of memory. The default compression mode is the maximum block size, which means that decompression requires 2.1Mb of RAM (preferably 3.5Mb), even for the tiniest files. - it can only compress and decompress single files - it doesn't store any file metadata whatsoever, not even a single file's name or size. - it has no idea how large the output data is in advance, and it can't even tell how big each block is in the file. It can only run through the data and tell you once it's decompressed how big it was. Even with these problems, the advantage in compression over gzip is enough to make it popular with new archive sites. The original bzip2 and libbzip2 are available from http://sourceware.cygnus.com/bzip2/ CHANGES TO LIBBZIP2 The XAD client only uses the official interface to the library, so it should work fine with an unmodified libbzip2, compiled with BZ_NO_STDIO. However, for general improvement, I've made a few changes. Firstly, all unneccessary code has been removed: - blocksort.c and compress.c have been deleted. - bzlib.c only has the bzDecompress* functions left. - huffman.c only has hbCreateDecodeTables() left. The decompresser has been modified to try and enter 'small mode' automatically if it cannot allocate enough memory for 'fast mode'. The library normally expects you to say which mode you want and sticks rigidly to that. Assertion failures in decompression (that is, failures of the code that can't logically occur) are turned into returning error codes, as the XAD slave can't support the immediate exit() that libbzip2 would like - instead we have to pop back to the original entry point and return, it's the only way. The random number table 'rNums' has been made into Int16 instead of Int32. This halves the size of the table (and saves exactly 1Kb), yet because all the numbers are small enough to fit into an Int16, it doesn't require changing code at all. All I've added is an explicit cast when a table entry is retrieved, to be on the safe side. This optimisation was suggested by Dirk Stöcker. CREDITS - Stuart Caie wrote the XAD client for bzip2. - Julian R. Seward wrote libbzip2. - Dirk Stöcker wrote extheader.c, SDI_Compiler.h and the entire XAD library system. He also made the bzip2 client possible. - Mike Burrows, David Wheeler, Peter Fenwick, Alistair Moffat, Radford Neal, Ian H. Witten, Robert Sedgewick and Jon L. Bentley devised the algorithims used in libbzip2.