compress.library Version 1.0 (C) 1991 Bryan Ford First let's get the legaleze over with. The compress.library (or at least *this* (my) compress.library) is not public domain. However, it can be freely copied for noncommercial purposes. You may distribute it on its own, or with another program (i.e. one that uses the compress.library) as long as that program is also freely distributable. In any distribution, alone or with another program, this document must accompany the compress.library itself. The programmer's files (in the "programmers" directory) do not need to be included when the compress.library is included with another program, but they must be included with any stand-alone distribution of the compress.library. If you would like to distribute the compress.library with a commercial program, you must make arrangements with me first. (Don't worry, I'm not planning on bleeding anybody dry.) Now on to the fun part. The compress.library is a very small run-time library intended for general use by other programs. Its purpose is to take a chunk of data given to it by a program, compress it "losslessly", and pass the compressed data back to the application. Later, the application can call on the compress.library to expand the data into its original shape and size. The data can be anything the program wants, and does not even need to reside on disk or be anything related to a "file". Therefore, it could be used for "stashing" a program's data in memory while the user is in some other part of the program. Or it could be used simply for real-time compression and decompression of a program's data. (Can you say "Aquarium"?) Or maybe we could persuade Matt Dillon to compress data blocks as DNET sends them over the modem... I'm sure programmers could find many uses for a general-purpose compressor such as this. You are probably by now quite skeptical about the practical value of something like this, with the delays incurred by compression and/or decompression. Don't worry, the compress.library is very fast, in both compression and decompression. On a stock 68000 Amiga, it compresses data at over 16K/sec and decompresses at over 32K/sec. (Both of these figures are probably significantly underestimated, since I've done quite a bit of optimization since I timed them on a 68000.) On my 25MHz A3000 I recently benchmarked the code at 120K/sec compression 270K/sec decompression! Faster decompression is possible (I believe the Imploder decompresses more quickly for example), but usually compression speed must be sacrificed. My intention is to provide both compression and decompression at near real-time speeds. With small chunks of data, compression and decompression are often not noticeable at all. Now then, how well does it compress? See for yourself: Load up Zoo 2.00 (I haven't tried Zoo 2.10 yet) and try compressing a few things with it. The compress.library uses almost the same algorithm as Zoo (a variation of LZW compression), and gets almost identical compression ratios. (Often slightly better, depending on the number of bits selected.) And, of course, the compress.library leaves Zoo in the dust in terms of speed. Programs like PkZip and Lharc (LZ) get better compression ratios, but at a great cost in speed. The compress.library does require a small amount of extra memory during compression and decompression. (It will return an error to the calling program if the memory is not available.) The amount of memory used depends on the number of bits used in the LZW algorithm. The compress.library can be set to use anywhere between 9 and 15 bits. The default is 12, which requires about 25K during compression and about 12K during decompression. For every extra compression bit you add on, you roughly double the memory requirement for both compression and decompression. The application has the option of setting the number of compression bits, or it can just use the default. So, how do you use the compress.library in your programs? Look in the file "compress_lib.prodoc" in the "programmers" directory. I purposely kept the interface very simple and direct, so you should have no problem using the library. Note that I named this library "compress.library" rather than "bryanscompress.library" or somesuch for a good reason. This is intened not just as a stand-alone piece, but as the first implementation of a standard library interface. I want to encourage other people to come up with alternate compression libraries, maybe with different compression ratio/speed tradeoffs. I tried to strike a good balance between speed and compression, but users with different systems might have different priorities. Decompression could be sped up by sacrificing compression speed. (Good if you do mostly reading of once-compressed data.) Better compression ratios could be gotten with a slower algorithm. (Maybe Jonathan Forbes would like to make a compress.library that uses LZ's code...?) Multiple compression libraries (and therefore multiple data formats) floating around could create a slight problem, though, if we start seeing programs distributed with data already compressed using a particular compress.library - the user might want to use a different library than the one the data was compressed with. Although a library will (or at least should) always detect a "foreign" data format that it can't decompress, and therefore shouldn't crash the machine or scramble data, it still creates lots of trouble. Therefore, I think we should keep use of the compress.library to the individual: distribute a program (and its data) in uncompressed form, then let the user tell the program how to compress its data, which library to use, etc. Since freely distributable programs are usually distributed in compressed form anyway, there would not be significant gains to distributing already-compressed data anyway. So, there you go - let's see what exotic ways to use compression you can come up with. I can be contacted for comments, suggestions, or flames at any of the addresses below. Bryan Ford 27104 Ballif Hall Salt Lake City, UT 84112 (801) 585-4619 baf0863@cc.utah.edu baf0863@utahcca.bitnet Have fun!