TABLE OF CONTENTS xadIO/--general-- xadIO/--Why use it-- xadIO/--The sourcecode-- xadIO/xadIOAlloc *xadIO/xadIODropBitsHigh *xadIO/xadIODropBitsLow *xadIO/xadIOGetBitsHigh *xadIO/xadIOGetBitsLow *xadIO/xadIOGetChar *xadIO/xadIOPutChar *xadIO/xadIOReadBitsHigh *xadIO/xadIOReadBitsLow xadIO/xadIOWriteBuf xadIO/xio_GetFunc xadIO/xio_InFunc xadIO/xio_OutFunc xadIO/xio_PutFunc VERSION $VER: xadIO.doc 1.0 (24.03.2001) by SDI xadIO/--general-- xadIO/--general-- How to use this xadIO-system? It is very easy. The callers side does not known anything about the algorithm except its name. An example call may look like this: struct xadInOut *io; if((io = xadIOAlloc(XADIOF_ALLOCINBUFFER|XADIOF_ALLOCOUTBUFFER |XADIOF_NOCRC16, ai, xadMasterBase))) { io->xio_InSize = fi->xfi_CrunchSize; io->xio_OutSize = fi->xfi_Size; io->xio_CRC32 = ~0; switch(method) { case METHOD1: err = decrunchM1(io); break; case METHOD2: err = decrunchM2(io); break; case STORED: while(!(io->xio_Flags & (XADIOF_LASTOUTBYTE|XADIOF_ERROR))) xadIOPutChar(io, xadIOGetChar(io)); err = io->xio_Error; break; default: err = XADERR_DATAFORMAT; } if(!err) err = xadIOWriteBuf(io); if(!err && ~io->xio_CRC32 != crc) err = XADERR_CHECKSUM; xadFreeObjectA(io, 0); } else err = XADERR_NOMEMORY; This easy structure can be expanded a lot. So it maybe necessary to replace the xio_PutFunc() to do inline RunLength decoding or add input and output functions for decryption and different checksum calculations. All this is possible and very easy. The STORED method shows an example how to copy the contents of the file only. Normally the XADAC_COPY method should be used for that. In case of password decryption or nonstandard checksum calculation, the above is an easy way to do this in the same unique interface as the decompression. The decompressors side does not know about this specials and only does it work. Its side looks like that: static LONG decrunchM1(struct xadInOut *io) { struct priv *dat; struct xadMasterBase *xadMasterBase = io->xio_xadMasterBase; if((dat = (struct priv *) xadAllocVec(sizeof(struct priv), MEMF_CLEAR|MEMF_PUBLIC))) { /* init stuff */ while(!(io->xio_Flags & (XADIOF_LASTOUTBYTE|XADIOF_ERROR))) { /* do something, use xadIOGetChar() and xadIOPutChar() or better use the bit functions if possible. */ /* Error codes are set in io->xio_Error TOGETHER with setting the flag XADIOF_ERROR. */ } xadFreeObjectA(dat, 0); } return io->xio_Error; } It maybe ncessary to add additionally arguments to the function call (e.g. the number of bits or global bufferes for multifile compression like in LZX). Generally it is best, if nothing of this is needed and the algorithm allocates its bufferes itself. This makes reusing it much easier. xadIO/--Why use it-- xadIO/--Why use it-- This system is very open and allows support of different decompression routines. Some things, which are possible: - The Input/Output functions maybe changed to calculate checksums, encrypt, decrypt the data or even do another decompression step (e.g. run length decoding) directly. - The interface can be used to decrunch from file to file, from file to memory, from memory to memory or from memory to file without any additional work in the decrunching function. - The system is not direct part of the xadmaster.library to make it much faster (no system call interface inbetween. The functions are really fast, but speed maybe improved by inlining the bits functions directly. The source code provides mechanisms to do so. Thoughts about the implementation: - The xadIOPutChar and xadIOGetChar interfaces aren't optimal when compared to buffer based algorithms and anything like that. But when doing tests the speed decrease is not really a problem. This is because the put and get operations aren't most important in modern compression algorithms anymore. - The main advantage is an easy to use and powerful interface, which is as fast as most other interfaces. And it is a standardized interface. - The reliability of the program code is increased when doing changes, as often the code itself needs no modifications, but only new ways to call (e.g. with CRC call, ...) are needed. - It is very easy to reuse the algorithms (and most time algorithms are used in different archiver systems). xadIO/--The sourcecode-- xadIO/--The sourcecode-- The source code consists of 2 files called xadIO.h and xadIO.c. In case you want to use it as link library, call the file xadIO.h from your sources always. This is like for any other module in your projects. The link library is produces, when compiling the file xadIO.c with the define XADIOFUNCMODE defined to "". The other way is calling xadIO.c from your source code. In this case the functions default to be static and this maybe inlined in your code. Defining the XADIOFUNCMODE to anything else than static may change the type of the functions xadIOAlloc() and xadIOWriteBuf(). Changing XADIOFUNCMODEBITS allows you to change the bit functions (e.g. make the inline functions). This define default to the value of XADIOFUNCMODE normally. You need to enable the required bit-functions in this mode. Define following before calling the source file: XADIOGETBITSHIGH XADIOGETBITSLOW XADIOREADBITSHIGH XADIOREADBITSLOW xadIO/xadIOAlloc xadIO/xadIOAlloc NAME xadIOAlloc - Allocate the xadInOut structure and necessary buffers SYNOPSIS struct xadInOut * xadIOAlloc(ULONG flags, struct xadArchiveInfo *ai, struct xadMasterBase *xadMasterBase) FUNCTION This function allocates the xadInOut structure and initializes it. Also required buffers are allocated. The allocated structure is afterwards freed with xadFreeObjectA(). The flags allow to change the behaviour of this function and other functions of this system. XADIOF_ALLOCINBUFFER This flags cause the function to allocate an input buffer. If not set (e.g. when working from memory) the fields xio_InBufferSize xio_InBuffer must be set after this function call. XADIOF_ALLOCOUTBUFFER This function causes the system to allocate an output buffer. If not set (e.g. when working from memory) the fields xio_OutBufferSize xio_OutBuffer must be set after this function call. XADIOF_NOINENDERR The function xadIOGetChar() does not produce an error at buffer end, but inserts 0 characters as long as needed. XADIOF_NOOUTENDERR The function xadIOPutChar() does not check the size of output stream, but outputs data anyway. This is required when the output file size is not know before. Note that this has problems, when extracting to memory, as the function xadIOWriteBuf() gets called if buffer is full and needs to be flushed. XADIOF_NOCRC16 Do not calculate standard CRC16. Saves some time. The CRC is calculated in xadIOWriteBuf() function. XADIOF_NOCRC32 Do not calculate standard CRC32. Saves some time. The CRC is calculated in xadIOWriteBuf() function. XADIOF_COMPLETEOUTFUNC The supplied output-function does complete output itself. This maybe necessary sometimes. Normally it only works with the output buffer and lets xadWriteBuf() do the output. Although the cannot be passed as flags directly, here the description of the missing flags. These are set by other functions to supply status information: XADIOF_LASTINBYTE Is set in case the last input byte was read. XADIOF_LASTOUTBYTE Is set in case the last output byte was writen. XADIOF_ERROR Is set in case an error occured. This flag reduces the need to do an additional check for error conditions. INPUT flags - XADIOF_xxx flags ai - Communication structure for buffer reads and stuff like that. Maybe zero in case the xioIOGetChar() and xadIOWriteBuf() functions need not access input or output buffers. xadMasterBase - The base library pointer. RESULT ptr - Pointer to xadInOut structure or zero for error. SEE ALSO xadIO.h, xadIO.c, xadIOGetChar(), xadIOPutChar(), xadIOWriteBuf(), xadFreeObjectA() xadIO/xadIOWriteBuf xadIO/xadIOWriteBuf NAME xadIOWriteBuf - Flushed output buffer SYNOPSIS LONG xadIOWriteBuf(struct xadInOut *io) FUNCTION This function is called from xadIOPutChar() in case the buffer is full AND a new byte needs to be written! This is essential! It means this function is never called if size of output buffer and output file size are equal. It needs to be called directly after the sucessful extraction to flush partial buffers. This behaviour allows to extract to memory directly without ever calling this function. It also calculates CRC16 and CRC32 values and allows to call special output functions processing the buffer. These functions may takeover the complete output process (althought not recommended). INPUT io - the xadInOut structure RESULT err - an XAD error code SEE ALSO xadIO.h, xadIO.c, xadIOPutChar(), xadIOWriteBuf() xadIO/xio_GetFunc xadIO/xio_GetFunc NAME xio_GetFunc - Gets one character SYNOPSIS UBYTE xio_GetFunc(struct xadInOut *io) XADINOUTGET xio_GetFunc FUNCTION This function reads one character from source. It is called through the io-structure using the macro xadIOGetChar(). The standard function is assigned by xadIOAlloc() and maybe replaced by own functions. Replacements need to take care of following: - The fields io->xio_Error and io->xio_Flags |= XADIOF_ERROR must be set in case of errors. - If XADIOF_NOINENDERR is not set the error code XADERR_DECRUNCH is returned if no more bytes are available, else zero bytes are returned. - The function io->xio_InFunc must be called after complete buffer reads. - The field io->xio_InBufferPos needs to be used and set correctly. - The field io->xio_InSize is decreased for every read byte. - The Flag XADIOF_LASTINBYTE is set for last read byte. - The field io->xio_GetFuncPrivate is for private use. INPUT io - the xadInOut structure RESULT res - one byte read from source SEE ALSO xadIO.h, xadIO.c, xadIOAlloc(), xadIOGetChar() xadIO/xio_InFunc xadIO/xio_InFunc NAME xio_InFunc - Input processing buffer SYNOPSIS void xio_InFunc(struct xadInOut *io, ULONG size) XADINOUTFUNC xio_InFunc FUNCTION This function allows the preprocess the input buffer. This maybe used to call decryption routines or checksum algorithms. It gets the xadInOut structure and the buffer size as argument. It is allow to modify the contents of xio_InBuffer from position zero until passed size. Nothing else except the private pointer maybe changed. It does not support buffer expansion or size changes. This function is called from xadIOGetChar() or xio_GetFunc() after reading a new buffer. Replacements need to take care of following: - The fields io->xio_Error and io->xio_Flags |= XADIOF_ERROR must be set in case of errors. - The field io->xio_InFuncPrivate is for private use. INPUT io - the xadInOut structure size - the input buffer size RESULT none SEE ALSO xadIO.h, xadIO.c, xadIOGetChar(), xio_GetFunc() xadIO/xio_OutFunc xadIO/xio_OutFunc NAME xio_OutFunc - Output processing buffer SYNOPSIS void xio_OutFunc(struct xadInOut *io, ULONG size) XADINOUTFUNC xio_OutFunc FUNCTION This function allows the preprocess the output buffer. This maybe used to call decryption routines or checksum algorithms. It gets the xadInOut structure and the buffer size as argument. It is allow to modify the contents of xio_OutBuffer from position zero until passed size. Nothing else except the private pointer maybe changed. This function is called from xadIOWriteBuf() before writing the buffer. If the flag XADIOF_COMPLETEOUTFUNC is set, this function also needs to store the output buffer using xadHookAccess(). Replacements need to take care of following: - The fields io->xio_Error and io->xio_Flags |= XADIOF_ERROR must be set in case of errors. - The field io->xio_OutFuncPrivate is for private use. INPUT io - the xadInOut structure size - the output buffer size RESULT none SEE ALSO xadIO.h, xadIO.c, xadIOAlloc(), xadIOWriteBuf(), xadHookAccess() xadIO/xio_PutFunc xadIO/xio_PutFunc NAME xio_PutFunc - Stores (one) character in output stream SYNOPSIS UBYTE xio_PutFunc(struct xadInOut *io, UBYTE c) XADINOUTPUT xio_PutFunc FUNCTION This function stores (one) character into destination. It is called through the io-structure using the macro xadIOPutChar(). The standard function is assigned by xadIOAlloc() and maybe replaced by own functions. This function maybe used to do inline decompression (e.g. RLE) and thus not necessary stores the passed byte or may store more than one byte. Replacements need to take care of following: - The fields io->xio_Error and io->xio_Flags |= XADIOF_ERROR must be set in case of errors. - If XADIOF_NOOUTENDERR is not set the error code XADERR_DECRUNCH is returned if no more bytes can be stored, else the additional bytes are stored. - The field io->xio_OutBufferPos needs to be used and set correctly. - The field io->xio_OutSize is decreased for every read byte. - The Flag XADIOF_LASTOUTBYTE is set for last read byte. - The passed byte needs to be returned as result! - If the buffer is full and a new byte needs to be stored xadIOWriteBuf() needs to be called. Never before! - The field io->xio_PutFuncPrivate is for private use. INPUT io - the xadInOut structure c - the byte to store or process RESULT res - the byte passed as argument SEE ALSO xadIO.h, xadIO.c, xadIOAlloc(), xadIOPutChar(), xadIOWriteBuf()