Welcome to CacheCDFS!!! ----------------------------------------------------------------------------- LEGAL STUFF CacheCDFS & CDFSprefs © Elaborate Bytes, Oliver Kastl This software has been licensed to bsc büroautomation AG for use with their hardware. All other rights reserved. ----------------------------------------------------------------------------- GENERAL TandemCacheCDFS is an ISO9660 file system (DOS device driver), providing a sophisticated caching system offering a n-set associated cache. This FileSystem is restricted for use with the Tandem package, a full version for any cdrom device with additional support for RockRidge and MAC HFS file systems is available directly from Elaborate Bytes at the following address: Elaborate Bytes, Oliver Kastl Bisdorfer Weg 7 53332 Bornheim Germany The filesystem requires Kickstart version 2.04 or greater to work. INSTALLATION Let the supplied installation program do this job! It will install the FileSystem, but you can use any editor to modify the created mountlist (see later in this document). THEORY OF OPERATION Before continuing, there are a few things about cache organization that you must know. In order to make cache access as fast and efficient as possible, the main cache 'buffer' must be organized in a special way. We talk about "lines", "prefetch", "data buffer" and "minimal direct read". These values all relate very closely to sector numbers. All devices that are used to store files on, have sectors. One sector is a tiny bit of information - in case of a CDROM - 2048 bytes in size. All devices have a controlling task (a running program) called a device driver, whos primary task it is to read and write these sectors. This might be the scsi.device for an A3000 with a SCSI-CD Rom drive, or the tandemcd.device for the wonderful TandemCD package from bsc. If a file system (like CachCDFS) wishes to read sectors off a drive, it must politely ask the device driver to please fetch those sectors, thank you. This device driver is sometimes called an "Exec" device driver. The file system is a kind of protocol that describes how data is organized on the physical device. The CacheCDFS can read ISO9660, RockRidge and MAC HFS formatted disks. (Please note: The version TandemCacheCDFS, licensed by bsc büroautomation for use with the TandemCD package can ONLY read ISO 9660 formatted CD-ROMS). As the filesystem is a kind of interface between the exec device driver controlling the hardware and AmigaDOS, it is sometimes called a "DOS" device driver. The term "prefetch" is an indication of how many sectors the cache will read or write for each physical device access. If prefetch is set to 4, then this means that even if the DOS only asks for two sectors, four will be read. Since each drive access is a lengthy and time consuming procedure, much time can be saved by "guessing" what sectors will be needed next. Since the best guess is often that "the next sector will be needed next", prefetching sectors in clusters is indeed a good thing. Keep in mind that it only takes slightly longer to read four sectors than to read one. It would take much longer to read four sectors individually than to read them all in one go. Furthermore, as the FileSystem really knows, how large a file actually is, and how the cd is organized, those guesses have a very good chance to be correct. Prefetch, or "read-ahead" if you want, can boost performance a great deal, providing you set the prefetch value not too big. If you use too much prefetch it will not help you one bit. In fact, it will probably work against you, slowing everything down. This is because the next sector in sequence probably is NOT the next sector that will be needed, and so the cache will be reading much more than it should, on each access. The morale is to use prefetch with some thought. Generally a prefetch of 2 or 4 gave very good results on ISO9660 formatted disks. Moving on to the term "lines". Lines is a means of grouping together sectors in "blocks" of a certain length. If a cache buffer uses 32 sets of lines, this means that the cache can remember up to 32 different entries, with each entry consisting of 'prefetch' number of sectors. To illustrate this, consider the following sketch of a 32 line cache, with a prefetch of 4: Set number Line contents 0 [sector][sector][sector][sector] (4*2048=8k) 1 [sector][sector][sector][sector] (4*2048=8k) 2 [sector][sector][sector][sector] (4*2048=8k) 3 [sector][sector][sector][sector] (4*2048=8k) 4 [sector][sector][sector][sector] (4*2048=8k) ....etc.... 31 [sector][sector][sector][sector] (4*2048=8k) 32*8k=256k Each line can hold any four consequtive disk sectors with this prefetch configuration. If the file system asks for sector number 8, the cache will read sectors 8, 9, 10 and 11 into one of the lines. The cache will choose the actual line number to use based on the sector number, but the exact procedure is irrelevant here. By now it must be obvious that more lines also mean more space available for sector data, and therefore a higher chance that a read or write operation will be able to complete with no or only little physical disk access. Unfortunately, it also means you have to set aside more memory for the buffers. A compromise you often encounter with computers. A typical cache configuration is an 50-set, cache, with a prefetch (line size) of 4. The memory required by such a cache buffer will be: 50 sets x (4 x 2048 bytes) = 400k bytes The CacheCDFS has also a special buffer, called "data buffer". This buffer has always the multiple size of one line. If the line size is 4, and the data buffer has a size of 4, it is organized like this: Data Buffer [line][line][line][line] (4*4*2048=32k) If the FileSystem does caching (it does not always, see the term "direct read" later), it checks the actual file length. In case the file is longer as the amount actually requested, the rest is read immediatly into the data buffer, and then copied to the appropriate cache line (Yes, the technical informed reader will know this method... it is called "double buffering"). The main different between the Data Buffer "read ahead" and the normal "read ahead" is as follows: The cache lines are always filled (a "classic" read ahead cache), so even different directory sectors or sectors from different files are cached. The data buffer is a more "intelligent" special read-ahead cache line, existing only once and caching only files, if the filesystem "believes" they are worth it. So, you can do nice caching even on machines with not much free memory. Look at this: 50 sets with a line size of 1 (no "directory" prefetch!), but a data buffer of 8. 50 sets x (1 x 2048 bytes) = 100k bytes + 8 x ( 1 x 2048 bytes) (data cache) = 116k bytes This gives pretty good results, even on a machine with not much memory available. Now moving to the term "direct read". If you read large files (e.g. "off the disk animations") it is pretty useless to cache all these blocks. The file will purge all other sectors out of the cache, even such blocks like directory entries. To avoid this, you can specify a "minimum direct read" value. If DOS requests more sectors than specified here, the filesystem will bypass the cache and the device driver will copy its data directly into the callers buffer. However, if any sectors are already found in cache memory, these will be used, of course. Specifying a value of zero (0) will give you a pretty much sophisticated default. A very high value (e.g. 2000) will only do requests greater than 4MB directly, what is very unlikely to happen. Specifying a value smaller or equal to the data buffer size is not very useful, as you would in fact waste memory. For best performance, use the default (0). Before doing direct read, the Memory Type and Memory Mask of the callers buffer will be checked, if it fits to the device requirements (specified in the mountlist as BufMemType and Mask, see later). If it doesn't, tha data buffer will be used as a cache. It's simply not possible for me to present you with the recipie on how to make a perfect cache. The range of possible combinations of all the above values and the memory you which to spend for caching is simply too vast. You will have to experiment to find the combination that best suit your needs. The default setup installed by the installation program is a 50 set cache with 2 sectors standard "directory" prefetch and a data buffer size of 4. The memory required for this setup is shown below: 50 sets x (2 x 2048 bytes) = 200k bytes + 4 x (2 x 2048 bytes) (data cache) = 216k bytes The "minimum direct read" value defaults to zero, giving a nice default. In fact, in the above example, direct read will start with requests greater than 16k bytes. You can adjust these values in the filesystem's mountlist entry. MODIFYING THE MOUNTLIST The installation program has created a mountfile for you. The CDFSprefs program allows you to change some values interactively or to "try" settings without storing it in the mountfile. The values that CDFSprefs is not able to change have been preset by the installation program and don't need to be changed. You may of course modify values using a standard editor (like "ED" or "MicroEmacs" from your system disks), but this is strongly discouraged. The created mountlist may look like this: /***************************************************************/ /* TandemCD mountlist entry © 1993 Elaborate Bytes, O. Kastl */ /***************************************************************/ FileSystem = L:CacheCDFS /* The name of the game */ Device = "tandemcd.device" /* Name of exec device driver */ Unit = 0 /* exec device unit */ Flags = 0 /* OpenDevice flags */ BlocksPerTrack = 1 /* Unused */ BlockSize = 2048 /* True, but unused */ Mask = 0x7ffffffe /* Memory mask for direct read */ MaxTransfer = 0x100000 /* Maximum amount of bytes for direct read */ Reserved = 0 /* Unused */ Interleave = 0 /* Unused */ LowCyl = 0 /* Unused */ HighCyl = 5000 /* Unused */ Surfaces = 1 /* Unused */ Buffers = 40 /* Number of cache lines */ BufMemType = 1 /* MEMF_PUBLIC, use whatever you require */ GlobVec = -1 /* Do not change! */ Mount = 1 /* Mount it immediately */ Priority = 10 /* Priority of FileSystem task */ DosType = 0x43443031 /* Unused, subject to change */ StackSize = 3000 /* Minimum stack required is 3000! */ Control = "MD=0 LC=2 DC=4 L LFC=1" /* The Control field is for special adjustments */ /* L/S convert all file names to lowercase */ /* LV/S convert volume names to lowercase */ /* LFC/N start converting at this character */ /* LC/N/A number of blocks per cache line */ /* DC/N/A number of cache lines for the data cache */ /* MD/N/A number of blocks, when starting direct read, not using */ /* the cache. 0 will be a reasonable default (LC*DC+1) */ /* S/S Do SCSI direct commands, no Trackdisk like commands! */ /* NC/S Do NOT use TD_ADDCHANGEINT, poll for DiskChange! */ /* M/S Issue a TD_MOTOR (OFF) command after read */ Most of the comments above are self explaining. To modify the Mask, MaxTransfer and BufMemType consult the manual of your controller. The above values are correct for the TandemCD package, and shouldn't be changed. The most interesting fields are: Device: Name of the exec device driver. This can be tandemcd.device, ALF.device, oktagon.device or scsi.device, depending on the hardware you are going to use. Please note, that the TandemCacheCDFS (licensed by bsc) will ONLY work with the tandemcd stuff! Unit: Unit number. In case of tandemcd usually 0, only if you use more than one tandem board at the same time you may need to change this. With scsi host adapters, this is usually the SCSI-ID of your cd rom drive. Buffers: This is the default amount of sets (number of prefetch lines), that the filesystem will allocate. You can change this value during operation with the CLI command "addbuffers". Please consult your AmigaDOS manual. Control: Here various parameters can be changed. L: This will force the filesystem to convert all ISO file names to lowercase. Most ISO disks are mastered with all file names uppercase. If you find this annoying, you may specify L. LV: This will force the filesystem to convert ISO volume names to lowercase. Most ISO disks are mastered with its name uppercase. If you find this annoying, you may specify LV. LFC: lower first character. Enter the number of the character in a name, when conversion to lower case should begin. E.g. LFC=0 means, that the whole name will be converted. LFC=1 leaves the first character as it is and converts the rest of the name to lowercase. If neither "L" nor "LV" are specified, this option is ignored. LC: LC stands for "LineCache". Here you enter the amount in sectors of the "standard" read-ahead cache (line size). The default value is 2. The minimum value is 1 (no pretetch). DC: DC means "Data Cache". Here you enter the size of the data buffer in multiples of line caches. (e.g.: in the above example, the data cache is 16k: 4 * line size, and line size is 2 * 2k) MD: MD is the "minimal direct read value". A read request greater or equal the amount of sectors specified here will bypass the cache and transfer data "directly" to the callers buffer, if the buffer fulfills the requirements specified in BufMemType and Mask and the length of the request is not greater as the value specified in MaxTransfer. 0 will give you a nice automatic calculated default, based on the algorithm MD = LC x DC + 1. The following control entries are only for some special configurations: S: This will cause CacheCDFS to use SCSI-direct commands instead of the standard "trackdisk-like" commands. This will only work with a SCSI host adapter and a SCSI cdrom drive, of course. This option is useful for older SCSI host adapters (e.g. very early ALF scsi controllers) having problems with CDROM drives. You should only specify this option, if you encounter problems. M: This will cause a TD_MOTOR off command to be issued after CMD_READ. It is ignored, if "S" is set. This option is neccessary for some scsi adapters, that issue a "prevent medium removal" command (e.g. GVP-Series II). If you can't remove an inserted CD, try this option. NC: This is the "NoChangeInt" option. It will tell the filesystem, NOT to install a diskchange interrupt server. Instead it will "poll" every three seconds, if the medium has been changed. It is useful for host adapters having trouble to recognize a disk change with certain cd rom brands (e.g. Commodores A2091 and Apple CD Rom. Specify "S" and "NC", and it will work). You must use this option, too, if your controller rejects working without the S option described above. Only specify NC if you REALLY encounter any problems with medium change recognition! ERRORS WHILE MOUNTING Mounting may indeed fail, if a severe error condition occures. The filesystem gives you lots of nice error messages, described below. "Filesystem startup invalid!" Come on... you tried to mount the filesystem as a handler. "Environment vector invalid!" Same as above. "Control entry in mountlist missing!" You must specify a "Control" field in your mountlist for the filesystem to work! "Error while parsing Control field in Mountlist: ..." You specified something weird in the control field. Check your mountlist! "Illegal number of cache lines (Buffers)" "Buffers" must be greater than zero. "Illegal value for data cache" DC in control field must be greater than zero. "Illegal number of sectors for cache line size" LC in control field must be greater than zero. "Illegal number of sectors for minimal direct read" MD in control field must be greater or equal than zero. "Out of memory! Cannot allocate buffers!" Should be obvious. Reduce "Buffers", or DC and LC in control field. "Cannot open ...device unit ..." The hardware driver for your cdrom can't be opened for some reason. This can have various reasons. Check the following: Is the driver present? Is the device name corecctly spelled? Note that the device name is case sensitive! Is the correct unit number specified? Is the hardware (e.g. the cdrom controller or scsi-host adapter) present? Is the cdrom correctly connected? Is it switched on? "You really shouldn't start this program directly!" You loaded the filesystem as a CLI command by typing its name. Very funny. ERRORS DURING OPERATION The cdrom does not respond at all: If you are using a scsi host adapter, try specifying the "S" flag in the control field in the mountlist. The filesystem does not detect medium changes: Probably the hardware driver has problems with your cdrom drive. Try specifying the "NC" option in the control field in the mountlist. You can also manually cause a diskchange by using the "DiskChange" command from the CLI. END OF DOCUMENT