Short: Sector size converter, with Cache. Author: sndev@bigfoot.com (Steven Norburn) Uploader: sndev@bigfoot.com (Steven Norburn) Type: disk/misc Requires: a device to patch. Version: 0.12 This is part of range of packages designed originally for use with DVD-RAM drives, which has now become useful for other types of drives. About SectorCache: ------------------ This is a CLI Shell device patch program for changing the sector size of a device, but it also features a cache and command converter. The minimum sector size of DVD-RAM for example is 2048 bytes, the maximum PFS3 sector size is 1024 bytes, so we have a problem ! The SectorCache patch lets PFS3 think DVD-RAM is a 512 or 1024 bytes per sector device and converts the sizes accordingly. It also caches small reads by age and can convert 64 bit device commands into another form. It doesn't just work with PFS and DVD-RAM, it can work with any track disk type device or any filesystem. It can also be used to mount for example a 512 bytes per sector partition, which has been block copied to device with a larger sector size (maybe CD-R or CD-RW), the FileSystem will still see 512 byte sectors. * Warning * This is a not completely tested * BETA * and because it patches the device, any remaining bugs may destroy your data or corrupt your disk filesystem structure. * USE AT YOUR OWN RISK ! * Like all patches, the code can also be corrupted by other programs, it's only therefore recommended to be used when needed. Usage: SectorCache Device/A,Unit/A/N, "Device/A,Unit/N," "E=EmuSize/N,D=DevSize/N,C=CacheBlock/N,L=LineLength/N,S=SetSize/N," "ETD64/S,NSD64/S,TD64/S,SD64=SCSIDirect64/S,SD=SCSIDirect/S," "SCL=SCSICommandLength/N," "DT=DevType/N,MT=MemType/N," "NDR=NoDirectRead/S,NDW=NoDirectWrite/S," "NCC=NoCommandConvert/S," "NC=NoCache/S,AC=AllocCache/S,MF=MemFlush/S," "WAV=WriteAndVerify/S," "SS=SwapStack/N,ATP=AsyncTaskPriority/N," "AsyncIO/S,SyncIO/S,NoRead/S,NoWrite/S," "NGG=NoGetGeometry/S,NSG=NoSCSIGeometry/S,CRC32=CheckSumCRC32/S," "MTD=MapToDevice,MTU=MapToUnit/N,MUS=MapUnitSize/N,MTZ=MapToZero/S" *New* Device/A, the name of the device or partition to patch, i.e. scsi.device or DH1, DF0 etc. Unit/A/N, the unit number of the device to patch, i.e. 3. E=EmuSize/N [DevSize], the size of the sector to emulate, i.e. 512. Note, SectorCache needs to be active before the driver is mounted ! D=DevSize/N [???], this is normally set automatically and shouldn't be changed. With this you can manually set the value. C=CacheBlock/N [4096 Bytes], this is the size in bytes of a single cache block, it will always be divisible by DevSize. When a sector is requested which is not in the cache, the whole cache block will be filled with data. Each cache block has a identification key with the age it was created. When the cache is full, the oldest cache blocks will be reused. The key is the alined sector number at the start of a cache block. L=LineLength/N [5 = 2^5 = 32], cache blocks are grouped together by dividing sectors into blocks of cache lines. A sector will always occupy the same place in a cache block and a cache block will always occupy the same place in the cache line. This helps to also improve cache searching speeds. S=SetSize/N [8], independent cache lines are grouped together in a set. This allows sectors to be cached which have the same relative position in a cache line. If a free cache block is not found in any of the lines, the oldest cache block will be reused. ETD64/S, convert some SCSI Direct and other types of 64 bit commands to New Style Extended Track Disk 64 bit commands. NSD64/S, convert some SCSI Direct and other types of 64 bit commands to New Style Track Disk 64 bit commands and ignore extended data. TD64/S, convert some SCSI Direct and other types of 64 bit commands to Track Disk 64 bit commands. *New* SD64=SCSIDirect64/S, convert other types of 64 bit commands to SCSI Direct commands. *New* SD=SCSIDirect/S, convert all read and write commands to SCSI Direct commands. *New* SCL=SCSICommandLength/N, set the SCSI command length. Supports SCSI command types 6,10,12,16 bytes, the default is 12. In SCSI Block Commands - 2 (SBC-2), READ(12) implementation is optional. If any of WRITE (6)/(10)/(12) is implemented, WRITE(16) shall also be implemented. +=================================================+ | SBC | LOGICAL BLOCK ADDRESS | TRANSFER LENGTH | |-----+------------------------+------------------| | 6 | 21 bit (1 GB) | 8 bit (256 B) | | 10 | 32 bit (2 TB) | 16 bit (65535 B) | | 12 | 32 bit (2 TB) | 32 bit (2 TB) | | 16 | 64 bit (8589934586 TB) | 32 bit (2 TB) | |-----+------------------------+------------------| | | (With logical block sizes of 512 bytes) | +=================================================+ DT=DevType/N, this changes the SCSI-2 Device Type number returned from the device. So setting DT to 0 (Direct-access device) allows HDToolBox to use DVD-RAM discs which normally returns 5 (CD-ROM). +=======================================================================+ | DT | Doc. | Description | |-------+-------+-------------------------------------------------------| | 00 | SBC | Direct-access device (e.g. magnetic disk). | | 01 | SSC | Sequential-access device (e.g. magnetic tape). | | 02 | SSC | Printer device. | | 03 | SPC-2 | Processor device. | | 04 | SBC | Write-once device (e.g. some optical disks). | | 05 | MMC-2 | CD-ROM device. | | 06 | SCSI-2| Scanner device. | | 07 | SBC | Optical memory device (e.g. some optical disks). | | 08 | SMC | Medium changer device (e.g. jukeboxes). | | 09 | SCSI-2| Communications device. | |10->11 | | Defined by ASC IT8 (Graphic arts pre-press devices). | | 12 | SCC-2 | Storage array controller device (e.g. RAID). | | 13 | SES | Enclosure services device. | | 14 | RBC | Simplified direct-access device (e.g. magnetic disk). | | 15 | OCRW | Optical card reader/writer device | | 16 | | Reserved for use by Bridging Expanders. | | 17 | OSD | Object-based Storage Device. | |18->30 | | Reserved. | | 31 | | Unknown or no device type. | +=======================================================================+ MT=MemType/S [MEMF_ANY | MEMF_PUBLIC], this forces the cache memory to be allocated using the given Memory Requirement Type, some older controller devices can only use 24 bit DMA memory. These types should work with any controller device. 512 (MEMF_24BITDMA, 24 Bit DMAable Memory, 16 MB) 2 (MEMF_CHIP, 21 Bit CHIP Memory, 2 MB) These types should work with any modern controller device. 0 (MEMF_ANY, use any type, but FAST first) 4 (MEMF_FAST, never use CHIP type memory) To use MEMF_PUBLIC memory, add 1 to the Memory Type value. To use MEMF_REVERSE memory, add 262144 (2^18). So, MEMF_24BITDMA | MEMF_REVERSE would be MT=262656. NDR=NoDirectRead/S, large reads of incremental sectors are not normally cached, this forces all reads to go through the cache system. NDW=NoDirectWrite/S, large writes of incremental sectors are not normally cached, this forces all writes to go through the cache system. NCC=NoCommandConvert/S, try to use a similar type of device commands as those originally requested. NC=NoCache/S, this disables the main cache except for one alinement correction cache block. AC=AllocCache/S, this allocates all the cache memory, this means that the cache memory is always available when needed and may also help to reduce system memory fragmentation. MF=MemFlush/S, this returns cache block memory to the system when a clear or disk change is detected. It may however increase system memory fragmentation. WAV=WriteAndVerify, this replaces the standard Write commands with a SCSI Direct WriteAndVerify 12 byte command. It maybe useful for some types of optical drives. SS=SwapStack/N [4096 Bytes], this allows the Synchronous Quick IO stack (which is needed when using FastFileSystem) to be changed. ATP=AsyncTaskPriority/N [10], this allows the Asynchronous Task Priority to be changed. AsyncIO/S, all IO is forced to be Asynchronous. Normally Quick IO requests are processed using the fast Synchronous IO system. SyncIO/S, Read and Write commands will be forced to be Synchronous (FFS doesn't like this because Asynchronous commands have to wait). *New* NoRead/S, Read commands are disabled so nothing should be read from a device, it's mainly used for testing. When used with NDR and NDW the cache can be used to create a limited virtual disk. NoWrite/S, Write, Format, Update and Clear commands are disabled so nothing should be written, it's mainly used for testing. When used with NDR and NDW the cache can be used to create a limited virtual disk. Blocked commands; TD, NSD, TD64 and SCSI_Direct except, TESTUNITREADY, REQUESTSENSE, INQUIRY, SENDDIAGNOSTIC, READCAPACITY *New* NGG=NoGetGeometry/S, disable the trackdisk GetGeometry command, this stops some filesystems from ignoring the mount settings and getting the geometry directly from the drive. *New* NSG=NoSCSIGeometry/S, if a device doesn't support the GetGeometry command, SCSI Direct is used to create a fake GetGeometry, this will not happen with this option. *New* CRC32=CheckSumCRC32/S, CheckSums to protect the Cache Memory are normally calculated using the simple and fast Sum Long method as used in OFS/FFS. This allows the use of the better but slower CRC-32 (Cycle Redundancy Check 32-bit) method, as used by Zip, GZip, LZX, RAR, Arj etc. *Not Yet Implemented* MTD=MapToDevice MTU=MapToUnit/N MUS=MapUnitSize/N MTZ=MapToZero/S" For example, to use the cache with DF2: trackdisk.device unit 2, 'SectorCache trackdisk.device 2' or 'SectorCache DF2' Possible result, * Beta Release, Use at your Own Risk ! * SectorCache Removable Media, unit 2 on trackdisk.device It's a New Style (TD) device. Asynchronous Task Priority set to 10 Emulated Sector = 512 Bytes Device Sector = 512 Bytes Cache Block = 4096 Bytes Line Length = 2^5 (32 Blocks) Set Size = 8 Lines Maximum Cache = 1048576 Bytes Swap Stack = 4096 Bytes Active on Device (0x70083C4) [ After Ctrl C ] Read : Cached = 5657, Direct = 0 Write : Cached = 0, Direct = 0 Cache : Hit = 5441, Miss = 216, Made = 216, Flush = 0 IO Commands = 5721 Stack Used = 108 Bytes Removed from, unit 2 on trackdisk.device Silly example, to use HDToolBox with an emulated sector size 512 bytes, 'SectorCache scsi.device 3 512 dt=0' Possible result, * Beta Release, Use at your Own Risk ! * SectorCache Removable Media, unit 3 on scsi.device It's a New Style (TD, SD, NSD64) & SCSI Direct device. The recommended 64 bit command type is New Style (NSD64). Converting SCSI & 64 bit commands to New Style (NSD64). Changing from a Type 5 to a Type 0 device. Asynchronous Task Priority set to 10 Emulated Sector = 512 Bytes Device Sector = 2048 Bytes Cache Block = 4096 Bytes Line Length = 2^5 (32 Blocks) Set Size = 8 Lines Maximum Cache = 1048576 Bytes Swap Stack = 4096 Bytes Active on Device (0x700A08C) [ After Ctrl C ] Read : Cached = 921, Direct = 69 Write : Cached = 718, Direct = 9 Cache : Hit = 1408, Miss = 231, Made = 211, Flush = 2 IO Commands = 1464 Stack Used = 446 Bytes Removed from, unit 3 on scsi.device If you don't want the above output to be displayed, you can always redirect it to >NIL: Supported 32 bit Device Commands: --------------------------------- TD_GETGEOMETRY CMD_CLEAR CMD_READ, CMD_WRITE, CMD_SEEK, CMD_FORMAT ETD_READ, ETD_WRITE, ETD_SEEK, ETD_FORMAT Supported 64 bit Device Commands: --------------------------------- TD_READ64, TD_WRITE64, TD_SEEK64, TD_FORMAT64 NSCMD_TD_READ64, NSCMD_TD_WRITE64, NSCMD_TD_SEEK64, NSCMD_TD_FORMAT64 NSCMD_ETD_READ64, NSCMD_ETD_WRITE64, NSCMD_ETD_SEEK64, NSCMD_ETD_FORMAT64 Supported SCSI Direct Commands: ------------------------------- SCSI_READ6, SCSI_READ10, SCSI_READ12, SCSI_READ16 SCSI_WRITE6, SCSI_WRITE10, SCSI_WRITE12, SCSI_WRITE16 SCSI_INQUIRY, SCSI_READCAPACITY, SCSI_TESTREADY SCSI_WRITEANDVERIFY10, SCSI_WRITEANDVERIFY12, SCSI_WRITEANDVERIFY16 This Distribution: ------------------ This is still a beta and may have some bugs that I've missed, but the more time I take trying to find bugs, the longer it takes to get released. Any reported and confirmed bugs will be corrected in a hopefully quickly released next version. If you have any suggestions or improvement you would like to see included in the next released version, please feel free to contact me. History: -------- 0.12ß (20.07.02) Added SD64=SCSIDirect64/S, SD=SCSIDirect/S,SCL=SCSICommandLength/N Added NoRead/S, NGG=NoGetGeometry/S, NSG=NoSCSIGeometry/S Added CRC32=CheckSumCRC32/S Added Partition detection. 0.11ß (29.10.01) Non Quick Asynchronous IO is now sent by default from this Process. (FFS Forbid fix) Synchronous Quick IO now has a 4096 byte default Swap Stack. (FFS swap stack overflow fix) Added ATP=AsyncTaskPriority/N, AsyncIO/S, SyncIO/S Added NoWrite/S, SS=SwapStack/N Fixed two potentially data corrupting bugs. Added WAV=WriteAndVerify (Suggested by Mark Knibbs) Added NC=NoCache/S, AC=AllocCache/S, MF=MemFlush/S Changed CHIP to MT=MemType/S (Suggested by Mark Knibbs) Changed DType=DevType/N to DT=DevType/N Changed NC=NoConvert/S to NCC=NoCommandConvert/S A quick re-Format bug should be fixed with FastFileSystem >= 43.13. (LONG READ from, * move.l (a3)+,(a0)+ dbra d0, *) 0.10ß (24.09.01) First release. Thanks To: ---------- All the people who have sent encouragement. David Le Blanc - for SmartDisk 1.3. Dirk Stoecker - for GetCRC 1.16. Mark Knibbs - for suggestions and comments. Some Final Thoughts: -------------------- This is all Freeware ! If you have any problems, feel free to contact me. Best Regards, Steven Norburn Legal Stuff: ------------ License: -------- This is released under the concept of freeware. This means you are allowed to use and copy this archive freely, as long as the following requirements are fulfilled: All files are copied without any alterations or modifications. If any extra files are added, it must be obvious that they do not belong to the original distribution, and that they do not need to be included in any redistribution. Exception: So called "BBS ads" may not be added. The copying is done on a non-commercial basis. A small fee to cover media costs etc. may be charged. The copier is not claiming the copyright of this archive. Any exceptions from the above require a written permission from the author. If you want to publish this archive on a cover disk or similar, contact me first for approval (to make sure you have the latest version etc.). No Warranty: ------------ There is no warranty for the archive, to the extent permitted by applicable law. Except when otherwise stated in writing the copyright holder and/or other parties provide the archive "as is" without warranty of any kind, either expressed or implied, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The entire risk as to the quality and performance of the archive is with you. Should the archive prove defective, you assume the cost of all necessary servicing, repair or correction. In no event unless required by applicable law or agreed to in writing will any copyright holder, or any other party who may redistribute the archive as permitted above, be liable to you for damages, including any general, special, incidental or consequential damages arising out of the use or inability to use the archive (including but not limited to loss of data or data being rendered inaccurate or losses sustained by you or third parties or a failure of the archive to operate with any other programs), even if such holder or other party has been advised of the possibility of such damages.