This file forms part of PFS3 (Professional File System 3) Written by Michiel Pelt Copyright 1999 by GREat Effects Development $Id: Proginfo 1.7 1999/06/30 18:38:08 Michiel Exp Michiel $ 13 May 1999 PFS3 disk structure ------------------- This document describes the blocks and structures used by PFS. Only the new harddisk version will be described here. For the floppy version you can refer to previous releases. ********************* * the bootblocks ********************* The first two blocks are the bootblocks, as required by AmigaDOS. The first longword of the first bootblock contains the dostype: PFS\1. The rest is empty. So the first bootblock looks like this: ___________________ | | 000 | disktype | |___________________| 004 | | | | | . | | . | 200 |___________________| ******************** * the rootblock ******************** Directly behind the bootblocks you can find the rootblock, on block 2. This is the most important block of a PFS disk. PFS keeps this block cached at all times. It looks like this: ___________________ | | 000 | disktype | 'PFS\1' 004 | options | see below 008 | datestamp | |___________________| | | | 00c | cday | cminute | creation date (DOS format) 010 | ctick | prot | protection |_________|_________| | | 32 character diskname. Size of string followed 014 | diskname | by that number of characters | | |___________________| | | 034 | last reserved | first and last reserved block (blocknumbers) 038 | first reserved | 03c | reserved free | number of reserved blocks free |___________________| | | | 040 | blksize | rblkclt | reserved blocksize; rootblock cluster |_________|_________| | | 044 | blocks free | number of blocks free 048 | always free | number of blocks always kept free 04c | roving pointer | used for allocation 050 | deldir | (<4.3) the location of the delete directory 054 | disksize | size of the disk in blocks 058 | rblk extension | reference to rootblock extension 05c | reserved_2 | |___________________| 060 | | | bitmap index | reference to bitmap index blocks | | |___________________| 074 | | | anode index | reference to anode index blocks | | (not if MODE_SUPERINDEX enabled) 200 |___________________| With the disktype field PFS3 recognizes that the disk is a PFS disk. It is the same as what can be found in the bootblock. Options is a 32 bit field which specifies which PFS3 extensions are enabled. The table below states the bit number, the symbolic name, the release in which the option was introduced and its meaning. bit | name | rel | function ----|----------------------|-----|----------------------------------------- 0 | MODE_HARDDISK | 4.1 | harddisk mode if 1; currently always 1 1 | MODE_SPLITTED_ANODES | 4.1 | anodes splitted in block/offset if 1 2 | MODE_DIR_EXTENSION | 4.1 | enables directory-entry extension 3 | MODE_DELDIR | 4.1 | deldir enabled if 1 4 | MODE_SIZEFIELD | 4.1 | the disksize field (offset 0x54) is valid 5 | MODE_EXTENSION | 4.1 | the rootblock extension is enabled 6 | MODE_DATESTAMP | 4.1 | block datestamps enabled if 1 7 | MODE_SUPERINDEX | 4.2 | extension that makes disks >5G possible 8 | MODE_SUPERDELDIR | 5.0 | extension that makes larger deldir possible 9 | MODE_EXTROVING | 5.0 | reserved roving pointer is per block (was 32) 10 | MODE_LONGFN | 5.0 | support for long filenames PFS3 has, by default, all options enabled. One exception to this is MODE_SUPERINDEX, which is only enabled for disks larger than 5G. If MODE_SUPERINDEX is enabled, the list of bitmap index blocks is larger and covers offset 060 to 200 of the rootblock. The anodeindex blocks are referenced by a new blocktype: the superblock which can be reached from the rootblock extension. PFS3 checks the options when the volume is mounted. If MODE_DELDIR is cleared, PFS3 will automatically create a delete directory and set MODE_DELDIR. The rootblock extension is also automatically created if not present. If future versions of PFS3 extend the diskstructure this will be reflected by a new option bit. All 'not_used' fields are reserved for future use and must be set to 0. First and last reserved indicate the 'reserved area'. All disk information, like directories, allocation data etc are stored in this reserved area. The rest of the disk is available for data. Currently firstreserved always is 2, so the reserved area starts with the rootblock. 'Reserved free' indicates the number of reserved blocks that are unoccupied. The 'datestamp' is an update counter. When the disk is formatted, all blocks will have a datestamp of 0. Every time the disk is updated (goes from one state to the next using atomic commit), the datestamp is increased by one. Using the datestamp all blocks in the reserved area can be ordered in time. This is usefull for repair tools enabling easier and more reliable recovery of a broken volume. 'Blksize' indicates the size of reserved blocks in bytes. Currently this is always 1024 (1K), independent of the actual blocksize. The actual blocksize (as specified in the mountlist) can be either 512 or 1024 byte. So if blocks are 512 byte, which is the normal case, each reserved block takes two blocks on disk. BTW: the before mentioned first and last reserved indicate real blocknumbers, not reserved block numbers. However, 'reserved free' does refer to reserved blocks. It indicates the number of reserved blocks that can be allocated before the reserved area is full. Blocks free indicates the number of free blocks. It is equal to the total number of blocks on the disk - the number of reserved blocks - 2 - the number of blocks taken by data. PFS always keeps a small part of the data area free. This is indicated by 'always free'. The roving pointer is used for allocation. If you change any bitmap blocks it is wise to set this pointer to zero. This will never hurt. ******************** * Block ID ******************** The following blocks each have a 2 byte block id field. This indicates the blocktype. Currenly used IDs are: DB -- dirblock AB -- anodeblock IB -- anode index block BM -- bitmap block MI -- bitmap index block DD -- deldir block EX -- rootblock extension SB -- super block ******************** * Bitmap blocks ******************** To keep track of which blocks are occupied and which are not, PFS uses bitmap blocks. Such a block looks like this: ___________________ | | | 000 | id | x | |_________|_________| | | 004 | datestamp | 008 | sequence number | |___________________| 00c | | | the bitmap | | | 400 |___________________| Each block has room for 253 longwords allocating 8096 blocks. Each bit represents a block and a bit is set if it is available, and clear when it is taken. There are two bitmaps, one for the reserved area and one for the data area. For the data area several bitmap blocks are needed. They are numbered with the sequence number field, starting with 0. The first bit of the first bitmap block represents the first data block (the block directly behind the last reserved block) The bitmap for the reserved area is directly behind the rootblock. The rootblock cluster field in the rootblock (rblkclt) indicates the number of blocks occupied by the rootblock together with the reserved bitmap. The reserved bitmap has the same header as the data bitmap, but has a different size. The rootblock cluster is chosen such, that it can contain the rootblock itself and the entire reserved bitmap, and it is rounded up to a multiple of 1024 byte blocks. The first bit of the reserved bitmap represents the first reserved block (usually 2). Each bit represents a 1024 byte reserved block. ******************** * Index blocks ******************** PFS refers to bitmap and anode blocks by using index blocks. These blocks look like this: ___________________ | | | 000 | id | x | |_________|_________| | | 004 | datestamp | 008 | sequence number | |___________________| 00c | | | 253 indices | | | 400 |___________________| Each block contains references to 253 anode or bitmap blocks. For large disks several index blocks will be needed. They are numbered with the sequence number, starting with 0. There is a maximum of 5 bitmap index blocks and 99 anode index blocks. The rootblock contains the pointers to these blocks. An anode or bitmap block of a certain sequence number doesn't exist if the corresponding field in the index is zero. Holes in the index are allowed (e.g. it is possible for anode block 5 to exist, while 4 doesn't). ******************** * Anode blocks ******************** Files and directories take space on the disk. PFS keeps track of which objects takes which blocks by means of so called anodes. Anodes are stored in anode blocks, which look like this: ___________________ | | | 000 | id | x | |_________|_________| | | 004 | datestamp | |___________________| | | 008 | sequence number | |___________________| | | 00c | reserved | |___________________| 010 | | | 84 anodes | | | 400 |___________________| The way anodes are numbered depends on the MODE_SPLITTED_ANODES: MODE_SPLITTED_ANODES disabled: The anodes are numbered starting with 0. The first 84 are in anodeblock 0, the next 84 in anodeblock 1 etc. Each object (file or directory) is identified by such an anode number. MODE_SPLITTED_ANODES enabled: The first 16 bit of the anode indicate the anodeblock by sequence number. The following 16 bit indicate the offset within the anodeblock. So anode 0x00010004 is the fifth anode in the second anodeblock. An anode looks like this: ___________________ | | 000 | clustersize | 004 | blocknr | 008 | next | |___________________| The clustersize indicates how many blocks are allocated by this anode. The blocknr is the first block allocated. So an unfragmented file can always be allocated by a single anode. A fragmented file, however, needs one anode for each fragment. For that purpose is the next field. It indicates the anode number of the next fragment. If this is the last fragment, the next field will be zero. The clustersize and blocknr in an anode always refer to real block numbers, even if the blocks are in the reserved area. If all fields of the anode are zero, it is considered free and can be reused. It is possible to associate an anode with a file without allocation any blocks. This can be done by setting the clustersize and next fields to 0 and the blocknr field to -1. The first six anodes have a special meaning: anode name use ---------------------------------------------- 0 ANODE_EOF should always be 0 1 not used, set to 0 2 not used, set to 0 3 not used, set to 0 4 ANODE_BADBLOCKS Bad block list. Not used by PFS3. Repair tools can chain bad blocks to this anode 5 ANODE_ROOTDIR The anode of the root directory. You can find the root directory by looking at this anode All anodes from 6 and higher are used for directory and file allocation. ******************** * dirblocks ******************** Directories have an anode number, just like files. Each anode points to one directory block of 1024 byte (so, with 512 byte blocks, the clustersize field of an anode belonging to a directory is always 2). If 1024 byte is not enough for the directory, a new block with a new anode is allocated and chained to the first. Two anodes belonging to the same directory are never joined to form one (by increasing the clustersize). A dirblock looks like this. ___________________ | | | 000 | id | x | |_________|_________| 004 | | | datestamp | |___________________| | | 008 | reserved | |___________________| | | 00c | anodenr | 010 | parent | |___________________| 014 | | | directory entries | 400 |___________________| The anodenr (anode number) refers to the anode belonging to the directory, not the anode belonging to the block. So it always points to the head of the anode chain, and is the same for all blocks of the same directory. The parent is the anode number of the parent of this directory, or 0 if this is the rootdirectory (which doesn't have a parent). A directory entry looks like this: ___________________ | | | 000 | next | type | |_________|_________|___________________ | | 002 | anode number | 006 | size | |_______________________________________| | | 00a | creation day | 00c | creation minute | 00e | creation tick | |___________________| | | | 010 | prot | nlength | |_________|_________| | | 012 | name | | | |___________________| 012 + | | nlength | flength | |_________|_________ 013 + | | nlength | filenote | | | |___________________| | | MODE_DIREXTENSION only | extension | | | |___________________| The next field indicates the length of this directory entry in bytes. This must always be even. The type is the type of direntry, just like FFS: ST_FILE, ST_USERDIR etc. See include file for details. The anode number is the head of the anode chain for this object. The size is the filesize in bytes. The number of blocks in the anode chain must correspond to the number of blocks needed to store 'size' bytes. The creation day, minute and tick is the DOS timestamp of the object. The protection are the lower 8 file protection bits, like FFS. The name and the filenote are pascal strings (length followed by that number of characters). They are placed directly behind each other and are byte aligned. If the file has no filenote, flength will be zero. Each directory entry is padded if that is needed to word align the next directory entry. The end of the directory block is indicated by appending a zero at the end of the last entry. The directory ends with the last entry in the last block, where the blocks are chained by anodes. ******************** * dirextension ******************** If MODE_DIREXTENSION is enabled some optional extra WORD size fields are added after the word align pad. The last extension field is a WORD size bitfield that indicates which fields are present. Currently the extra fields include: ___________________ | | | link (high) | Anodenr of link |___________________| | | | link (low) | |___________________| | | | uid | User ID |___________________| | | | gid | Group ID |___________________| | | | prot (high) | 32 bit protection field |___________________| | | | prot (low) | |___________________| | | | virtualsize (high)| 32 bit virtual size (for rollover files) |___________________| | | | virtualsize (low) | |___________________| | | | rollpointer (high)| 32 bit rollpointer (for rollover files) |___________________| | | | rollpointer (low) | |___________________| | | | flagfield | Only field that always is present |___________________| 'Link' is for links, described later. The prot field is combined with the 8 bit protection field direntry->protection (OR operation) to form a 32 bit protection value. If the upper 24 bit are zero, this prot field will not be used. The filesystem only checks direntry->protection when accessing files. The following code fragments show how to extract and make extrafields. Used macros and types can be found in 'pfs.h'. void GetExtraFields (struct direntry *direntry, struct extrafields *extrafields) { UWORD *extra = (UWORD *)extrafields; UWORD *fields = (UWORD *)(((UBYTE *)direntry)+direntry->next); UWORD flags, i; flags = *(--fields); for (i=0; i < sizeof(struct extrafields)/2; i++, flags>>=1) *(extra++) = (flags&1) ? *(--fields) : NULL; } static void AddExtraFields (struct direntry *direntry, struct extrafields *extra) { UWORD offset, *dirext; UWORD array[16], i=0, j=0; UWORD flags = 0, orvalue; UWORD *fields = (UWORD *)extra; offset = (sizeof(struct direntry) + (direntry->nlength) + *FILENOTE(direntry))&0xfffe; dirext = (UWORD *)((UBYTE *)(direntry) + (UBYTE)offset); orvalue = 1; /* fill packed field array */ for (i=0; i < sizeof(struct extrafields)/2; i++) { if (*fields) { array[j++] = *fields++; flags |= orvalue; } else { fields++; } orvalue <<= 1; } /* add fields to direntry */ i=j; while (i) *dirext++ = array[--i]; *dirext++ = flags; direntry->next = offset + 2*j + 2; } BTW: if MODE_DIREXTENSION is enabled, the flags field must always be present, even if none of the other fields are used. An unused field is assumed to be zero. ******************** * hard links ******************** The main object (the object that is linked to) has type ST_FILE (ST_DIR) like an ordinary file (dir). Its anodenr points to the object data, as usual. The link field in the directory extension points to the head of the LINK LIST. The LINK LIST is mapped on an anode list. The 'link' is nothing more than an anode number. The fields have been redefined, however: ___________________ | | 000 | object dir | 004 | link dir | 008 | next | |___________________| Each entry in the LINK LIST represents a link (the object itself doesn't have a LINK LIST node). The 'object dir' is the anode number of the directory where the object can be found. The 'link dir' is the anodenr of the directory where the link can be found. 'Next' is the anodenr of the next LIST NODE or 0 for end of list. A link has type ST_LINKFILE (ST_LINKDIR). Its anode number points to its LINK LIST element. The link entry in the directory extension is the anode number of the object. A link can be resolved as follows: · Get LINK LIST element (anodenr in directory) · Get object dir (object dir field) · Locate object by looking for anodenr as stated in the directory extension link field. BTW: the anode number as stated in the directory is an unique directory entry identifier, even for links. ******************** * soft links ******************** A softlink is a file of length strlen(target) and type ST_SOFTLINK. The block containing the name of the target can be located by reading the link as a file (anode.blocknr). ******************** * deldir ******************** The deldir is a hidden directory that contains the 31 most recently deleted and overwritten files. It can be accessed from the root by its name '.deldir'. The delete directory looks like this: ___________________ | | | 000 | id | x | |_________|_________| | | 004 | datestamp | |___________________| | | 010 | roving | (<4.3) current delete position 012 | uid | (<4.3) deldir user id 014 | gid | (<4.3) deldir group id 016 | protection | (<4.3) deldir protection (LW) 01a | creationday | (<4.3) 01c | creationminute | (<4.3) 01e | creationtick | (<4.3) |___________________| 020 | | | | | 31 deldirentries | | | 400 |___________________| The deldir contains 31 slots containing deldirentries. These slots are filled in a round-robin manner, using 'roving' as position pointer. A deldirentry looks like this: ___________________ | | 000 | anodenr | 004 | size | size in bytes 008 | creationday | datestamp (copied from original file) 00a | creationminute | 00c | creationtick | |___________________| 00e | | | filename | filename, truncated to 17 characters 020 |___________________| 'filename' is a pascal string; length followed by 17 characters. When a file is deleted or overwritten it is added to the deldir in the slot indicated by 'roving'. Only files are put in the deldir, no directories or links. The diskspace taken by these files is freed at the moment the original file is deleted, so an entry in the deldir can become invalid at any time data is written to the disk. When the deldir is scanned (e.g. by calling "dir pfs:.deldir") PFS will check the vadility of each entry and will return only the valid entries. The scan routine adds the deldir slot number to each entry (e.g. foo@003) to make sure each entry has an unique name. Files in the deldir should also be accessed by that name. The anodechain taken by a deldirentry is not freed until the deldirentry is reused. ******************** * rootblock extension ******************** AFS version 16.4 introduced the rootblock extension. This is a reserved area block that contains general information about the disk that didn't fit in the rootblock. The block is referenced by the 'extension' field in the rootblock. Its presence is indicated by the MODE_EXTENSION bit in the rootblock options field. PFS3 automatically creates the rootblock extension on older disks, so every disks that is operated on PFS3 should have an extension block. The rootblock extension looks like this: ___________________ | | | 000 | id | x | |_________|_________| | | 004 | ext. options | flags for (future) options. Currently 0. 008 | datestamp | 00c | pfsversion | PFS revision id. |___________________| | | 010 | rootdate day, min | the root directory datestamp |___________________| | | 014 | tick | |_________|_________ | | 016 | voldate day, min | the volume datestamp |___________________| | | 01a | tick | |_________|_________ | | 01c | postponed - | | operation | |___________________| | | 02c | reserved roving | reserved area roving pointer |___________________| | | 030 | roving bit | roving pointer bitnumber (normal area) |___________________| | | 032 | current anode seq | roving pointer for anode allocation |___________________| | | | 034 | dd_rov | dd_size | roving pointer and size of deldir |_________|_________| | | 038 | fnsize | filename size (18 +) |_________|_________ | | 03a | x | |___________________| | | 040 | superindex | MODE_SUPERINDEX only: upto 16 superblocks | | | · | | · | |___________________| | | | 080 | dd_uid | dd_gid | uid and gid of delete directory |_________|_________| | | 084 | dd_protection | protection of delete directory |___________________| | | 088 | deldir day, min | deldir datestamp |___________________| | | | 08c | tick | x | |_________|_________| | | 090 | deldir | upto 32 deldir blocks | | | · | | · | |___________________| 110 | . | | . | | . | 400 |___________________| PFS stores the current revision id (version number in high word, revision number in low word, e.g. $00100004 for version 16.4) in the 'pfsversion' field when formatting a disk. This way the oldest PFS revision that accessed the volume can be identified. The two new dates, rootdate and volumedate, complement the creationdate in the rootblock. The three datestamps are used as follows: Volume creation date (rootblock): Only set by format and by ACTION_SERIALIZE_DISK. Volume date (extension): Zerod by format. Touched each update. Root date (extension): Root directory datestamp. The postponed operation is a 4 longword field used internally by PFS3. It is to be considered an error if these fields are non-zero on an inhibited PFS3 volume. The superindex is for MODE_SUPERINDEX only. The longwords in the superindex are references to superblocks. A superblock is an indexblock which indexes anodeindex blocks. ******************** * rollover files ******************** Rollover files are files with a filesystem enforced maximum size. When the file is appended to it will be trimmed to keep the size under the maximum. Rollover files are identified by the filetype ST_ROLLOVERFILE (== -16). The size field in the directory specifies how much diskspace is allocated for the rollover file. The virtualsize field in the extrafields indicates how much of the allocated diskspace currently is in use. This will initially be 0 and grow until it is equal to the allocated diskspace (extrafields->vitualsize == direntry->size). The virtual size can never be larger than the allocated size. The rollpointer field in the extrafields specifies the location in the allocated area where the rollover file starts. Rollover files are read circular, starting and ending at the rollover pointer. Example rollover file with 2048 allocated bytes: 0 real offsets 2047 ______________________________________________________________ | | | | | L|N|F | |____________________________|_|_______________________________| virtual offsets 2046 0 <--------------------------------------------------------------> allocated area 2048 bytes L = last byte of file N = end of file F = first byte of file The usable size of rollover files always is a multiple of the blocksize minus one.