Reading the unreadable : Using ARexx   to extract information from ILBM files    by John Collett  PLUS     Iffy : A small (and limited) IFF viewer    :-) 40 :^) 40 :) 40 8) 40 8-) 40 B-) 40 :-O 40 8*)  Contents  1 Extracting brush dimensions from an IFF file 2 Extracting other information 3 Using this information to drive a image displayer An ARexx program derived from Item 3 is attached, but it is very restricted in what it can do, and produces odd results if applied to image files which are beyond its limited range. I've called it all sorts of things while I've been writing it. Finally, I've called it 'Iffy'.  Acknowledgements  I ran out of knowledge and patience while working on this, but the following documents and programs helped : · About ILBM, Docs on the IFF format, by Carolyn Scheppner (Fish Disk 185) · Amiga Rom Kernel Reference Manual, Includes and Autodocs, pages I-1 to I-164. This is a very curious collection, full of repetitions, omissions, and admissions, and on the whole about as clear as mud, though a few paragraphs gave vital information. · WhatIs, by J.Tyberghein (MD UT90) · NewZAP3.18, by John Hodgson (MD UT151)  Setting the scene  In the middle of writing an ARexx program, I may want to include a function call which will place an image in a particular position on the screen. 'Rexxarplib.library' will be on line, and so I can use call IFFImage(port, x, y, width, height, flag) 1 2 3 4 5 6 This function call is central to what follows, so we had better clarify those six arguments : 1 The name assigned to the port which was created by a preceding call to the CreateHost() function. 2, 3 The x any y coordinates of the upper left corner of the proposed image position. 4, 5 The size of the chunk to be displayed. 6 From the library docs : "The colormap of the IFF file, if any, is NOT used on the WorkBench, but IS used on custom screens. The 'flag' option can be set to 'NOCOLOR' (N is sufficient) to prevent modifying the custom screen's colors." All arguments except 1 may be omitted. The docs say that if any of 2 to 5 are omitted (just marking their place by a comma if necessary), suitable defaults are chosen. Items 4 and 5 were the catalyst which set me off on my most recent explorations, and eventually led to the writing of the accompanying ARexx program 'Iffy'. The file you are now reading is an account of the process. When placing an image I usually want to display all of it - no more and no less. So I need to be able to specify the width and height of the image accurately, but unless I happen to have noted them at the time the image was created, or obtained them by some other method, I have only a rough idea of the dimensions, and that information did not appear to be readily available.  One solution  'WhatIs' provides various pieces of information on any file, depending on the arguments attached to the command. WhatIs [PAGE ] [WHATIS | SHORT | MEDIUM | FULL | ...  ... DIAG | IDENT | ASCII ] [PATH] [NOANSI] The file holding this growing text is currently called 'Article', and when, just now, I entered 'WhatIs Article Short', I obtained :  File : Article  DirEntryType : -3   Protect : rwed   Size : 2857 NumBlocks : 3   Last changed : 07-Apr-94 12:43:02 (Thursday)  Comment :   ± 96 % ASCII chars If I use 'WhatIs' on an image file, even with the 'Short' argument, I get something like the above, and then a whole lot more, of which here are the first few lines : IFF file  FORM ILBM (2646 data bytes)   Interleaved BitMap  BMHD   width : 70 height : 96  <<------  x : 0 y : 0   nPlanes : 4 Masking : 2   Compression : 1   transparent color : 0   xAspect : 1 yAspect : 1   PageWid : 640 PageHei : 256  And there's even more under the headings of CMAP, GRAB, CRNG, CAMG, and BODY. At the moment we're seeking just 'width' and 'height', and there they are, by the arrow : 70 and 96 respectively. Now I can obtain that information from within a running program : 1 Run 'WhatIs' is required, but redirect its output into a file : MyCommand = 'WhatIs ' || filename || ' Short >Ram:temp'  address command MyCommand  2 Open the 'temp' file, find the line containing 'width' (by whatever method, possibly similar to the one described later on for locating headers in graphics files), and close the file. 3 Parse the line into its pieces, and retrieve the values we need. 'WhatIs' is handy to have around, and I used it for as thorough a check as possible of much of what follows. Though the above method may sound clumsy, I can fall back on it when my own attempts at extracting data from image files fail (as they often do - see below). But when I am on fairly safe ground, such as handling files which have been created or processed via DeLuxe Paint, I can now do it more elegantly than via 'WhatIs'. To achieve that, I needed to review and extend my knowledge of IFF files. I turned to Fish Disk 185, which concentrates on that topic, and to NewZAP3.18, which I could use to see the contents of files (though the word 'see' should not here be taken to mean 'identify and understand at a glance').  From the docs  To start with, the documents are not completely clear, and full of warnings about over-confidence. And when the only recognised methods of representing data are unprintable characters and groups of hexadecimal characters which vary in length, you know that trouble lies ahead. The following are some of the main points to come out of the documents on Fish Disk 185, and if you think my summary errs on the side of vagueness or inaccuracy, you're right.   The only readily recognisable combinations of characters in a graphics file may include some but not necessarily all of the following 'headers' : FORM ILBM BMHD CMAP CRNG CAMG BODY.   Other such headers may be found, but not all that are found are orthodox, recognised, documented, or registered.   Attempts to encourage all makers of graphics files to adopt the same structure have been in vain. You can't rely on anybody doing what they have been asked to do.   Other than the content of the first few bytes, you cannot assume that any piece of information in an IFF file will be in a particular position. Even that first few ... well, it's best to check.   Data between headers should include one item which indicates how many bytes there are to the next header. For example, if a translation into plain English reveals 'CMAP0024', the next 24 bytes of 'colour mapping' will be eight sets of RGB colour settings (8 * 3 = 24). You know how many colours there are, and can unravel the settings for each of them.   The CAMG group contains the file's ViewMode, and pigs might fly.   And there's an awful lot more ....  Using an editor to investigate  To clarify, extend, and perhaps confirm what I had read in the IFF docs, I used NewZAP3.18 to look more closely at the contents of IFF files. The main display looks something like this :  ------------------------------------------------------------------ | 8 columns of 8 digits, each set of | 32 characters, | | digits containing 4 pairs of | only some of which| | hexadecimal values. | are recognisable. | . . . . . . ... and it goes on like that, line after line, for ages. In the case of a sample which I have just used as a source, the first left-hand set of 64 characters looked like this : 464F524D 00000A56 494C424D 424D4844 00000014 00460060 0000000 04020100  These are in hex, and to satisfy my curiosity I transcribed a few of them, a pair at a time, into something more recognisable, by using the ARexx 'x2d' function (converts a hex string into a character equivalent) : The first pair : x2c(46) = F The first eight : x2c(464F524d) = FORM The next eight : x2c(00000A56) = a carriage return and V ... (?) The third eight : x2c(494C424D) = ILBM .... Some of it made sense, and agreed with what I saw in the corresponding right-hand set : | |FORM VILBMBHMD F ` | But this first line is not typical, in that some of it is readable at sight. Most of what follows on the right-hand section of the screen is either invisible or undecipherable. On the left-hand side it may not be comfortable to read, but it is visible, and can even be changed and saved if you are really courageous. Rashly, I began to believe that it would be possible to scan a graphics file for any header which I expected to be present, and then to examine the following set of bytes for specific pieces of information. I started to draw up an ARexx program for the purpose of extracting, first, the width and height of an image, and then anything else which might be of interest and not too complicated. Before reaching the limits of my patience and understanding, I managed to obtain just a little bit more than the particular pieces of information I wanted.  Probing with ARexx  Let's work our way through some of the steps. These seem to work. Assume that we are looking into a file called 'thispic'. We shall use the presence in that file of the headers 'FORM' and 'ILBM' as a fairly reliable indicator that it is a graphics file, and we shall also want to locate the strings 'BMHD', 'CMAP' and, later on, CAMG. 'FORM' alone may be enough for the first check. For the next level of analysis the second step suggested is barely adequate, and there are other headers, but this will do for now. The method I use to find a given group of characters in a file is to read the whole file into a single string. The ARexx User's Reference Manual says (p.109) that strings are limited to a maximum length of 65,535 bytes, a restriction which I have not found particularly onerous, though I suppose full-screen interlace pictures may well get up into that range. The file can be read into a string a byte at a time until end-of-file is detected, or, if the length of the file is known, in a single move. The latter is clearly more efficient, so we need first to obtain the size of the file. The following line parses out the size from the information returned by the ARexx function 'statef()'. parse value statef(thispic) with type size other We open the file : op = open(p,thispic,'r')  Read it into a string : str = readch(p,size)  And close it : cl = close(p) Check that it is an IFF file. Despite what was said above about not relying on the position of anything, I expect the first four bytes to be 'FORM' and the third four to be 'ILBM'. If such is not the case, I conclude that the file is probably not an IFF file. (Since I first wrote this I have met a few cases in which this test rejects files which I know to be image files!) form = index(str,'FORM')  ilbm = index(str,'ILBM')  if form ~= 1 & ilbm ~= 9 then ...  ... exit with a message. With some help from the docs but mainly by trial and error, I found that the following arrangement of bytes contains the width and height (ww and hh) of the image : BMHD....wwhh. So, if 'BMHD' is at position p, then width will be at p+8 and p+9, and height at p+10 and p+11. As soon as they are found, they can be made readable by the c2d() function, which converts a character string into an integer number. bmhd = index(str,'BMHD')  width = c2d(substr(str,bmhd+8,2))  height = c2d(substr(str,bmhd+10,2)) Similarly, the number of colours can be extracted from the four bytes following 'CMAP'. I described earlier how CMAP0024 tells us that there are 8 colours. cmap = index(str,'CMAP') ncols = c2d(substr(str,cmap+4,4)) % 3 This was already more information than I had set out to find, so I started building an easy-to-use interface which would enable me to extract what I needed from any image file.  Building a usable module  Modularity is always being recommended as an aid to programming, and so I decided to arrange the above lines of code into a separate function that would be easy to pull into any program in which I might need it. The result is the function headed PicParams which is a part of the following short program. Some parameters are already entered, and there are no checks on anything. If you enter the path and name of an IFF file, you should get its width and height. If you enter anything else, you'll get garbage. The sole purpose of this program is to show how the function works, that is, how it obtains the required image data. To use that data, of course, you have to be in the more complex environment of a user- designed window created via 'rexxarplib.library'. We'll come to that later. From CLI, give the command : rx PicModule (or whatever you call it) The program assumes that the name of the ControlPort will be 'HO', that the upper-left corner values will be 50,30 , and that a 'NOCOLOR' flag is not being added. /* ARexx Program */  say '0a'x || ' Pic Data Module' || '0a'x Options Prompt " Enter path and file name : "  parse pull fullname /* Retains upper and lower case */   w_and_h = PicParams(fullname) /* The function call */   parse var w_and_h w h /* Divides result into two parts */ arg = " call IFFImage(HO,'" || fullname || "',50,30,"  NewString = arg || w || ',' || strip(h) || ')'  say ' The full IFFImage function call will be :' || '0a'x say NewString || '0a'x  exit  PicParams:   thispic = arg(1)  parse value statef(thispic) with type size other   op = open(p,thispic,'r') ; str = readch(p,size)   cl = close(p)  bmhd = index(str,'BMHD')   width = c2d(substr(str,bmhd+8,2))   height = c2d(substr(str,bmhd+10,2))   return width || ' ' || height  /* End of module */ I have just run the above program, and entered 'df1:pic/sixteen' as the path-plus-name. Apart from the colour used here, this was the output :    Pic Data Module     Enter path and file name : df1:pic/sixteen   The full IFFImage function call will be :     call IFFImage(HO,'df1:pic/sixteen',50,30,70,96)  | | Here they are! The width and height values obtained here happen to be the same as those reported above in the 'WhatIs' discussion because I examined the same file in each case. One way of checking results is to confirm that they are identical when they should be!  The next phase  Having got this far, I went on to extract the following data from files of both full screen and smaller images : ScreenWidth ScreenHeight PicWidth PicHeight Depth ViewMode It works with the following types of image file : Fullscreen : Hires, Lowres, Interlace, Ham  Brush : Hires, Lowres, Interlace  I have to be honest. Except for the HAM type, all the samples I tried out at this stage were created by or processed through DeLuxe Paint III. Pictures obtained from other sources did not deliver all the values sought, and I have spent hours trying in vain to improve this. The ViewMode was usually the culprit, and what I found in some of the files bore only a fleeting resemblance to anything the docs predicted. In such circumstances, for the time being at least, I can fall back on 'WhatIs' for help. The HAM samples I tried were just as they came on a selection of PD disks (my version of DPaint can't handle them), and not all HAM pics from that source behaved as expected. Though I frequently felt like giving up, I eventually got the following set of procedures to work more or less reliably :   An exploratory program 'GetData', ready to run.   One of each of the seven types of file, with the following titles: highres, lowres, lace, highresbrush, lowresbrush, lacebrush, ham   A short script file called 'GetSet':  rx GetData highres   rx GetData lowres   rx GetData lace   rx GetData highresbrush  rx GetData lowresbrush   rx GetData lacebrush   rx GetData ham    The CLI command : EXECUTE GetSet The following is an extract from the actual output from such a run, here arranged to suit this medium, but the values have been left exactly as they were produced. Size ScrH ScrW PicH PicW dpth mode 1 highres 7134 256 640 256 640 4 HIRES 2 lowres 9682 256 320 256 320 5 LORES 3 lace 12264 256 320 512 320 5 LACE 4 highresbrush 2790 256 640 74 170 3 HIRES 5 lowresbrush 3256 256 320 85 132 4 LORES 6 lacebrush 3482 256 320 89 80 5 LACE 7 ham 41456 256 320 256 320 6 HAM All as it should be Lines 1, 2, and 7: Full screen images, screen and pic dimensions are the same. Line 3: In Lace mode, PicHeight is twice the ScreenHeight. Lines 4, 5, and 6: Screen size and image size are different, which is how we can recognise them as brush images. Even all the modes displayed here are correct, something which at one stage I never thought I would see. I was going to include the 'GetSet' program here to document more fully this part of the discussion, but it contains little which is not also in the program discussed in the next section.  A sample application  Some sort of application was the next step, and the result was the attached program 'Iffy', the major components of which are listed here. The numbers below more or less match those included in the program itself, where you will also find a few further comments. Some of the finer points about running the program are also contained there, so it might be worth your while having a look at the program listing, even if you would not normally wish to plunge into that jungle.  1  Error trapping  2  Add required libraries to active list  3  If 'PicList' exists, load in its contents, and set 'auto' flag. If you want a list of pics to be processed, you can create a file called 'PicList' containing their names. Those names must include their path, and 'PicList' should be in the same directory as the program. For example : Ram:Tree Ram:Bush df1:pics/Branch df1:pics/animals/sloth  and so on. If such a list exists, a mouse click anywhere (or a menu selection) is all it takes to move on to the next picture. Without 'PicList', the next picture has to be selected each time from a file requester. If you are not going to be looking at the parameters which are the main topic of this article, but just want to use the program as an IFF viewer, then one additional line can be added to 'PicList' containing the number of seconds you want each pic to pause before the next one is screened. The presence of such a line turns the 'auto' flag on. For example, a final line of '2' (no quotes) will cause a pause of two seconds between the screening of each of the pictures (full-screen images or 'brushes'). You can just sit back and watch. I know that this can also be done by, for example, 'Display', but it's good to be able to produce your own version, knowing that you can modify it as you please, and perhaps borrow it and build on it as a background for other programs. Back to the program outline ...  4  Screen first pic as listed, or as selected from File Requester  5  Start a loop to handle user actions between pics  6  Bypass the wait for user action if 'auto' has been set, otherwise deal with : A mouse click - Get next picture ) from the 'PicList' set A menu choice - Get next picture ) or via a file requester Show pic parameters Show colour settings Quit  7  Finish : Close everything  8  'Syntax' and 'Error' errors dealt with Main function : Screen the pic  9  Get the name of the next file and its size  10  Check for expected headers  11  Pull out the data required  12  Determine the ViewMode This is the area in which I had the most difficulty, and the difficulty seemed to increase rather than diminish as I worked. There are contradictions in the docs and surprising inconsistencies in the file structure in some widely used graphics tools. Just one example : there are (should be) two pairs of width/height parameters, one pair for the screen size and another for the image size. For a full screen image these pairs will be the same. But one PD graphics tool I often use puts the image size in both data locations, making it impossible to rely on the presence of, say, 640 or 320 anywhere in those files as a quick indicator of screen resolution. Using 'WhatIs' in such cases, for example : Instead of  width : 70 height : 96   .   PageWid : 640 PageHei : 256  we get  width : 70 height : 96   .   PageWid : 70 PageHei : 96   13  Open a screen  14  Use CreateHost() for a messageport, window, etc.  15  Open the window and the port  16  Create a menu  17  At last, call the IFFImage function /* 17 */ Supporting functions  18  Check on presence of expected headers in file  19  Show screen width/height, image width/height (if different), and number of colours  20  Show RGB settings per colour. These are positioned at the right of the picture, and with overlarge pictures may be off-screen and therefore invisible. Adjust the value of 'sampos' in the program if you want to overcome this.  Final Notes  'Iffy' is a ready-to-run ARexx program, mainly menu-driven. It expects to find 'rexxarplib.library' in LIBS:. If the program is to be able to do anything, it must also have some IFF images within reach. It will handle LORES, HIRES, LACE and HAM types. You have the program. If you have Arexx, and if you use it, then : · Anything you don't like, change. · Anything you can add or improve, do it. · Anything I've got wrong, fix it if you can. · Any complaints or suggestions, write to me. I readily admit that this is a mere beginning, that there are problems and complexities lurking in IFF files which I have not mentioned, and even more of which I am not yet aware. It was just a learning exercise which produced one tool to do the intended trivial task and another one, as a by-product, which is rather more interesting. If it is at a level which is useful for you, I am pleased to share it. If you find it irritatingly simplistic or pathetically incorrect, either ignore it, or take up the challenge to write an improved alternative for the next issue of MegaDisc. Hamilton, N.Z. April, 1994 ------------------------------- :-) 40 :^) 40 :) 40 8) 40 8-) 40 B-) 40 :-O 40 8*)