ARexx in Desk Top Publishing: A Super Index Builder! Using TurboText and ARexx Together by Merrill Callaway Non-Programming Desk Top Publishers ReadME! Many thanks to all the people who responded to the April issue and my offer of the ADPro Scale Utility. It was so popular that I'm going to do it again, only this time, if you do DTP on the Amiga (with any package) and own ARexx and have TurboText (or are willing to get it -- highly recommended), then you'll want to get a copy of what I'm calling the ARexx DTP Index Builder (ADIB). It will save you literally hundreds of hours of drudge work (including the considerable typing of the code), but your Indices and ToC's will be guaranteed to warm the heart of Marian the Librarian. I invented these programs to help me finish The ARexx Cookbook sometime before the next century, and I can testify from experience that programming AND using these programs took much less time than doing it all by hand would have taken. Read on and skip the listings if they intimidate you, but if my programs' descriptions meet your needs, you may send me $15 to get an even better version (postage paid) with docs formatted for AmigaGuide on-line hyper-help capability (provided you have amigaguide and amigaguide.library -- not Freely Distributable just yet, but they frequently come with the newer commercial programs; e.g. Presentation Master by Oxxi). They are still readable if you do not have AmigaGuide on your system. Send $15 c/o the magazine or to Merrill Callaway, 511-A Girard SE, Albuquerque, NM 87106, if you are in a hurry. The Problem One of the most aggravating things about the DTP software for the Amiga is that there is no way that PageStream (or any other DTP packages) will generate an index for you or a Table of Contents. WordPerfect has a crude index builder, but how will you then format the pages and pictures? AmigaTex (with ARexx support) will do a great job setting your type, but it isn't, and doesn't claim to be, a page layout program. Also it isn't for the faint of heart as you must imbed format commands in the text. How to Find the End of Page? Think of PageStream as a row of tall glasses that you must fill sequentially from a very large pitcher of water. That's the way the "pages" are in PageStream: They are just forms which you fill with text. THERE ARE NO END OF PAGE MARKERS. I looked at a hex file of one of my documents. The text is simply poured into the forms and it is the forms that are numbered and NOT your text. Everyone said to give it up. Not only doesn't PageStream have ARexx, but even if it did, you could not implement an automatic ARexx index builder because of the above problem. Indeed, ProPage does have ARexx but you can't make an index automatically with it. None of my "Job's Comforters" thought of doing the obvious: make a SEMI-automatic system and do everything which is impossible to do with software manually. The "impossible core" information that must be done manually is simply to find the end of each page! (And mark them with some unique string -- I used "$$"). Every index must come from the mind of the author, because if you made a system that indexed every word, you'd have a thick concordance! Since making selections for the index is part of authorship, I don't count the time a user takes selecting his or her words and strings to index, but I made this chore as automatic as possible, using that paragon of editors, TurboText (TTX), which has a full ARexx interface. The Procedure First, you must finish the document with all revisions completed, at least to the point that no text will shift page numbers on you. Print out a draft to help you locate the End of Page (EOP). Next, if your book is like mine, you probably have one file for each chapter and your pages go 1-1, 1-2 for Chapter One page 1, Chapter One page 2, etc. You therefore need to EXPORT ASCII text from PageStream by selecting all text and saving it as ASCII to files with the names C.1 for Chapter One, C.2 for Chapter Two, etc. If you have multiple, non-linked columns and so forth, you will need to export the complete chapter in sequential pieces or re- compose the pieces in TTX. The goal is to have an ASCII file where the text in the chapter is all there in sequence. Format doesn't matter at all. We are only going to use these files for indexing. They will NEVER be imported back into your document. Step One: Make the Working Files After we make C.1, C.2, ... C.n and save them, we are ready to mark the ends of all pages. I made a simple key-stroke recorded macro in TTX to type in $$ at my cursor. Then, using my draft copy of The ARexx Cookbook, and the Find command in TTX, I marked the end of every page with $$ and also a $$ at the very end of each Chapter file. Even a book with 250 pages was easily done in just a few hours. The first program, run from a Shell, is INDEX-1-Chapter.rexx (the naming of the programs reflects the sequence in which you use them, what they do, and whether they are an ARexx program run from a Shell ".rexx" or a TTX Macro ".ttx"). This one extracts from each Chapter file many smaller files, one for EACH PAGE. These are named P1.10, P3.5, etc. for Chapter One Page 10, Chapter Three Page 5, etc. The program generates them sequentially, however. The program will let you start with any (existing) Chapter and it steps through all the Chapter files until it's done or you quit. This program also allows you to set the default path for your INDEX document files and it saves it in ENVARC: for you. Every time this first program is run, you have the opportunity to edit your default path or use the stored one. You DON'T NEED to edit the code (!) Also, the data for each chapter (its number and how many pages it has) is stored in a file "defaultpath/ChPg" for you to use later if you like. The program lets you edit this file as well. I had an Appendix numbered A-1, A-2 etc. and I changed the last entry from "Chapter Nine" in sequence with 36 pages, "9.36", to "A.36" to make the page numbers read properly when attached to the index entries. The ChPg file maybe used or you may enter these data manually for testing purposes or partial runs. Step Two: Select the Index Strings Now we load the Chapter files, "C.n" where n=Chapter number into TurboText in order to grab our Index or Table of Contents strings. You need not worry about redundancy or alphabetic order, because these are taken care of automatically! You also don't need to worry about page numbers or even which chapter you're in -- just that you eventually cover all chapters. The first program was an ARexx program run from the Shell, but this one is a TTX Macro. Note that all the listing have the relevant facts in comments so you can see what's what. The TTX Macro INDEX-2-Words.ttx (note the typography -- ttx = TurboText) should be connected to your Alt-I key in the TurboText TTX_Startup.dfn file so that you can just hit Alt-I to grab a word. When you hit Alt-I, any word under the cursor the macro will grab and put into a temporary file (RAM:out). If we need an entire string, such as for a ToC, then we select it first and then hit Alt-I. The text is not affected, as the macro only copies the word or block. Step Three: Combine the Output with the "Master Index List" Next, at the end of the session, we hit another macro key, Alt-C for "Combine" to append the words and strings in the temporary file RAM:out to our storage file for the whole index, the "Master Index List" (MIL), named "INDEX.ascii". This Macro, INDEX-3-Combine.ttx lets you choose where you want the (MIL) "INDEX.ascii" stored and what it is to be called. These "bells and whistles" I added for you, as my originals were "quick and dirty". The MIL is the heart of the series. It gets updated at every turn, either to have strings added to it, or to be sorted alphabetically, or finally to have the page numbers attached at the very end. Even the page numbers are attached sequentially with the MIL being updated many times over. This is the only way to get the page numbers the right way around in the final listing. In the meantime, INDEX.ascii may be edited by hand or you may remove duplicates with the next program, also a TTX Macro. Step Four: Remove Duplicates & Sort This macro, launched by Alt-L, INDEX-4-UniLine.ttx is for removing duplicate lines and for sorting a document's lines in alphabetic. This improved version lets you choose whether to ignore case or not when removing duplicates, and whether to ignore case or not when doing sorts. There are two requesters that pop up to ask you, so there are four possibilities. The way I did The ARexx Cookbook was to ignore case in both cases but if you need your document (perhaps about case-sensitive programming commands) to account for case, it's all there. We can run this program on RAM:out or INDEX.ascii at any time. We run it one last time before attaching page number information at the last step. Step Five: Re-format Page Files for String Searches This is a BATCH ARexx program run from a Shell. INDEX-5-Documentize.rexx reformats all those PAGE files P1.1 through P#.# in the following way: It turns every paragraph into ONE LONG LINE! Why? The program that matches up the strings and words we have selected will not match multiple word strings if they wrap from one line to the next! We need to unravel them into one line per paragraph so that these ToC and other longer strings won't get ignored. The program calls a modification of one of the macros that comes with TTX. I changed the macro to allow it to be called as a function from within another ARexx program. Step Six: Finish & Attach Page Number Information The final program INDEX-6-Finish.rexx is also an ARexx program run from a shell. BE SURE to have INDEX.ascii in the FINAL form you want before doing this step. It's a very good idea to make a backup of INDEX.ascii before using. This program is where you save all the time. Go have lunch while it's running, because it compares EVERY string in your MIL (INDEX.ascii) with EVERY PAGE in the document, performing a FIND operation for each; and it'll do it MUCH faster than you could have, and it won't miss anything. I needed a multiple index for The ARexx Cookbook covering the Hawes documentation AND the Commodore documentation AND the manuals for all the software examples (e.g. for TurboText) used in the book. The index has over 600 items. You may really find anything you need there; I'd never have had the patience to do all that by hand! The Other Listings: Functions & Utilities The rest of the listings have comments as to what they are and do. They are either external functions, or an auxiliary program IDXuniword.ttx used to extract a list of EVERY word from a document. I found it useful for some of my program listings. By the way, The ARexx Cookbook is available for $39.90 postage paid from me. You can learn ARexx and see firsthand how powerful ARexx can be in DTP!