@DATABASE "ARB35" @INDEX "ARB:Misc/Index.Text/Main" @HELP "ARB:Misc/Help/Main" @NODE "Main" "ARexx For Beginners - Article 35 - Parsing & Templates" @{B}@{U}@{JCENTER}AREXX FOR BEGINNERS ARTICLE 35 - PARSING AND TEMPLATES - PART 1 BY FRANK BUNTON@{UB}@{UU} @{"COPYRIGHT © FRANK P. BUNTON 1995-1998" LINK "ARB:Misc/Read_Me_First!!/Copyright"} @{" Please Read Me First " LINK "First"} @{" An Illustration of Parsing With a Template " LINK "Illustration"} @{" Types Of Parsing " LINK "Types"} @{" The PARSE Instruction " LINK "Parse"} @{" The VAR Input Source " LINK "Var"} @{" The Template " LINK "Template"} @{" Pattern Parsing " LINK "ParsePatterns"} @{" Scanning String When Parsing By Patterns " LINK "Patterns"} @{" The Pattern Markers " LINK "PatternMarkers"} @{" Tokenisation Parsing " LINK "Tokenisation"} @{" The Place Holder " LINK "PlaceHolder"} @{" Differences in Parsing By Patterns & Tokenisation " LINK "Differences1"} @{" Differences in Pattern & Positional Marker " LINK "Differences2"} @{" Position Parsing " LINK "ParsePosition"} @{" Scanning Input When Parsing By Position " LINK "Position"} @{" The Absolute Markers " LINK "Absolute"} @{" The Relative Markers " LINK "Relative"} @{" Mixing Various Types Of Parsing " LINK "Mixing"} @{" Multiple Templates " LINK "Multiple"} @{" Input String NOT Changed By Parsing " LINK "InputString"} @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "First" "Article 35 - Parsing & Templates - Please Read Me First" @{B}@{U}@{JCENTER}PARSING AND TEMPLATES - READ ME FIRST@{UB}@{UU} @{JLEFT} In my experience, "Parsing and Templates" is a topic that is very difficult to explain and/or learn as it is hard to understand parsing without knowing what templates are, and it is hard to understand templates until you know how to parse!! So what I intend to do is to jump around a bit and, while this may seem to be bit rambling to anyone who already knows a bit about the subject, it was the way that I, myself, found was the best way to learn the topic. I am hoping, therefore, that it will also prove the best way for my readers who have not yet come to grips with it. So please bear with me. @{B}So, for the beginner@{UB}, read through this article in the order of the headings below. @{B}For the experienced user@{UB}, you may like to skip this article and go straight to @{"Part 2" LINK "ARB:Articles_31-40/36.Parse-Templates-2/MAIN"} or pick out what you want from the index "Part 2 - Instructions & Functions" @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Illustration" "Article 35 - Parsing & Templates - Illustration Of Parsing With Template" @{B}@{U}@{JCENTER}ILLUSTRATION OF PARSING WITH TEMPLATE@{UB}@{UU} @{JLEFT} Before getting into an involved discussion of all the correct format of parsing and templates, let's look at a little example which is not really syntactically correct but may help you to get the hang of what it is all about. @{B}The Input String@{UB} First of all, let's assume that we have an @{B}input string@{UB} which reads as follows:- 'Joe Bloggs:99 Somewhere Street:123 4567' We call it an @{B}input@{UB} string as it is from this string that we will be able to take values for assignment to various symbols that are contained in a @{B}template.@{UB} As we will see later, this input string can come from a number of different sources which are called the @{B}input source.@{UB} @{B}The Template@{UB} Now let's assume that we have a @{B}template@{UB} that looks like this:- Name':'Address':'Phone It is called a @{B}template@{UB} as it is a sort of @{B}pattern@{UB} with which we can tell ARexx which parts of the input string we want to assign as values to the various symbols. This example template is made up of three symbols (@{B}not@{UB} in quotes):- Name Address Phone which are separated by strings (in quotes) each consisting of a colon (:). Thus the @{B}template@{UB} is made up of @{B}targets@{UB} and @{B}markers@{UB}. @{B}The Targets@{UB} The symbols are @{B}targets.@{UB} A target, in everyday life, is something at which you can aim. In parsing, the target is a symbol to which you are going to assign a value. @{B}The Markers@{UB} Between each target is a @{B}marker@{UB} which, in this case, is the string ':'. It must be in quotes to make sure that ARexx sees it as a string. It is called a marker as it tells ARexx which characters in the input string are to be used as the "mark" at which to start or end symbol values. @{B}Matching And Assigning@{UB} The parsing process will match up the marker characters in the template with those same characters in the input string as follows:- 'Joe Bloggs:99 Somewhere Street:123 4567' Name ':' Address ':' Phone In this illustration I have placed the markers directly below the same characters in the input string so that you can easily see the parts of the input string that will be assigned to each of the symbols beneath. These assignments of symbol values will therefore be made as follows:- Name = Joe Bloggs Address = 99 Somewhere Street Phone = 123 4567 @{B}It is important@{UB} to note that the @{B}markers@{UB} in the input string do @{B}NOT@{UB} become part of the values assigned to the target symbols. @{B}Scanning@{UB} Parsing is said to be done by a @{B}scanning@{UB} process. ARexx scans along the input string looking for markers that tell it what parts of the input string to assign to the symbols in the template. @{B}In Summary@{UB} To summarise, @{B}Parsing@{UB} is an operation that takes an @{B}input string@{UB} from a nominated @{B}input source@{UB} and assigns part or parts of the input string to some or all of the symbols contained in the @{B}template@{UB} according to the way the template is set up. As we will see later, markers of the kind used above are only one of a number of marker types. We will also see that there can be eight different types of input source and that there are three different types of parsing. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Types" "Article 35 - Parsing & Templates - Types Of Parsing" @{B}@{U}@{JCENTER}TYPES OF PARSING@{UB}@{UU} @{JLEFT} The above @{"illustration" LINK "Illustration"} represents one of three types of parsing, i.e. parsing by pattern. The three types are:- - @{" Parsing by Pattern " LINK "ParsePatterns"} - using pattern markers - @{" Parsing by Tokenisation " LINK "Tokenisation"} - using only a space between targets - @{" Parsing by Position " LINK "ParsePosition"} - using absolute and relative markers However, before going into these, we should look at the @{"PARSE" LINK "Parse"} instruction itself. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Parse" "Article 35 - Parsing & Templates - The PARSE Instruction" @{B}@{U}@{JCENTER}THE PARSE INSTRUCTION@{UB}@{UU} @{JLEFT} The format for PARSE is as follows:- PARSE [UPPER] InputSource [Template] [,Template] [,........] Where:- PARSE - is the instruction UPPER - is an @{B}optional@{UB} keyword that, if used, causes the input string to be converted to upper case. InputSource - is one of 8 option keywords that tell PARSE where the input string is coming from. These option keywords, are:- @{" ARG " LINK "ARB:Articles_31-40/36.Parse-Templates-2/Arg"} Discussed in @{"Articles 36." LINK "ARB:Articles_31-40/36.Parse-Templates-2/Main"} @{" EXTERNAL " LINK "ARB:Articles_31-40/36.Parse-Templates-2/External"} Discussed in Articles 36. @{" NUMERIC " LINK "ARB:Articles_31-40/36.Parse-Templates-2/Numeric"} Discussed in Articles 36. @{" PULL " LINK "ARB:Articles_31-40/36.Parse-Templates-2/Pull"} Discussed in Articles 36. @{" SOURCE " LINK "ARB:Articles_31-40/36.Parse-Templates-2/Source"} Discussed in Articles 36. @{" VALUE " LINK "ARB:Articles_31-40/36.Parse-Templates-2/Value"} Discussed in Articles 36. @{" VAR " LINK "Var"} Discussed in this article. @{" VERSION " LINK "ARB:Articles_31-40/36.Parse-Templates-2/Version"} Discussed in Articles 36. Template - is a symbol, or a group of symbols, plus other tokens the total of which indicates to ARexx the way in which the input string is to be assigned as values to the target symbol or symbols. - More than one template can be included in the one PARSE instruction clause provided that the templates are separated by commas. This is discussed under the heading "Multiple Templates" below. I would suggest that beginners follow through the current article (35) by going through its @{"contents" LINK "MAIN"10} in the order shown. This means that you should go to @{"VAR" LINK "Var"} next and finish this article before going on to the other options in Article 36 @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Var" "Article 35 - Parsing & Templates - The VAR Input Source" @{B}@{U}@{JCENTER}THE VAR INPUT SOURCE@{UB}@{UU} @{JLEFT} I will discuss all the @{B}InputSource@{UB} options in the @{"next article" LINK "ARB:Articles_31-40/36.Parse-Templates-2/MAIN"}. But to get us started in understanding templates, I will need to have one simple usage of PARSE for examples. This usage of PARSE is as follows:- PARSE [UPPER] VAR SymbolName [Template] [,Template] [,........] The keyword @{B}VAR@{UB} tells ARexx that the token immediately following it, i.e. the symbol name, is the input source. The value of the symbol is the input string. Let's put this into an example using the @{"input string" LINK "Illustration"8} and @{"template" LINK "Illustration"20} that we had at the start of this article:- /* Example35-1 */ 1. TestSymbol = 'Joe Bloggs:99 Somewhere Street:123 4567' 2. PARSE VAR TestSymbol Name ':' Address ':' Phone 3. SAY 'Name =' Name 4. SAY 'Address =' Address 5. SAY 'Phone =' Phone If you RX this example you will see:- Name = Joe Bloggs Address = 99 Somewhere Street Phone = 123 4567 Line 1 simply sets a value for the symbol "TestSymbol" Line 2 is the PARSE instruction clause that consists of these parts:- PARSE - The instruction VAR - The instructions's keyword that tells ARexx that the input source is a symbol value TestSymbol - the symbol whose value is the input string Name':'Address':'Phone - the template with two markers, each one being a colon, and three symbol names as the targets. The target symbols are assigned their values in Line 2 by the PARSE instruction which scans along the input string looking for the markers. Lines 3 to 5 display the newly assigned values of the three symbols. I will now use this "VAR Input Source" parsing method in the examples for templates that I use below. After we have come to understand templates we can then go on to look more closely at how to use the various input sources. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Template" "Article 35 - Parsing & Templates - The Template" @{B}@{U}@{JCENTER}THE TEMPLATE@{UB}@{UU} @{JLEFT} As indicated @{"earlier" LINK "Illustration"39}, a template is made up of @{B}markers@{UB} and @{B}targets.@{UB} @{B}Template Targets@{UB} Apart from one exception (see next paragraph), the targets in a template are always variable symbols. As the whole purpose of parsing is to assign values to the targets, they could not be anything else! The exception is the "." character (period, full stop or whatever you wish to call it). We will discuss this later under the heading @{"The Place Holder" LINK "PlaceHolder"}. @{B}Template Markers@{UB} The markers used in @{"Example35-1" LINK "Var"15} were of a type called "Pattern Markers". The full list of marker types is:- - @{" Pattern Markers " LINK "PatternMarkers"} - @{" Positional Markers - Absolute " LINK "Absolute"} - @{" Positional Markers - Relative " LINK "Relative"} @{"Tokenisation" LINK "Tokenisation"} is not included here as it does not use markers. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "ParsePatterns" "Article 35 - Parsing & Templates - Parsing By Patterns" @{B}@{U}@{JCENTER}PARSING BY PATTERNS@{UB}@{UU} @{JLEFT} When we parse by @{B}patterns@{UB} we use @{"markers" LINK "Illustration"47} in both the @{"template" LINK "Illustration"20} and the @{"input string" LINK "Illustration"8}. @{B}Pattern parsing@{UB} uses matches between the @{B}pattern@{UB} in the template and the input string to decide which parts of the input string to @{"assign as values" LINK "Illustration"55} to the @{"target symbols" LINK "Illustration"41}. In studying pattern parsing we will look at these topics:- @{" Scanning String When Parsing By Patterns " LINK "Patterns"} @{" The Pattern Markers " LINK "PatternMarkers"} @{" Differences in Parsing By Patterns & Tokenisation " LINK "Differences1"} @{" Differences in Pattern & Positional Marker " LINK "Differences2"} @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Patterns" "Article 35 - Parsing & Templates - Scanning String When Parsing By Patterns" @{B}@{U}@{JCENTER}SCANNING INPUT STRING WHEN PARSING BY PATTERNS@{UB}@{UU} @{JLEFT} When parsing by patterns, a temporary string is set up for the use of the parsing process. This @{B}temporary@{UB} string has the pattern characters @{B}removed@{UB} from it but the @{B}input@{UB} string remains @{"unaltered" LINK "InputString"}. This is illustrated by @{"Example 35-16" LINK "Mixing"15} and @{"Example35-17" LINK "Multiple"}. Thus, in our example above:- Joe Bloggs:99 Somewhere Street:123 4567 Name ':' Address ':' Phone the temporary string becomes:- Joe Bloggs99 Somewhere Street123 4567 and this temporary string is marked at the spots indicated by asterisks:- Joe Bloggs99 Somewhere Street123 4567 * * * * Each target symbol is then allocated a value as follows:- Name - from 1st mark to 1 before 2nd mark = Joe Bloggs Address - from 2nd mark to 1 before 3rd mark = 99 Somewhere Street Phone - from 3rd mark to 1 before 4th mark = 123 4567 When we come to look at absolute positional markers, we will see that this is very similar. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "PatternMarkers" "Article 35 - Parsing & Templates - Parsing By Pattern Markers" @{B}@{U}@{JCENTER}THE PATTERN MARKERS@{UB}@{UU} @{JLEFT} A @{B}Pattern Marker@{UB} is a string that represents a particular character, or group of characters. The colon used in @{"Example35-1" LINK "Var"15} is an example of a pattern marker. Example35-1 had the same pattern for each marker but this need not be the case. It can be a different pattern for each marker. Example35-2 below uses two markers that are more than one character each and are different to each other:- /* Example35-2 */ TestSymbol = 'Joe Bloggs:99 Somewhere Street:123 4567' PARSE VAR TestSymbol Name '99' Address '123' Phone SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone If you RX this you will see:- Name = Joe Bloggs: Address = Somewhere Street: Phone = 4567 The two patterns used are:- '99' and '123' Three points to note here are:- - The two patterns ':' used in Example35-1 have now become part of the temporary string and are included as part of the values read into the symbols "Name" and "Address". - The values of "Address" and "Phone" both have a leading space as the input string has a space immediately after each of the patterns. - Our new patterns "99" and "123" have been removed from the temporary string and do not appear anywhere in the values assigned to the three target symbols. When creating the input string, pattern characters are only removed for the @{B}first@{UB} occurrence of that pattern:- - from the start of the string for the first marker - from the previous marker onwards for subsequent markers. If the same pattern of character(s) is found again in the input string but it is @{B}not@{UB} part of the @{B}next@{UB} marker then it is no longer considered as a pattern but is taken to be a normal part of the input string. This is illustrated by the following example:- /* Example35-3 */ TestSymbol = 'Joe Bloggs:99 Somewhere Street:123 4567' PARSE VAR TestSymbol Name 'o' Address 't' Phone SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone The output from this program is:- Name = J Address = e Bloggs:99 Somewhere S Phone = reet:123 4567 The temporary string has been set up as follows:- Joe Bloggs:99 Somewhere Street:123 4567 Name 'o' Address 't' Phone --> Je Bloggs:99 Somewhere Sreet:123 4567 * * * The first pattern "o" tells ARexx to assign everything up to, but not including, the first "o" to the symbol "Name" which therefore becomes "J". The next pattern "t" says to start at the next character @{B}after@{UB} the pattern "o" and assign everything up to, but not including, the next pattern "t" to the symbol "Address" which therefore becomes "e Bloggs:99 Somewhere S". Note that this string @{B}does@{UB} include two other "o"'s. Finally, the last symbol, "Phone" is assigned everything after the above mentioned "t" which is "reet:123 4567". Note that this contains another "t". So far we have used actual strings as the patterns, However, the string value held by a variable symbol can be used as a pattern provided that the symbol's name is enclosed in parentheses (). This is shown by this example:- /* Example35-4 */ P = ':' TestSymbol = 'Joe Bloggs:99 Somewhere Street:123 4567' PARSE VAR TestSymbol Name (P) Address (P) Phone SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone This is almost identical to @{"Example35-1" LINK "Var"15} as the symbol "P" holds the value ":". The output is identical to Example35-1:- Name = Joe Bloggs Address = 99 Somewhere Street Phone = 123 4567 To duplicate @{"Example35-2" LINK "PatternMarkers"} we would use:- /* Example35-5 */ P1 = '99' P2 = '123' TestSymbol = 'Joe Bloggs:99 Somewhere Street:123 4567' PARSE VAR TestSymbol Name (P1) Address (P2) Phone SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone and this would give the same output as Example35-2, i.e.:- Name = Joe Bloggs: Address = Somewhere Street: Phone = 4567 @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Tokenisation" "Article 35 - Parsing & Templates - Parsing By Tokenisation" @{B}@{U}@{JCENTER}PARSING BY TOKENISATION@{UB}@{UU} @{JLEFT} A @{B}token@{UB} is the smallest individual unit of a clause. When within a string, this is a @{B}word.@{UB} A word in a string is considered to be any group of characters that does not have a space within it. Words are therefore separated from other words by spaces. As an example, the string:- Joe Bloggs 99 Somewhere Street 123 4567 has 7 "words" in it:- Joe Bloggs 99 Somewhere Street 123 4567 "Parsing By Tokenisation" uses a template in which there are @{B}no markers@{UB} as such. The target symbols are separated by spaces. For example:- PARSE VAR TestSymbol One Two Three Four Five Six Seven . (I will discuss the period (.) after the last target symbol "Seven" under the heading @{"The Place Holder" LINK "PlaceHolder"}. For now just take it that it is necessary.) Each of the seven target symbols "One" to "Seven" are separated by spaces. Although some may consider that these spaces could be compared to markers containing a single space character, this is not strictly correct. Examples @{"35-10 and 35-11" LINK "Differences1"} demonstrate why this is not so. ARexx looks at the template, sees that each target symbol is separated by a space and so assigns each word (which, as we said, are separated by spaces) in the input string to a different target symbol. This can be illustrated by this example:- /* Example35-6 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol One Two Three Four Five Six Seven . SAY 'One =' One SAY 'Two =' Two SAY 'Three =' Three SAY 'Four =' Four SAY 'Five =' Five SAY 'Six =' Six SAY 'Seven =' Seven The output of this program is:- One = Joe Two = Bloggs Three = 99 Four = Somewhere Five = Street Six = 123 Seven = 4567 As you can see, each word in the input string has been assigned to a target symbol in the order in which they appear. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "PlaceHolder" "Article 35 - Parsing & Templates - The Place Holder" @{B}@{U}@{JCENTER}THE PLACE HOLDER@{UB}@{UU} @{JLEFT} In order to help us understand what a @{B}Place Holder@{UB} is, let's first rewrite @{"Example35-6" LINK "Tokenisation"40} without the last two target symbols and without the final period (.):- /* Example35-7 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol One Two Three Four Five SAY 'One =' One SAY 'Two =' Two SAY 'Three =' Three SAY 'Four =' Four SAY 'Five =' Five this will give this output:- One = Joe Two = Bloggs Three = 99 Four = Somewhere Five = Street 123 4567 Note that the last target symbol "Five" has been given a value equal to:- " Street 123 4567" which has a leading space. When using parsing by tokenisation, the @{B}last@{UB} target will always pick up the @{B}space before@{UB} the next word as well as @{B}all of the rest@{UB} of the input string!! The @{B}Place Holder,@{UB} which is the period ".", is a special type of target which "takes the place of" a normal target. By putting a place holder at the @{B}end@{UB} of a template, it ensures that the last @{B}normal@{UB} target will @{B}not@{UB} be assigned a leading space and will be assigned @{B}only one word@{UB} if there is more than one word left in the input string. This is because the last normal target is no longer the last actual target. This is illustrated by the following program. The only difference to Example35-7 is the inclusion of the place holder at the end of the PARSE clause. /* Example35-8 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol One Two Three Four Five . SAY 'One =' One SAY 'Two =' Two SAY 'Three =' Three SAY 'Four =' Four SAY 'Five =' Five The output will now be:- One = Joe Two = Bloggs Three = 99 Four = Somewhere Five = Street The place holder can also be used anywhere within the template to stop a word being assigned to a target symbol. For example, if we add the target "Seven" to Example35-8 we get:- /* Example35-9 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol One Two Three Four Five . Seven . SAY 'One =' One SAY 'Two =' Two SAY 'Three =' Three SAY 'Four =' Four SAY 'Five =' Five SAY 'Seven =' Seven and the output is:- One = Joe Two = Bloggs Three = 99 Four = Somewhere Five = Street Seven = 4567 As you can see, the word "123" has not been assigned to any of the target symbols. By putting a place holder in its spot, it is, in effect, bypassed. You can put as many place holders in whatever positions in the template that you like but it is wise to @{B}always put a place holder at the end of the template!@{UB} @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Differences1" "Article 35 - Parsing & Templates - Differences Parsing By Patterns & Tokenisation" @{B}@{U}@{JCENTER}DIFFERENCES BETWEEN PARSING BY PATTERNS AND BY TOKENISATION@{UB}@{UU} @{JLEFT} The following two examples (35-10 and 35-11) will illustrate the difference between:- - @{B}Parsing by Pattern@{UB} using a pattern marker which is a space characters @{B}within quotes.@{UB} - @{B}Parsing by Tokenisation@{UB} using a space (@{B}not@{UB} in quotes) between each target symbol. To do this we need to be able to go back to the @{B}start@{UB} of the temporary string and reuse it within the @{B}same@{UB} template. This will illustrate that the two methods of parsing @{B}are different@{UB} even though they may appear very similar. To do this, we need to put an extra bit onto the end of each of the templates:- =1 Eight The "=1" is @{"Parsing By Positional Markers" LINK "ParsePosition"} which we will cover later. Briefly, what it does is send the @{B}position index@{UB} back to the start of the temporary string being parsed or, in other words, puts the marker at the 1st character of the string. The "Eight" target following the "=1" is then allocated everything from the @{B}start@{UB} of the temporary string to the @{B}end@{UB} of the temporary string as there is no marker after the "Eight". Actually, using the "=1" in the "Parsing by Pattern" is usually a big @{B}NO-NO@{UB} (See @{"Mixing Various Types of Parsing" LINK "Mixing"}) but it serves to illustrate the point I am about to make. Example35-10 is @{B}Parsing By Pattern@{UB} as the spaces between each target symbol are enclosed in quotes and thus becomes strings. /* Example35-10 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol One' 'Two' 'Three' 'Four' 'Five' 'Six' 'Seven =1 Eight SAY 'One =' One SAY 'Two =' Two SAY 'Three =' Three SAY 'Four =' Four SAY 'Five =' Five SAY 'Six =' Six SAY 'Seven =' Seven SAY 'Eight =' Eight Example35-11 is @{B}Parsing by Tokenisation@{UB} as it has a space between each of the target symbols "One" to "Seven". /* Example35-11 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol One Two Three Four Five Six Seven . =1 Eight SAY 'One =' One SAY 'Two =' Two SAY 'Three =' Three SAY 'Four =' Four SAY 'Five =' Five SAY 'Six =' Six SAY 'Seven =' Seven SAY 'Eight =' Eight The two examples will output the same as @{"Example35-6" LINK "Tokenisation"40} but with these extra lines:- Example35-10 - Parsing by Pattern Matching:- Eight = JoeBloggs99SomewhereStreet1234567 This illustrates that, when the string is reread, the spaces are @{B}not@{UB} included in the temporary string and so have not been included in the value assigned to "Eight". Example35-11 - Parsing by Tokenisation:- Eight = Joe Bloggs 99 Somewhere Street 123 4567 This shows that, when the string is reread, the spaces are still there and have been included in the value assigned to "Eight". So, @{B}parsing by pattern@{UB} has resulted in all the patterns (one space character in this case) being @{B}removed@{UB} from the temporary string whereas @{B}tokenisation@{UB} has left the spaces in the temporary string. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "ParsePosition" "Article 35 - Parsing & Templates - Parsing By Position" @{B}@{U}@{JCENTER}PARSING BY POSITION@{UB}@{UU} @{JLEFT} When we parse by @{B}position@{UB} we use @{"markers" LINK "Illustration"47} in the @{"template" LINK "Illustration"20} that indicate a numeric position within the @{"input string" LINK "Illustration"8}. Values are @{"assigned" LINK "Illustration"55} to the @{"target symbols" LINK "Illustration"41} by taking the characters in the input string that are between the two positions marked by the values of the markers to the left and right of the target symbol. In studying position parsing we will look at these topics:- @{" Scanning Input When Parsing By Position " LINK "Position"} @{" The Absolute Markers " LINK "Absolute"} @{" The Relative Markers " LINK "Relative"} @{" Differences in Pattern & Positional Marker " LINK "Differences2"} @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Position" "Article 35 - Parsing & Templates - Scanning Input String When Parsing By Position" @{B}@{U}@{JCENTER}SCANNING INPUT STRING WHEN PARSING BY POSITION@{UB}@{UU} @{JLEFT} Before starting on the two positional type of markers (@{"absolute" LINK "Absolute"} and @{"relative" LINK "Relative"}), which use index positions within the input strings, we need to look at the way ARexx scans an input string while it is assigning values to the target symbols. The scanning process maintains a "position" or "index" within the input string. This index can range in value:- @{B}from@{UB} 1 (the first character of the input string) @{B}to@{UB} length of string + 1 (1 beyond the last character) Thus, in our example string, the index can be at any position from 1 to 40 inclusive:- Joe Bloggs:99 Somewhere Street:123 4567 1234567891111111111222222222233333333334 0123456789012345678901234567890 (I hope you can read off the numbers from 1 to 40 when they are set out this way!) Before any markers are read from the template, the scan index is at position number 1. Absolute and relative markers will shift the index position around within these numbered positions as we will see below. Assignments to a target symbol will always:- - @{B}start@{UB} at the character @{B}at@{UB} the index position set by the marker on the @{B}left side@{UB} of the target symbol, and - @{B}end@{UB} at the character @{B}one before@{UB} the index position set by the marker on the @{B}right side@{UB} of the target symbol. Thus, if a target symbol has positional markers valued at 12 to its left and 32 to its right, it will be allocated characters from 12 to 31. If the @{B}first@{UB} target has @{B}no marker@{UB} on its @{B}left side@{UB} then ARexx assumes a marker equal to the 1st position. If the @{B}last@{UB} target has no marker on its @{B}right side@{UB} then ARexx assumes a marker equal to @{B}one past@{UB} the last character. Thus in our input string, if the first target has markers at positions 1 and 12 each side of it, and if we represent the marker positions with asterisks, it will look like this:- Joe Bloggs:99 Somewhere Street:123 4567 1234567891111111111222222222233333333334 0123456789012345678901234567890 * Name * This will result in the assignment of characters from number 1 to number 11:- Name = Joe Bloggs: If the next target "Address" has markers at positions 12 and 32 on each side of it:- Joe Bloggs:99 Somewhere Street:123 4567 1234567891111111111222222222233333333334 0123456789012345678901234567890 * Address * it will be allocated characters from number 12 to number 31:- Address = 99 Somewhere Street: and if the final target has markers at positions 32 and 40 on each side of it:- Joe Bloggs:99 Somewhere Street:123 4567 1234567891111111111222222222233333333334 0123456789012345678901234567890 * Phone * it will be allocated characters from number 32 to 39:- Phone = 123 4567 @{B}NOTE@{UB} that the original pattern marker characters (:) that we used previously when talking about pattern markers have now been picked up as part of the assignments made to the target symbols. Therefore, when using absolute and relative markers, it would be best if our input strings did @{B}not@{UB} include characters that are only marker characters. Our input string could therefore look like this:- Joe Bloggs 99 Somewhere Street 123 4567 @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Differences2" "Article 35 - Parsing & Templates - Difference Pattern & Positional Marker" @{B}@{U}@{JCENTER}IMPORTANT DIFFERENCE BETWEEN PATTERN AND POSITIONAL MARKER@{UB}@{UU} @{JLEFT} We have seen that, when using pattern markers, the characters in the pattern marker are @{B}not@{UB} assigned to any of the target symbols. In fact, as mentioned @{"earlier" LINK "Patterns"}, the pattern characters are @{B}removed@{UB} from the temporary string that is created (but @{B}not@{UB} from the original input string). However, with absolute and relative markers, @{B}all@{UB} characters in the input string can be assigned to the various target symbols. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Absolute" "Article 35 - Parsing & Templates - Parsing With Positional Absolute Markers" @{B}@{U}@{JCENTER}ABSOLUTE MARKERS@{UB}@{UU} @{JLEFT} @{B}Absolute Markers@{UB} are ones which indicate an absolute position within the input string, e.g. "12th character". They must therefore always be @{B}positive integer@{UB} numbers or be capable of being evaluated as positive integer numbers. The numeric value follows an equal sign (=) to differentiate them from @{"relative markers" LINK "Relative"} which follow either a plus sign (+) or a minus sign (-). Absolute markers are illustrated by:- /* Example35-12 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol =1 Name =12 Address =32 Phone =40 SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone This will give the same output as we have had before:- Name = Joe Bloggs Address = 99 Somewhere Street Phone = 123 4567 What has happened here is that ARexx looks at the markers on each side of the first target which are "=1" and "=12" and allocates to the target "Name" the characters from number 1 to number 11 (see @{"Scanning By Position" LINK "Position"}). If we measured the length of the string held by "Name" we would find that it holds 11 characters, not the 10 represented by "Joe Bloggs". The space at character number 11 (the space between "Bloggs" and "99" is included at the end of the string. For the next target "Address" ARexx looks at the two markers on its left and right which are "=12" and "=32" and so it allocates to "Address" all the characters from number 12 to number 31. Again, a space is at the 31st character which is tacked onto the end of the string assigned to "Address". The last target "Phone" has the markers "=32" and "=40" on its left and right and so ARexx assigns to it all the characters from number 32 to 39 which is the end of the input string. In reality, when we want to start from number 1 we can omit the first marker and when we want the last target to use all of the rest of the input string we can omit the last maker. Thus the template in our PARSE clause:- PARSE VAR TestSymbol =1 Name =12 Address =32 Phone =40 could be rewritten as:- PARSE VAR TestSymbol Name =12 Address =32 Phone As mentioned above, this is because, if there is no marker to the left of the first target, a marker of 1 is assumed, and if there is no marker after the last target, a marker of "Length of string + 1" is assumed. Note that, this time, we do @{B}not@{UB} use the colons in the input string. If we had left them in the output would have been:- Name = Joe Bloggs: Address = 99 Somewhere Street: Phone = 123 4567 With absolute targets, the assignment to a symbol takes into account the markers on @{B}both sides@{UB} of the target symbol. It starts assigning from the character represented by the value of the marker on the left side and stops assigning at the character one @{B}less than@{UB} the marker on the right side. If you wanted to make sure that the spaces at the 11th and 31st characters were @{B}not@{UB} included in the assignments then you could use:- PARSE VAR TestSymbol Name =11 =12 Address =31 =32 Phone If we used this parsing line, there is no target between the =11 and =12, and so the 11th character (a space) is not used. Similarly, the space at the 31st character is not used. Note that, when the marker is a fixed symbol, i.e. a number, as is the case in @{"Example35-12" LINK "Absolute"}, then the equals signs can be omitted and you can use, for example,:- PARSE VAR TestSymbol Name 12 Address 32 Phone I mention this as you will often see this sort of usage in programs. However, I prefer to use the equal sign as it makes it more obvious what is going on and maintains consistency of use compared to when you use a symbol as an absolute marker (see below). @{B}Variable symbols@{UB} that hold values equal to @{B}positive integer numbers@{UB} can be used as absolute markers. However, they @{B}must@{UB} use the equal sign in front of them. This ensures that ARexx can distinguish the symbol as a marker instead of a target. For example, this is a rewrite of Example35-12:- /* Example35-13 */ N1 = 1 N2 = 12 N3 = 32 N4 = 40 TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol =N1 Name =N2 Address =N3 Phone =N4 SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone Once again, the output will be:- Name = Joe Bloggs Address = 99 Somewhere Street Phone = 123 4567 @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Relative" "Article 35 - Parsing & Templates - Parsing With Positional Relative Markers" @{B}@{U}@{JCENTER}RELATIVE MARKERS@{UB}@{UU} @{JLEFT} @{B}Relative Markers@{UB} are ones that indicate a position @{B}relative@{UB} to the current position, e.g. 5 characters forward, or 10 characters backward. They must therefore always be @{B}integer@{UB} numbers or be capable of being evaluated as integer numbers. In reality, a relative marker is converted to an absolute position. Take this template as an example:- PARSE VAR TestSymbol Name +11 Address +20 Phone The default marker to the left of the target "Name" is an absolute of 1. The relative marker to the right of "Name", i.e. "+11", is converted an absolute marker as follows:- 1 + 11 = 12 and so our template becomes the equivalent of:- PARSE VAR TestSymbol Name =12 Address +20 Phone The relative marker "+20" is converted to an absolute marker as follows:- 12 + 20 = 32 and so our template becomes the equivalent of:- PARSE VAR TestSymbol Name =12 Address =32 Phone This is really the same as the template used in @{"Example35-12" LINK "Absolute"11} which was:- PARSE VAR TestSymbol =1 Name =12 Address =32 Phone =40 or, put more simply:- PARSE VAR TestSymbol Name =12 Address =32 Phone The use of these relative markers is illustrated by:- /* Example35-14 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol Name +11 Address +20 Phone SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone This one gives the same output:- Name = Joe Bloggs Address = 99 Somewhere Street Phone = 123 4567 Relative markers can be made to go backwards as well as forwards as this example shows us:- /* Example35-15 */ TestSymbol = 'Joe Bloggs 99 Somewhere Street 123 4567' PARSE VAR TestSymbol +31 Phone +8 -28 Address +19 -30 Name +10 SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone This time, the target symbols are in the reverse order. The absolute equivalents of the relative markers are calculated as follows:- PARSE VAR TestSymbol @{B}+31@{UB} Phone +8 -28 Address +19 -30 Name+10 --> PARSE VAR TestSymbol @{B}1+31@{UB} Phone +8 -28 Address +19 -30 Name+10 --> PARSE VAR TestSymbol @{B}=32@{UB} Phone +8 -28 Address +19 -30 Name+10 --> PARSE VAR TestSymbol =32 Phone @{B}32+8@{UB} -28 Address +19 -30 Name+10 --> PARSE VAR TestSymbol =32 Phone @{B}=40@{UB} -28 Address +19 -30 Name+10 --> PARSE VAR TestSymbol =32 Phone =40 @{B}40-28@{UB} Address +19 -30 Name+10 --> PARSE VAR TestSymbol =32 Phone =40 @{B}=12@{UB} Address +19 -30 Name+10 --> PARSE VAR TestSymbol =32 Phone =40 =12 Address @{B}12+19@{UB} -30 Name+10 --> PARSE VAR TestSymbol =32 Phone =40 =12 Address @{B}=31@{UB} -30 Name+10 --> PARSE VAR TestSymbol =32 Phone =40 =12 Address =31 @{B}31-30@{UB} Name+10 --> PARSE VAR TestSymbol =32 Phone =40 =12 Address =31 @{B}=1@{UB} Name+10 --> PARSE VAR TestSymbol =32 Phone =40 =12 Address =31 =1 Name @{B}1+10@{UB} --> PARSE VAR TestSymbol =32 Phone =40 =12 Address =31 =1 Name @{B}=11@{UB} If you follow down the figures in bold, you can see how ARexx starts at the left end of the template and converts the relative markers one by one into absolute markers. At each conversion, the absolute value of the marker to the left of the one being converted is used to create the absolute value of the one being converted. Earlier on, I said:- "If you wanted to make sure that the spaces at the 11th and 31st characters were @{B}not@{UB} included in the assignments then you could use:- PARSE VAR TestSymbol Name =11 =12 Address =31 =32 Phone which is equivalent to:- PARSE VAR TestSymbol =1 Name =11 =12 Address =31 =32 Phone=40 The template used in @{"Example35-15" LINK "Relative"58} can be represented, by the process shown above, as:- PARSE VAR TestSymbol =32 Phone =40 =12 Address =31 =1 Name =11 If you look at the markers on each side of the target symbols you will see that we have arrived at exactly the same thing! @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Mixing" "Article 35 - Parsing & Templates - Mixing Various Types Of Parsing" @{B}@{U}@{JCENTER}MIXING VARIOUS TYPES OF PARSING@{UB}@{UU} @{JLEFT} @{"Absolute" LINK "Absolute"} and @{"Relative" LINK "Relative"} markers can be mixed quite happily in the same template as they are both forms of @{"positional parsing" LINK "ParsePosition"}. @{"Tokenisation" LINK "Tokenisation"} and positional parsing appear to mix in the same template without problems. However, @{"pattern markers" LINK "PatternMarkers"} cannot be safely mixed with tokenisation or positional markers!@{UB} As you may have deduced by now, pattern markers react in a different way to positional markers. As already mentioned above, with positional markers, @{B}all@{UB} characters from an input string can be assigned to the target symbols whereas with pattern markers, those characters in the input string that match up to the characters in the marker are @{B}removed@{UB} from the temporary string and so cannot ever be allocated if the string is reread by the same template. This is illustrated by this program:- /* Example35-16 */ TestSymbol = 'Joe Bloggs:99 Somewhere Street:123 4567' PARSE VAR TestSymbol Name '99' Address '123' Phone =1 x =12 y =32 z=40 SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone SAY 'X =' x SAY 'Y =' y SAY 'Z =' z This will output:- Name = Joe Bloggs: Address = Somewhere Street: Phone = 4567 X = Joe Bloggs: Y = Somewhere Street: 4 Z = 567 The values assigned to the target symbols "Name", "Address" and "Phone" are the same as those assigned by the template used in @{"Example35-2" LINK "PatternMarkers"} which was:- PARSE VAR TestSymbol Name '99' Address '123' Phone The template in Example35-16 is the same template but with this bit added to the end of it:- =1 x =12 y =32 z =40 You would think that this would produce the same output that we got with most of our other examples, i.e.:- X = Joe Bloggs: Y = 99 Somewhere Street: Z = 123 4567 @{B}Not so!!@{UB} As seen above, it produces:- X = Joe Bloggs: Y = Somewhere Street: 4 Z = 567 So what has happened?!?! The pattern markers have, in fact, created a new temporary input string as follows:- String is:- Joe Bloggs:99 Somewhere Street:123 4567 Markers are:- 99 123 Temporary string:- Joe Bloggs: Somewhere Street: 4567 Now, when we use the part of the template that reads:- =1 x =12 y =32 z =40 The absolute markers are positioned at the spots marked by the asterisks:- Joe Bloggs: Somewhere Street: 4567 * * * * which explains the values given by the example program. @{B}The moral is, NEVER mix pattern and absolute/relative markers in the same template!!@{UB} @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "Multiple" "Article 35 - Parsing & Templates - Multiple Templates" @{B}@{U}@{JCENTER}MULTIPLE TEMPLATES@{UB}@{UU} @{JLEFT} Example35-17 (below) is a rewrite of @{"Example35-16" LINK "Mixing"15} (which did not work properly). It has @{B}two@{UB} templates, each separated by a comma, instead of just the one template. The templates from each example program are as follows:- Example35-16:- PARSE VAR TestSymbol Name '99' Address '123' Phone =1 x =12 y =32 z=40 Example35-17:- PARSE VAR TestSymbol Name '99' Address '123' Phone,=1 x =12 y =32 z=40 Note that the only difference is the comma in Example35-17 between "Phone" and "=1". /* Example35-17 */ TestSymbol = 'Joe Bloggs:99 Somewhere Street:123 4567' PARSE VAR TestSymbol Name '99' Address '123' Phone,=1 x =12 y =32 z=40 SAY 'Name =' Name SAY 'Address =' Address SAY 'Phone =' Phone SAY 'X =' x SAY 'Y =' y SAY 'Z =' z and our output would be more like we expected:- Name = Joe Bloggs: Address = Somewhere Street: Phone = 4567 X = Joe Bloggs: Y = 99 Somewhere Street: Z = 123 4567 Example35-17 demonstrates that the use of multiple templates is possible. A comma is used to separate each template that is used in a parsing instruction. You can have as many templates as you like in the one instruction. With the VAR input source that we have been using so far, the @{"temporary string" LINK "Patterns"} created for the first template is used by that template. When ARexx finds a comma, it discards the first temporary string and creates a new temporary string from the @{B}same input string@{UB} for the second template. If a third template is there, then the second temporary string is discarded and a third one created; and so on for as many times as there are templates in the same instruction clause. We will see in the next article that, in some cases (e.g. PULL), each template can read a @{B}different@{UB} input string. We will discuss these aspects more fully in the next article as we discuss each input source. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE @NODE "InputString" "Article 35 - Parsing & Templates - Input String NOT Changed By Parsing" @{B}@{U}@{JCENTER}THE INPUT STRING IS NOT CHANGED BY PARSING@{UB}@{UU} @{JLEFT} @{"Example 35-17" LINK "Multiple"} leads us to an important point, i.e. @{B}the input string is not changed@{UB} in any way by the parsing process. We saw that a temporary string was created for the use of the template and that this temporary string was used for all of the targets in the @{B}same@{UB} template. However, we also saw that @{B}another@{UB} template using the @{B}same@{UB} input string was able to use characters from the input string that, in the first template, had been @{B}removed@{UB} from the temporary string. In the @{"next article" LINK "ARB:Articles_31-40/36.Parse-Templates-2/MAIN"999} I will talk about the various input sources for the input strings. @{JCENTER}=== End of Text === @{JLEFT} @ENDNODE