This is Info file gawk.info, produced by Makeinfo version 1.67 from the input file /ade-src/fsf/gawk/doc/gawk.texi. INFO-DIR-SECTION Programming Languages START-INFO-DIR-ENTRY * Gawk: (gawk.info). A Text Scanning and Processing Language. END-INFO-DIR-ENTRY This file documents `awk', a program that you can use to select particular records in a file and perform operations upon them. This is Edition 1.0.2 of `The GNU Awk User's Guide', for the 3.0.2 version of the GNU implementation of AWK. Copyright (C) 1989, 1991, 92, 93, 96 Free Software Foundation, Inc. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Foundation.  File: gawk.info, Node: Scanning an Array, Next: Delete, Prev: Array Example, Up: Arrays Scanning All Elements of an Array ================================= In programs that use arrays, you often need a loop that executes once for each element of an array. In other languages, where arrays are contiguous and indices are limited to positive integers, this is easy: you can find all the valid indices by counting from the lowest index up to the highest. This technique won't do the job in `awk', since any number or string can be an array index. So `awk' has a special kind of `for' statement for scanning an array: for (VAR in ARRAY) BODY This loop executes BODY once for each index in ARRAY that your program has previously used, with the variable VAR set to that index. Here is a program that uses this form of the `for' statement. The first rule scans the input records and notes which words appear (at least once) in the input, by storing a one into the array `used' with the word as index. The second rule scans the elements of `used' to find all the distinct words that appear in the input. It prints each word that is more than 10 characters long, and also prints the number of such words. *Note Built-in Functions for String Manipulation: String Functions, for more information on the built-in function `length'. # Record a 1 for each word that is used at least once. { for (i = 1; i <= NF; i++) used[$i] = 1 } # Find number of distinct words more than 10 characters long. END { for (x in used) if (length(x) > 10) { ++num_long_words print x } print num_long_words, "words longer than 10 characters" } *Note Generating Word Usage Counts: Word Sorting, for a more detailed example of this type. The order in which elements of the array are accessed by this statement is determined by the internal arrangement of the array elements within `awk' and cannot be controlled or changed. This can lead to problems if new elements are added to ARRAY by statements in the loop body; you cannot predict whether or not the `for' loop will reach them. Similarly, changing VAR inside the loop may produce strange results. It is best to avoid such things.  File: gawk.info, Node: Delete, Next: Numeric Array Subscripts, Prev: Scanning an Array, Up: Arrays The `delete' Statement ====================== You can remove an individual element of an array using the `delete' statement: delete ARRAY[INDEX] Once you have deleted an array element, you can no longer obtain any value the element once had. It is as if you had never referred to it and had never given it any value. Here is an example of deleting elements in an array: for (i in frequencies) delete frequencies[i] This example removes all the elements from the array `frequencies'. If you delete an element, a subsequent `for' statement to scan the array will not report that element, and the `in' operator to check for the presence of that element will return zero (i.e. false): delete foo[4] if (4 in foo) print "This will never be printed" It is important to note that deleting an element is *not* the same as assigning it a null value (the empty string, `""'). foo[4] = "" if (4 in foo) print "This is printed, even though foo[4] is empty" It is not an error to delete an element that does not exist. You can delete all the elements of an array with a single statement, by leaving off the subscript in the `delete' statement. delete ARRAY This ability is a `gawk' extension; it is not available in compatibility mode (*note Command Line Options: Options.). Using this version of the `delete' statement is about three times more efficient than the equivalent loop that deletes each element one at a time. The following statement provides a portable, but non-obvious way to clear out an array. # thanks to Michael Brennan for pointing this out split("", array) The `split' function (*note Built-in Functions for String Manipulation: String Functions.) clears out the target array first. This call asks it to split apart the null string. Since there is no data to split out, the function simply clears the array and then returns.  File: gawk.info, Node: Numeric Array Subscripts, Next: Uninitialized Subscripts, Prev: Delete, Up: Arrays Using Numbers to Subscript Arrays ================================= An important aspect of arrays to remember is that *array subscripts are always strings*. If you use a numeric value as a subscript, it will be converted to a string value before it is used for subscripting (*note Conversion of Strings and Numbers: Conversion.). This means that the value of the built-in variable `CONVFMT' can potentially affect how your program accesses elements of an array. For example: xyz = 12.153 data[xyz] = 1 CONVFMT = "%2.2f" if (xyz in data) printf "%s is in data\n", xyz else printf "%s is not in data\n", xyz This prints `12.15 is not in data'. The first statement gives `xyz' a numeric value. Assigning to `data[xyz]' subscripts `data' with the string value `"12.153"' (using the default conversion value of `CONVFMT', `"%.6g"'), and assigns one to `data["12.153"]'. The program then changes the value of `CONVFMT'. The test `(xyz in data)' generates a new string value from `xyz', this time `"12.15"', since the value of `CONVFMT' only allows two significant digits. This test fails, since `"12.15"' is a different string from `"12.153"'. According to the rules for conversions (*note Conversion of Strings and Numbers: Conversion.), integer values are always converted to strings as integers, no matter what the value of `CONVFMT' may happen to be. So the usual case of: for (i = 1; i <= maxsub; i++) do something with array[i] will work, no matter what the value of `CONVFMT'. Like many things in `awk', the majority of the time things work as you would expect them to work. But it is useful to have a precise knowledge of the actual rules, since sometimes they can have a subtle effect on your programs.  File: gawk.info, Node: Uninitialized Subscripts, Next: Multi-dimensional, Prev: Numeric Array Subscripts, Up: Arrays Using Uninitialized Variables as Subscripts =========================================== Suppose you want to print your input data in reverse order. A reasonable attempt at a program to do so (with some test data) might look like this: $ echo 'line 1 > line 2 > line 3' | awk '{ l[lines] = $0; ++lines } > END { > for (i = lines-1; i >= 0; --i) > print l[i] > }' -| line 3 -| line 2 Unfortunately, the very first line of input data did not come out in the output! At first glance, this program should have worked. The variable `lines' is uninitialized, and uninitialized variables have the numeric value zero. So, the value of `l[0]' should have been printed. The issue here is that subscripts for `awk' arrays are *always* strings. And uninitialized variables, when used as strings, have the value `""', not zero. Thus, `line 1' ended up stored in `l[""]'. The following version of the program works correctly: { l[lines++] = $0 } END { for (i = lines - 1; i >= 0; --i) print l[i] } Here, the `++' forces `l' to be numeric, thus making the "old value" numeric zero, which is then converted to `"0"' as the array subscript. As we have just seen, even though it is somewhat unusual, the null string (`""') is a valid array subscript (d.c.). If `--lint' is provided on the command line (*note Command Line Options: Options.), `gawk' will warn about the use of the null string as a subscript.  File: gawk.info, Node: Multi-dimensional, Next: Multi-scanning, Prev: Uninitialized Subscripts, Up: Arrays Multi-dimensional Arrays ======================== A multi-dimensional array is an array in which an element is identified by a sequence of indices, instead of a single index. For example, a two-dimensional array requires two indices. The usual way (in most languages, including `awk') to refer to an element of a two-dimensional array named `grid' is with `grid[X,Y]'. Multi-dimensional arrays are supported in `awk' through concatenation of indices into one string. What happens is that `awk' converts the indices into strings (*note Conversion of Strings and Numbers: Conversion.) and concatenates them together, with a separator between them. This creates a single string that describes the values of the separate indices. The combined string is used as a single index into an ordinary, one-dimensional array. The separator used is the value of the built-in variable `SUBSEP'. For example, suppose we evaluate the expression `foo[5,12] = "value"' when the value of `SUBSEP' is `"@"'. The numbers five and 12 are converted to strings and concatenated with an `@' between them, yielding `"5@12"'; thus, the array element `foo["5@12"]' is set to `"value"'. Once the element's value is stored, `awk' has no record of whether it was stored with a single index or a sequence of indices. The two expressions `foo[5,12]' and `foo[5 SUBSEP 12]' are always equivalent. The default value of `SUBSEP' is the string `"\034"', which contains a non-printing character that is unlikely to appear in an `awk' program or in most input data. The usefulness of choosing an unlikely character comes from the fact that index values that contain a string matching `SUBSEP' lead to combined strings that are ambiguous. Suppose that `SUBSEP' were `"@"'; then `foo["a@b", "c"]' and `foo["a", "b@c"]' would be indistinguishable because both would actually be stored as `foo["a@b@c"]'. You can test whether a particular index-sequence exists in a "multi-dimensional" array with the same operator `in' used for single dimensional arrays. Instead of a single index as the left-hand operand, write the whole sequence of indices, separated by commas, in parentheses: (SUBSCRIPT1, SUBSCRIPT2, ...) in ARRAY The following example treats its input as a two-dimensional array of fields; it rotates this array 90 degrees clockwise and prints the result. It assumes that all lines have the same number of elements. awk '{ if (max_nf < NF) max_nf = NF max_nr = NR for (x = 1; x <= NF; x++) vector[x, NR] = $x } END { for (x = 1; x <= max_nf; x++) { for (y = max_nr; y >= 1; --y) printf("%s ", vector[x, y]) printf("\n") } }' When given the input: 1 2 3 4 5 6 2 3 4 5 6 1 3 4 5 6 1 2 4 5 6 1 2 3 it produces: 4 3 2 1 5 4 3 2 6 5 4 3 1 6 5 4 2 1 6 5 3 2 1 6  File: gawk.info, Node: Multi-scanning, Prev: Multi-dimensional, Up: Arrays Scanning Multi-dimensional Arrays ================================= There is no special `for' statement for scanning a "multi-dimensional" array; there cannot be one, because in truth there are no multi-dimensional arrays or elements; there is only a multi-dimensional *way of accessing* an array. However, if your program has an array that is always accessed as multi-dimensional, you can get the effect of scanning it by combining the scanning `for' statement (*note Scanning All Elements of an Array: Scanning an Array.) with the `split' built-in function (*note Built-in Functions for String Manipulation: String Functions.). It works like this: for (combined in array) { split(combined, separate, SUBSEP) ... } This sets `combined' to each concatenated, combined index in the array, and splits it into the individual indices by breaking it apart where the value of `SUBSEP' appears. The split-out indices become the elements of the array `separate'. Thus, suppose you have previously stored a value in `array[1, "foo"]'; then an element with index `"1\034foo"' exists in `array'. (Recall that the default value of `SUBSEP' is the character with code 034.) Sooner or later the `for' statement will find that index and do an iteration with `combined' set to `"1\034foo"'. Then the `split' function is called as follows: split("1\034foo", separate, "\034") The result of this is to set `separate[1]' to `"1"' and `separate[2]' to `"foo"'. Presto, the original sequence of separate indices has been recovered.  File: gawk.info, Node: Built-in, Next: User-defined, Prev: Arrays, Up: Top Built-in Functions ****************** "Built-in" functions are functions that are always available for your `awk' program to call. This chapter defines all the built-in functions in `awk'; some of them are mentioned in other sections, but they are summarized here for your convenience. (You can also define new functions yourself. *Note User-defined Functions: User-defined.) * Menu: * Calling Built-in:: How to call built-in functions. * Numeric Functions:: Functions that work with numbers, including `int', `sin' and `rand'. * String Functions:: Functions for string manipulation, such as `split', `match', and `sprintf'. * I/O Functions:: Functions for files and shell commands. * Time Functions:: Functions for dealing with time stamps.  File: gawk.info, Node: Calling Built-in, Next: Numeric Functions, Prev: Built-in, Up: Built-in Calling Built-in Functions ========================== To call a built-in function, write the name of the function followed by arguments in parentheses. For example, `atan2(y + z, 1)' is a call to the function `atan2', with two arguments. Whitespace is ignored between the built-in function name and the open-parenthesis, but we recommend that you avoid using whitespace there. User-defined functions do not permit whitespace in this way, and you will find it easier to avoid mistakes by following a simple convention which always works: no whitespace after a function name. Each built-in function accepts a certain number of arguments. In some cases, arguments can be omitted. The defaults for omitted arguments vary from function to function and are described under the individual functions. In some `awk' implementations, extra arguments given to built-in functions are ignored. However, in `gawk', it is a fatal error to give extra arguments to a built-in function. When a function is called, expressions that create the function's actual parameters are evaluated completely before the function call is performed. For example, in the code fragment: i = 4 j = sqrt(i++) the variable `i' is set to five before `sqrt' is called with a value of four for its actual parameter. The order of evaluation of the expressions used for the function's parameters is undefined. Thus, you should not write programs that assume that parameters are evaluated from left to right or from right to left. For example, i = 5 j = atan2(i++, i *= 2) If the order of evaluation is left to right, then `i' first becomes six, and then 12, and `atan2' is called with the two arguments six and 12. But if the order of evaluation is right to left, `i' first becomes 10, and then 11, and `atan2' is called with the two arguments 11 and 10.  File: gawk.info, Node: Numeric Functions, Next: String Functions, Prev: Calling Built-in, Up: Built-in Numeric Built-in Functions ========================== Here is a full list of built-in functions that work with numbers. Optional parameters are enclosed in square brackets ("[" and "]"). `int(X)' This produces the nearest integer to X, located between X and zero, truncated toward zero. For example, `int(3)' is three, `int(3.9)' is three, `int(-3.9)' is -3, and `int(-3)' is -3 as well. `sqrt(X)' This gives you the positive square root of X. It reports an error if X is negative. Thus, `sqrt(4)' is two. `exp(X)' This gives you the exponential of X (`e ^ X'), or reports an error if X is out of range. The range of values X can have depends on your machine's floating point representation. `log(X)' This gives you the natural logarithm of X, if X is positive; otherwise, it reports an error. `sin(X)' This gives you the sine of X, with X in radians. `cos(X)' This gives you the cosine of X, with X in radians. `atan2(Y, X)' This gives you the arctangent of `Y / X' in radians. `rand()' This gives you a random number. The values of `rand' are uniformly-distributed between zero and one. The value is never zero and never one. Often you want random integers instead. Here is a user-defined function you can use to obtain a random non-negative integer less than N: function randint(n) { return int(n * rand()) } The multiplication produces a random real number greater than zero and less than `n'. We then make it an integer (using `int') between zero and `n' - 1, inclusive. Here is an example where a similar function is used to produce random integers between one and N. This program prints a new random number for each input record. awk ' # Function to roll a simulated die. function roll(n) { return 1 + int(rand() * n) } # Roll 3 six-sided dice and # print total number of points. { printf("%d points\n", roll(6)+roll(6)+roll(6)) }' *Caution:* In most `awk' implementations, including `gawk', `rand' starts generating numbers from the same starting number, or "seed", each time you run `awk'. Thus, a program will generate the same results each time you run it. The numbers are random within one `awk' run, but predictable from run to run. This is convenient for debugging, but if you want a program to do different things each time it is used, you must change the seed to a value that will be different in each run. To do this, use `srand'. `srand([X])' The function `srand' sets the starting point, or seed, for generating random numbers to the value X. Each seed value leads to a particular sequence of random numbers.(1) Thus, if you set the seed to the same value a second time, you will get the same sequence of random numbers again. If you omit the argument X, as in `srand()', then the current date and time of day are used for a seed. This is the way to get random numbers that are truly unpredictable. The return value of `srand' is the previous seed. This makes it easy to keep track of the seeds for use in consistently reproducing sequences of random numbers. ---------- Footnotes ---------- (1) Computer generated random numbers really are not truly random. They are technically known as "pseudo-random." This means that while the numbers in a sequence appear to be random, you can in fact generate the same sequence of random numbers over and over again.  File: gawk.info, Node: String Functions, Next: I/O Functions, Prev: Numeric Functions, Up: Built-in Built-in Functions for String Manipulation ========================================== The functions in this section look at or change the text of one or more strings. Optional parameters are enclosed in square brackets ("[" and "]"). `index(IN, FIND)' This searches the string IN for the first occurrence of the string FIND, and returns the position in characters where that occurrence begins in the string IN. For example: $ awk 'BEGIN { print index("peanut", "an") }' -| 3 If FIND is not found, `index' returns zero. (Remember that string indices in `awk' start at one.) `length([STRING])' This gives you the number of characters in STRING. If STRING is a number, the length of the digit string representing that number is returned. For example, `length("abcde")' is five. By contrast, `length(15 * 35)' works out to three. How? Well, 15 * 35 = 525, and 525 is then converted to the string `"525"', which has three characters. If no argument is supplied, `length' returns the length of `$0'. In older versions of `awk', you could call the `length' function without any parentheses. Doing so is marked as "deprecated" in the POSIX standard. This means that while you can do this in your programs, it is a feature that can eventually be removed from a future version of the standard. Therefore, for maximal portability of your `awk' programs, you should always supply the parentheses. `match(STRING, REGEXP)' The `match' function searches the string, STRING, for the longest, leftmost substring matched by the regular expression, REGEXP. It returns the character position, or "index", of where that substring begins (one, if it starts at the beginning of STRING). If no match is found, it returns zero. The `match' function sets the built-in variable `RSTART' to the index. It also sets the built-in variable `RLENGTH' to the length in characters of the matched substring. If no match is found, `RSTART' is set to zero, and `RLENGTH' to -1. For example: awk '{ if ($1 == "FIND") regex = $2 else { where = match($0, regex) if (where != 0) print "Match of", regex, "found at", \ where, "in", $0 } }' This program looks for lines that match the regular expression stored in the variable `regex'. This regular expression can be changed. If the first word on a line is `FIND', `regex' is changed to be the second word on that line. Therefore, given: FIND ru+n My program runs but not very quickly FIND Melvin JF+KM This line is property of Reality Engineering Co. Melvin was here. `awk' prints: Match of ru+n found at 12 in My program runs Match of Melvin found at 1 in Melvin was here. `split(STRING, ARRAY [, FIELDSEP])' This divides STRING into pieces separated by FIELDSEP, and stores the pieces in ARRAY. The first piece is stored in `ARRAY[1]', the second piece in `ARRAY[2]', and so forth. The string value of the third argument, FIELDSEP, is a regexp describing where to split STRING (much as `FS' can be a regexp describing where to split input records). If the FIELDSEP is omitted, the value of `FS' is used. `split' returns the number of elements created. The `split' function splits strings into pieces in a manner similar to the way input lines are split into fields. For example: split("cul-de-sac", a, "-") splits the string `cul-de-sac' into three fields using `-' as the separator. It sets the contents of the array `a' as follows: a[1] = "cul" a[2] = "de" a[3] = "sac" The value returned by this call to `split' is three. As with input field-splitting, when the value of FIELDSEP is `" "', leading and trailing whitespace is ignored, and the elements are separated by runs of whitespace. Also as with input field-splitting, if FIELDSEP is the null string, each individual character in the string is split into its own array element. (This is a `gawk'-specific extension.) Recent implementations of `awk', including `gawk', allow the third argument to be a regexp constant (`/abc/'), as well as a string (d.c.). The POSIX standard allows this as well. Before splitting the string, `split' deletes any previously existing elements in the array ARRAY (d.c.). `sprintf(FORMAT, EXPRESSION1,...)' This returns (without printing) the string that `printf' would have printed out with the same arguments (*note Using `printf' Statements for Fancier Printing: Printf.). For example: sprintf("pi = %.2f (approx.)", 22/7) returns the string `"pi = 3.14 (approx.)"'. `sub(REGEXP, REPLACEMENT [, TARGET])' The `sub' function alters the value of TARGET. It searches this value, which is treated as a string, for the leftmost longest substring matched by the regular expression, REGEXP, extending this match as far as possible. Then the entire string is changed by replacing the matched text with REPLACEMENT. The modified string becomes the new value of TARGET. This function is peculiar because TARGET is not simply used to compute a value, and not just any expression will do: it must be a variable, field or array element, so that `sub' can store a modified value there. If this argument is omitted, then the default is to use and alter `$0'. For example: str = "water, water, everywhere" sub(/at/, "ith", str) sets `str' to `"wither, water, everywhere"', by replacing the leftmost, longest occurrence of `at' with `ith'. The `sub' function returns the number of substitutions made (either one or zero). If the special character `&' appears in REPLACEMENT, it stands for the precise substring that was matched by REGEXP. (If the regexp can match more than one string, then this precise substring may vary.) For example: awk '{ sub(/candidate/, "& and his wife"); print }' changes the first occurrence of `candidate' to `candidate and his wife' on each input line. Here is another example: awk 'BEGIN { str = "daabaaa" sub(/a*/, "c&c", str) print str }' -| dcaacbaaa This shows how `&' can represent a non-constant string, and also illustrates the "leftmost, longest" rule in regexp matching (*note How Much Text Matches?: Leftmost Longest.). The effect of this special character (`&') can be turned off by putting a backslash before it in the string. As usual, to insert one backslash in the string, you must write two backslashes. Therefore, write `\\&' in a string constant to include a literal `&' in the replacement. For example, here is how to replace the first `|' on each line with an `&': awk '{ sub(/\|/, "\\&"); print }' *Note:* As mentioned above, the third argument to `sub' must be a variable, field or array reference. Some versions of `awk' allow the third argument to be an expression which is not an lvalue. In such a case, `sub' would still search for the pattern and return zero or one, but the result of the substitution (if any) would be thrown away because there is no place to put it. Such versions of `awk' accept expressions like this: sub(/USA/, "United States", "the USA and Canada") For historical compatibility, `gawk' will accept erroneous code, such as in the above example. However, using any other non-changeable object as the third parameter will cause a fatal error, and your program will not run. `gsub(REGEXP, REPLACEMENT [, TARGET])' This is similar to the `sub' function, except `gsub' replaces *all* of the longest, leftmost, *non-overlapping* matching substrings it can find. The `g' in `gsub' stands for "global," which means replace everywhere. For example: awk '{ gsub(/Britain/, "United Kingdom"); print }' replaces all occurrences of the string `Britain' with `United Kingdom' for all input records. The `gsub' function returns the number of substitutions made. If the variable to be searched and altered, TARGET, is omitted, then the entire input record, `$0', is used. As in `sub', the characters `&' and `\' are special, and the third argument must be an lvalue. `gensub(REGEXP, REPLACEMENT, HOW [, TARGET])' `gensub' is a general substitution function. Like `sub' and `gsub', it searches the target string TARGET for matches of the regular expression REGEXP. Unlike `sub' and `gsub', the modified string is returned as the result of the function, and the original target string is *not* changed. If HOW is a string beginning with `g' or `G', then it replaces all matches of REGEXP with REPLACEMENT. Otherwise, HOW is a number indicating which match of REGEXP to replace. If no TARGET is supplied, `$0' is used instead. `gensub' provides an additional feature that is not available in `sub' or `gsub': the ability to specify components of a regexp in the replacement text. This is done by using parentheses in the regexp to mark the components, and then specifying `\N' in the replacement text, where N is a digit from one to nine. For example: $ gawk ' > BEGIN { > a = "abc def" > b = gensub(/(.+) (.+)/, "\\2 \\1", "g", a) > print b > }' -| def abc As described above for `sub', you must type two backslashes in order to get one into the string. In the replacement text, the sequence `\0' represents the entire matched text, as does the character `&'. This example shows how you can use the third argument to control which match of the regexp should be changed. $ echo a b c a b c | > gawk '{ print gensub(/a/, "AA", 2) }' -| a b c AA b c In this case, `$0' is used as the default target string. `gensub' returns the new string as its result, which is passed directly to `print' for printing. If the HOW argument is a string that does not begin with `g' or `G', or if it is a number that is less than zero, only one substitution is performed. `gensub' is a `gawk' extension; it is not available in compatibility mode (*note Command Line Options: Options.). `substr(STRING, START [, LENGTH])' This returns a LENGTH-character-long substring of STRING, starting at character number START. The first character of a string is character number one. For example, `substr("washington", 5, 3)' returns `"ing"'. If LENGTH is not present, this function returns the whole suffix of STRING that begins at character number START. For example, `substr("washington", 5)' returns `"ington"'. The whole suffix is also returned if LENGTH is greater than the number of characters remaining in the string, counting from character number START. *Note:* The string returned by `substr' *cannot* be assigned to. Thus, it is a mistake to attempt to change a portion of a string, like this: string = "abcdef" # try to get "abCDEf", won't work substr(string, 3, 3) = "CDE" or to use `substr' as the third agument of `sub' or `gsub': gsub(/xyz/, "pdq", substr($0, 5, 20)) # WRONG `tolower(STRING)' This returns a copy of STRING, with each upper-case character in the string replaced with its corresponding lower-case character. Non-alphabetic characters are left unchanged. For example, `tolower("MiXeD cAsE 123")' returns `"mixed case 123"'. `toupper(STRING)' This returns a copy of STRING, with each lower-case character in the string replaced with its corresponding upper-case character. Non-alphabetic characters are left unchanged. For example, `toupper("MiXeD cAsE 123")' returns `"MIXED CASE 123"'. More About `\' and `&' with `sub', `gsub' and `gensub' ------------------------------------------------------ When using `sub', `gsub' or `gensub', and trying to get literal backslashes and ampersands into the replacement text, you need to remember that there are several levels of "escape processing" going on. First, there is the "lexical" level, which is when `awk' reads your program, and builds an internal copy of your program that can be executed. Then there is the run-time level, when `awk' actually scans the replacement string to determine what to generate. At both levels, `awk' looks for a defined set of characters that can come after a backslash. At the lexical level, it looks for the escape sequences listed in *Note Escape Sequences::.. Thus, for every `\' that `awk' will process at the run-time level, you type two `\'s at the lexical level. When a character that is not valid for an escape sequence follows the `\', Unix `awk' and `gawk' both simply remove the initial `\', and put the following character into the string. Thus, for example, `"a\qb"' is treated as `"aqb"'. At the run-time level, the various functions handle sequences of `\' and `&' differently. The situation is (sadly) somewhat complex. Historically, the `sub' and `gsub' functions treated the two character sequence `\&' specially; this sequence was replaced in the generated text with a single `&'. Any other `\' within the REPLACEMENT string that did not precede an `&' was passed through unchanged. To illustrate with a table: You type `sub' sees `sub' generates -------- ---------- --------------- `\&' `&' the matched text `\\&' `\&' a literal `&' `\\\&' `\&' a literal `&' `\\\\&' `\\&' a literal `\&' `\\\\\&' `\\&' a literal `\&' `\\\\\\&' `\\\&' a literal `\\&' `\\q' `\q' a literal `\q' This table shows both the lexical level processing, where an odd number of backslashes becomes an even number at the run time level, and the run-time processing done by `sub'. (For the sake of simplicity, the rest of the tables below only show the case of even numbers of `\'s entered at the lexical level.) The problem with the historical approach is that there is no way to get a literal `\' followed by the matched text. The 1992 POSIX standard attempted to fix this problem. The standard says that `sub' and `gsub' look for either a `\' or an `&' after the `\'. If either one follows a `\', that character is output literally. The interpretation of `\' and `&' then becomes like this: You type `sub' sees `sub' generates -------- ---------- --------------- `&' `&' the matched text `\\&' `\&' a literal `&' `\\\\&' `\\&' a literal `\', then the matched text `\\\\\\&' `\\\&' a literal `\&' This would appear to solve the problem. Unfortunately, the phrasing of the standard is unusual. It says, in effect, that `\' turns off the special meaning of any following character, but that for anything other than `\' and `&', such special meaning is undefined. This wording leads to two problems. 1. Backslashes must now be doubled in the REPLACEMENT string, breaking historical `awk' programs. 2. To make sure that an `awk' program is portable, *every* character in the REPLACEMENT string must be preceded with a backslash.(1) The POSIX standard is under revision.(2) Because of the above problems, proposed text for the revised standard reverts to rules that correspond more closely to the original existing practice. The proposed rules have special cases that make it possible to produce a `\' preceding the matched text. You type `sub' sees `sub' generates -------- ---------- --------------- `\\\\\\&' `\\\&' a literal `\&' `\\\\&' `\\&' a literal `\', followed by the matched text `\\&' `\&' a literal `&' `\\q' `\q' a literal `\q' In a nutshell, at the run-time level, there are now three special sequences of characters, `\\\&', `\\&' and `\&', whereas historically, there was only one. However, as in the historical case, any `\' that is not part of one of these three sequences is not special, and appears in the output literally. `gawk' 3.0 follows these proposed POSIX rules for `sub' and `gsub'. Whether these proposed rules will actually become codified into the standard is unknown at this point. Subsequent `gawk' releases will track the standard and implement whatever the final version specifies; this Info file will be updated as well. The rules for `gensub' are considerably simpler. At the run-time level, whenever `gawk' sees a `\', if the following character is a digit, then the text that matched the corresponding parenthesized subexpression is placed in the generated output. Otherwise, no matter what the character after the `\' is, that character will appear in the generated text, and the `\' will not. You type `gensub' sees `gensub' generates -------- ------------- ------------------ `&' `&' the matched text `\\&' `\&' a literal `&' `\\\\' `\\' a literal `\' `\\\\&' `\\&' a literal `\', then the matched text `\\\\\\&' `\\\&' a literal `\&' `\\q' `\q' a literal `q' Because of the complexity of the lexical and run-time level processing, and the special cases for `sub' and `gsub', we recommend the use of `gawk' and `gensub' for when you have to do substitutions. ---------- Footnotes ---------- (1) This consequence was certainly unintended. (2) As of December 1995, with final approval and publication hopefully sometime in 1996.  File: gawk.info, Node: I/O Functions, Next: Time Functions, Prev: String Functions, Up: Built-in Built-in Functions for Input/Output =================================== The following functions are related to Input/Output (I/O). Optional parameters are enclosed in square brackets ("[" and "]"). `close(FILENAME)' Close the file FILENAME, for input or output. The argument may alternatively be a shell command that was used for redirecting to or from a pipe; then the pipe is closed. *Note Closing Input and Output Files and Pipes: Close Files And Pipes, for more information. `fflush([FILENAME])' Flush any buffered output associated FILENAME, which is either a file opened for writing, or a shell command for redirecting output to a pipe. Many utility programs will "buffer" their output; they save information to be written to a disk file or terminal in memory, until there is enough for it to be worthwhile to send the data to the ouput device. This is often more efficient than writing every little bit of information as soon as it is ready. However, sometimes it is necessary to force a program to "flush" its buffers; that is, write the information to its destination, even if a buffer is not full. This is the purpose of the `fflush' function; `gawk' too buffers its output, and the `fflush' function can be used to force `gawk' to flush its buffers. `fflush' is a recent (1994) addition to the Bell Labs research version of `awk'; it is not part of the POSIX standard, and will not be available if `--posix' has been specified on the command line (*note Command Line Options: Options.). `gawk' extends the `fflush' function in two ways. The first is to allow no argument at all. In this case, the buffer for the standard output is flushed. The second way is to allow the null string (`""') as the argument. In this case, the buffers for *all* open output files and pipes are flushed. `fflush' returns zero if the buffer was successfully flushed, and nonzero otherwise. `system(COMMAND)' The system function allows the user to execute operating system commands and then return to the `awk' program. The `system' function executes the command given by the string COMMAND. It returns, as its value, the status returned by the command that was executed. For example, if the following fragment of code is put in your `awk' program: END { system("date | mail -s 'awk run done' root") } the system administrator will be sent mail when the `awk' program finishes processing input and begins its end-of-input processing. Note that redirecting `print' or `printf' into a pipe is often enough to accomplish your task. However, if your `awk' program is interactive, `system' is useful for cranking up large self-contained programs, such as a shell or an editor. Some operating systems cannot implement the `system' function. `system' causes a fatal error if it is not supported. Interactive vs. Non-Interactive Buffering ----------------------------------------- As a side point, buffering issues can be even more confusing depending upon whether or not your program is "interactive", i.e., communicating with a user sitting at a keyboard.(1) Interactive programs generally "line buffer" their output; they write out every line. Non-interactive programs wait until they have a full buffer, which may be many lines of output. Here is an example of the difference. $ awk '{ print $1 + $2 }' 1 1 -| 2 2 3 -| 5 Control-d Each line of output is printed immediately. Compare that behavior with this example. $ awk '{ print $1 + $2 }' | cat 1 1 2 3 Control-d -| 2 -| 5 Here, no output is printed until after the `Control-d' is typed, since it is all buffered, and sent down the pipe to `cat' in one shot. Controlling Output Buffering with `system' ------------------------------------------ The `fflush' function provides explicit control over output buffering for individual files and pipes. However, its use is not portable to many other `awk' implementations. An alternative method to flush output buffers is by calling `system' with a null string as its argument: system("") # flush output `gawk' treats this use of the `system' function as a special case, and is smart enough not to run a shell (or other command interpreter) with the empty command. Therefore, with `gawk', this idiom is not only useful, it is efficient. While this method should work with other `awk' implementations, it will not necessarily avoid starting an unnecessary shell. (Other implementations may only flush the buffer associated with the standard output, and not necessarily all buffered output.) If you think about what a programmer expects, it makes sense that `system' should flush any pending output. The following program: BEGIN { print "first print" system("echo system echo") print "second print" } must print first print system echo second print and not system echo first print second print If `awk' did not flush its buffers before calling `system', the latter (undesirable) output is what you would see. ---------- Footnotes ---------- (1) A program is interactive if the standard output is connected to a terminal device.