start page | rating of books | rating of authors | reviews | copyrights

sed & awk

sed & awkSearch this book
Previous: 7.10 Passing Parameters Into a Script Chapter 7
Writing Scripts for awk
Next: 8. Conditionals, Loops, and Arrays
 

7.11 Information Retrieval

An awk program can be used to retrieve information from a database, the database basically being any kind of text file. The more structured the text file, the easier it is to work with, although the structure might be no more than a line consisting of individual words.

The list of acronyms below is a simple database.

$ 

cat acronyms

 BASIC	Beginner's All-Purpose Symbolic Instruction Code CICS	Customer Information Control System COBOL	Common Business Oriented Language DBMS	Data Base Management System GIGO	Garbage In, Garbage Out GIRL 	Generalized Information Retrieval Language

A tab is used as the field separator. We're going to look at a program that takes an acronym as input and displays the appropriate line from the database as output. (In the next chapter, we're going to look at two other programs that use the acronym database. One program reads the list of acronyms and then finds occurrences of these acronyms in another file. The other program locates the first occurrence of these acronyms in a text file and inserts the description of the acronym.)

The shell script that we develop is named acro . It takes the first argument from the command line (the name of the acronym) and passes it to the awk script. The acro script follows:

$ 

cat acro

 #! /bin/sh # assign shell's $1 to awk search variable awk '$1 == search' search=$1 acronyms

The first argument specified on the shell command line ($1) is assigned to the variable named search ; this variable is passed as a parameter into the awk program. Parameters passed to an awk program are specified after the script section. (This gets somewhat confusing, because $1 inside the awk program represents the first field of each input line, while $1 in the shell represents the first argument supplied on the command line.)

The example below demonstrates how this program can be used to find a particular acronym on our list.

$ 

acro CICS

 CICS Customer Information Control System

Notice that we tested the parameter as a string ( $1 == search ). We could also have written this as a regular expression match ( $1 ~ search ).

7.11.1 Finding a Glitch

A net posting was once forwarded to one of us because it contained a problem that could be solved using awk. Here's the original posting by Emmett Hogan:

I have been trying to rewrite a sed/tr/fgrep script that we use quite a bit here in Perl, but have thus far been unsuccessful...hence this posting.  Having never written anything in perl, and not wishing to wait for the Nutshell Perl Book, I figured I'd tap the knowledge of this group.  Basically, we have several files which have the format:  item	info line 1 	info line 2 	  . 	  . 	  . 	info line n  Where each info line refers to the item and is indented by either spaces or tabs.  Each item "block" is separated by a blank line.  What I need to do, is to be able to type:  info glitch filename  Where info is the name of the perl script, glitch is what I want to find out about, and filename is the name of the file with the information in it.  The catch is that I need it to print the entire "block" if it finds glitch anywhere in the file, i.e.:  machine		Sun 3/75 		8 meg memory 		Prone to memory glitches 		more info 		more info  would get printed if you looked for "glitch" along with any other "blocks" which contained the word glitch.  Currently we are using the following script:  #!/bin/csh -f # sed '/^ /\!s/^/@/' $2 | tr '\012@' '@\012' | fgrep -i $1 | tr '@' '\012'  Which is in a word....SLOW.  I am sure Perl can do it faster, better, etc...but I cannot figure it out.  Any, and all, help is greatly appreciated.  Thanks in advance, Emmett  ------------------------------------------------------------------- Emmett Hogan              Computer Science Lab, SRI International

The problem yielded a solution based on awk. You may want to try to tackle the problem yourself before reading any further. The solution relies on awk's multiline record capability and requires that you be able to pass the search string as a command-line parameter.

Here's the info script using awk:

awk 'BEGIN { FS = "\n"; RS = "" } $0 ~ search { print $0 }' search=$1 $2

Given a test file with multiple entries, info was tested to see if it could find the word "glitch."

$ 

info glitch glitch.test

 machine		Sun 3/75 		8 meg memory 		Prone to memory glitches 		more info 		more info

In the next chapter, we look at conditional and looping constructs, and arrays.


Previous: 7.10 Passing Parameters Into a Script sed & awk Next: 8. Conditionals, Loops, and Arrays
7.10 Passing Parameters Into a Script Book Index 8. Conditionals, Loops, and Arrays

The UNIX CD Bookshelf Navigation The UNIX CD BookshelfUNIX Power ToolsUNIX in a NutshellLearning the vi Editorsed & awkLearning the Korn ShellLearning the UNIX Operating System