HP-UX File Processing

:: Viewing contents of a file

The cat (concatenate) command can be used to view the contents of a file.

% cat friends
Sally Johnson     123 Ellesmere Lane       Athens        GA  12876   (413)345-8976
Bob Beamon        913 Mockingbird Lane     Annapolis     MD  17701   (201)962-8765
Kim Smith         234 Working Street       Hollywood     CA  18765   (678)987-9876
%

You will find on large files that the output of cat will not fit on your screen. The output will scroll across the screen and you will not be able to view the information. The more command is used to display file information in screen sized chunks. If the file is more than one screen long the word -More- followed by the percentage of the file displayed is placed at the bottom of the screen. Press the SpaceBar to see the next screen of the file or press q or to quit. It is better to use more if your output is going directly to the screen and to use cat if your output is going to a file or pipe.

::Pattern Searching

The grep (global regular expression print) command is used to search for a particular word or phrase in a file. The format of the grep command is:

grep [options] string file(s)

Where string is the word, phrase or regular expression you want to find, and file is the file to be searched. Lets find all lines that contain the string Sally in our friends file.

% grep Sally friends
Sally Johnson     123 Ellesmere Lane       Athens        GA  12876   (413)345-8976
Sally Miller      510 Epping Way           Hobe Sound    FL  33455   (654)546-4526
%

To find a pattern that is more than one word long, enclose the string within single or double quotation marks.

% grep 'Sally Miller' friends
Sally Miller      510 Epping Way           Hobe Sound    FL  33455   (654)546-4526
%

So far we have seen how grep prints all lines which contain the given character string. The -v (invert) option instructs grep to print all lines except those which match the string. This is usefull when you want to remove lines from a file. Lets find all the lines in a file that do not contain the string Sally and place the output into the friends.new file.

% grep -v 'Sally' > friends.new
% more friends.new
Bob Beamon        913 Mockingbird Lane     Annapolis     MD  17701   (201)962-8765
Kim Smith         234 Working Street       Hollywood     CA  18765   (678)987-9876
Eddy Murphy       210 W. Hollywood         Los Angles    CA  90210   (876)238-9987
%

The grep command can be used with regular expressions to match a pattern.

% grep '26$' friends
Sally Miller      510 Epping Way           Hobe Sound    FL  33455   (654)546-4526
%

This will find all the lines that end in "26".

The grep command can search for a string in groups of files. When it finds a pattern that matches in more that one file, it prints the name of the file, followed by a colon before the line matching the pattern. The -i (ignore) option to grep will ignore the case.

% grep -i bob *
friends:Bob Beamon        913 Mockingbird Lane     Annapolis     MD  17701   (201)962-8765
friends.new:Bob Beamon        913 Mockingbird Lane     Annapolis     MD  17701   (201)962-8765
%

:: Comparisons between files

The cmp (compare) command is used to compare any two files, including executable files. The diff (differential file operator) command is used to display the differences between text files or directories. The order of the diff command is important. The format of the diff command is:

diff [options] old_file new_file diff [options] old_dir new_dir

The diff command will display a listing of the editing actions that need to be performed on the old file to change it into the new file. The three ways in which diff indicated changes to a file are a (lines are added), c (lines are changed) and d(lines are deleted). Lets find the difference between the friends and the friends.new files.

% diff friends friends.new
1d0
< Sally Johnson     123 Ellesmere Lane       Athens        GA  12876   (413)345-8976
4d2
< Sally Miller      510 Epping Way           Hobe Sound    FL  33455   (654)546-4526
%

This tells us that deletions need to be made to the friends file to get the friend.new file. The 1d0 tells us that line 1 of the friend file has to be deleted, and the 4d2 tells us that line 4 of the friend file has to be deleted.

The output of diff can be sent to the ed command to convert the first file into the second. In fact, the using diff with the -e option will produce output suitable for input to the ed command.

% diff -e friends friends.new > ed.script
% ( cat ed.script ; echo w ) | ed friends
415
249
% diff friends friends.new
%

If you eliminated the echo w command, ed would read the input from the cat ed.script, perform the operation and then exit. With the echo w command you are telling ed to write to the file friends. The semicolon between the cat ed.script and echo w tells the shell that these are separate commands.

:: Counting things in a file

The wc (word count) command counts the number of lines, words, and characters in a file. This is especially usefull for counting the number of lines of source code in a program orthe number of words in a document.

% wc friends
3 28 249 friends
%

The first number indicates the number of lines, followed by the number of words and the number of characters(including newlines).

:: Modifying character strings

The sed (stream editor) command reads lines, one by one, from an input file and applies a set of editing commands to the lines. The sed command is like grep except that it allows you to make changes to file. The format of the sed command is:

sed [options] action/string/newstring file(s)

% sed "s/(201)962-8765/(201)962-9999/" friends
Bob Beamon        913 Mockingbird Lane     Annapolis     MD  17701   (201)962-9999
Kim Smith         234 Working Street       Hollywood     CA  18765   (678)987-9876
Eddy Murphy       210 W. Hollywood         Los Angles    CA  90210   (876)238-9987
%

The s (substitute) option tells sed to substitute. We have changed the phone number. If we wanted to save these changes we would redirect the output to a file.

The sed command also accepts regular expressions.

% sed '/^$/d' hello.cc
#include 
main() {
cout << "Hello, cruel world! \n";
}
%

The d option is tells sed to delete. In this case we are using a regular expression to tell sed to remove all blank lines from our hello.cc file.

:: Sorting data in files

The sort command is used to sort the contents of a file into alphabetic or numerical order.

% sort +1 friends
Bob Beamon        913 Mockingbird Lane     Annapolis     MD  17701   (201)962-8765
Eddy Murphy       210 W. Hollywood         Los Angles    CA  90210   (876)238-9987
Kim Smith         234 Working Street       Hollywood     CA  18765   (678)987-9876
%

The +1 option tells sort to use the second field of the sort key rather than the beginning of the line. Fields are separated by whitespace, however you can use the -t option to separate by any character. The -n options tells sort to sort a numeric key, otherwise it will sort by the corresponding ASCii values.

Another command that is usefull when you are sorting is the uniq command. The uniq command will write only one instance of each line, whereas the sort will print every line. Usually the output of the sort command is piped to the uniq command.

---

::enjoy life...