Introducing UNIX and Linux

Overview
Using filters
      Collating sequence
      Character classes
Character-to-character transformation
Selecting lines by content
      Regular expressions
      Basic regular expressions
      Extended regular expressions
      Grep
Stream editor
      Sed addresses
Splitting a file according to context
Choosing between the three filters
More on Vi
Summary
Exercises

Using filters

Any command that reads data from standard input, and writes data to the standard output stream, is known as a filter. Most UNIX utilities are filters, or can be used as such. Most utilities are also simple in their functionality.

Much UNIX programming involves transforming input, of a known form, to output, also of a known form. This output depends on the input in a specified manner, and often involves creating a pipeline. In order to use UNIX effectively, you must develop the skill of being able to choose utilities to pipe together. It is not always obvious why the filters that are 'standard' for UNIX have been developed, but they have been found to be very powerful 'building blocks'. We now examine three filters, all significantly more complex than those we have met before, but whose relevance will be striking.

The editor Vi is an interactive tool, and is one of the few UNIX utilities that cannot be used as a filter. The commands we now look at perform tasks you might consider suitable for an editor such as Vi, but are designed so that they can work as filters.

You may be wondering why, having discussed detailed syntax for the shell, we now need to introduce further commands. After all, we did claim that the shell was a 'full programming language'. In UNIX there are rarely unique solutions to problems, although some techniques are arguably 'better' than others. Most tasks have many possible ways a UNIX programmer can solve them, but an experienced programmer will be able to chose a solution that can be implemented fast and efficiently. The purpose of commands such as grep, sed, tr and awk (discussed in the next chapter) is to provide speedy solutions for tasks that are frequently encountered, and difficult to program using the shell alone.

Whenever you write a shell script that takes more than a few lines, stand back from the problem for a moment and ask yourself whether some utility will do the job for you. Maybe there isn't, but often you will be able to save yourself time and trouble by recalling a utility you'd almost forgotten about. When you have finished reading this book, go back to the earlier chapters and remind yourself of the commands - such as comm and uniq - whose usefulness may not then have been apparent.