Dealing with Too Many Arguments (Unix Power Tools, 3rd Edition)

start page | rating of books | rating of authors | reviews | copyrights

28.17. Dealing with Too Many Arguments

Historically, one of the more annoying things about the design of many UNIX tools was their inability to handle large numbers of arguments. For example, if you wanted to print several hundred files using lpr, you either had to pass them a few at a time, perhaps using wildcards on the command line to split the list up into shorter groups, or call lpr once per file, perhaps using find or a loop. One other method, which is still useful today, involves the use of xargs.

xargs is one of those Unix utilities that seems pretty useless when you first hear about it -- but turns into one of the handiest tools you can have.

Go to http://examples.oreilly.com/upt3 for more information on: xargs

If your system doesn't already have xargs, be sure to install it from the web site.

xargs reads a group of arguments from its standard input, then runs a Unix command with that group of arguments. It keeps reading arguments and running the command until it runs out of arguments. The shell's backquotes (Section 28.14) do the same kind of thing, but they give all the arguments to the command at once. This can give you a Too many arguments error.

Here are some examples:

If you want to print most of the files in a large directory, put the output of ls into a file. Edit the file to leave just the filenames you want printed. Give the file to xargs' standard input:

< Section 43.1
```
% ls > allfiles.tmp
% vi allfiles.tmp
% xargs lpr < allfiles.tmp
```
What did that do? With lines like these in allfiles.tmp:
```
% cat allfiles.tmp
afile
application
   ...
yoyotest
zapme
```
xargs ran one or more lpr commands, each with a group of arguments, until it had read every word in the file:
```
lpr afile application ...
   ...
lpr ... yoyotest zapme
```
This has another advantage for lpr: each print job is fairly short, so you can delete one from the print queue without losing all of them.
The standard output of xargs is the standard output of the commands it runs. So, if you'd created allfiles.tmp above, but you wanted to format the files with pr (Section 45.6) first, you could type:
```
% xargs pr < allfiles.tmp | lpr
```
Then xargs would run all of these pr commands. The shell would pipe their standard outputs[89] to a single lpr command:

[89]Actually, the shell is piping the standard output of xargs. As I said above, xargs sends the standard output of commands it runs to its own standard output.
```
pr afile application ...
   ...
```

In the next example, find (Section 9.1) gets a list of all files in the directory tree. Next, we use xargs to read those filenames and run grep -l (Section 33.6) to find which files contain the word "WARNING". Next, we pipe that to a setup with pr and lpr, like the one in the previous example:
```
% find . -type f -print | xargs grep -l WARNING | xargs pr | lpr
```
"Huh?" you might say. Just take that step by step. The output of find is a list of filenames, like ./afile ./bfile ... ./adir/zfile and so on. The first xargs gives those filenames to one or more grep -l commands:
```
grep -l WARNING ./afile ./bfile ...
   ...
grep -l WARNING ./adir/zfile ...
```
The standard output of all those greps is a (shortened) list of filenames that match. That's piped to another xargs -- it runs pr commands with the filenames that grep found.

Unix is weird and wonderful!
Sometimes you don't want xargs to run its command with as many arguments as it can fit on the command line. The -n option sets the maximum number of arguments xargs will give to each command. Another handy option, -p, prompts you before running each command.

Here's a directory full of files with errors (whose names end with .bad) and corrected versions (named .fixed). I use ls to give the list of files to xargs; it reads two filenames at once, then asks whether I want to run diff -c to compare those two files. It keeps prompting me and running diff -c until it runs out of file pairs:
```
% ls
chap1.bad
chap1.fixed
chap2.bad
chap2.fixed
    ...
chap9.bad
chap9.fixed
% ls | xargs -p -n2 diff -c
diff -c chap1.bad chap1.fixed ?...y
   ...Output of diff command for chap1...
diff -c chap2.bad chap2.fixed ?...n
diff -c chap3.bad chap3.fixed ?...y
   ...Output of diff command for chap3...
```

--JP and SJC


28.16. Separating Commands with Semicolons		28.18. Expect