oreilly.comSafari Books Online.Conferences.


Unix Power Tools
Handle Too-Long Command Lines with xargs

by Jerry Peek

xargs is one of those UNIX utilities that seems pretty useless when you first hear about it -- but turns into one of the handiest tools you can have.

xargs reads a group of arguments from its standard input, then runs a UNIX command with that group of arguments. It keeps reading arguments and running the command until it runs out of arguments. The shell's backquotes do the same kind of thing, but they give all the arguments to the command at once. This can give you a Too many arguments error.

Here are a couple of examples:

  • If you want to print most of the files in a large directory, put the output of ls into a file. Edit the file to leave just the filenames you want printed. Give the file to xargs' standard input:

    % ls > allfiles.tmp
    % vi allfiles.tmp
    % xargs lpr < allfiles.tmp

    What did that do? With lines like these in allfiles.tmp:

    % cat allfiles.tmp

    xargs ran one or more lpr commands, each with a group of arguments, until it had read every word in the file:

    lpr afile application ...
    lpr ... yoyotest zapme

  • The standard output of xargs is the standard output of the commands it runs. So, if you'd created allfiles.tmp above but you wanted to format the files with pr first, you could type:

    % xargs pr < allfiles.tmp | lpr

    Then xargs would run all of these pr commands. The shell would pipe their standard outputs to a single lpr command. Actually, the shell is piping the standard output of xargs. As I said above, xargs sends the standard output of commands it runs to its own standard output:

    pr afile application ...
    pr ... yoyotest zapme

    In this next example, find gets a list of all files in the directory tree. Next, we use xargs to read those filenames and run grep - l to find which files contain the word "WARNING". Next, we pipe that to a setup with pr and lpr, like the one in the previous example:

    % find . -type f -print | xargs grep -l WARNING | xargs pr | lpr

    "Huh?" you might say. Just take that step by step. The output of find is a list of filenames, like ./afile ./bfile ... ./adir/zfile and so on. The first xargs gives those filenames to one or more grep - l commands:

    grep -l WARNING ./afile ./bfile ...
    grep -l WARNING ./adir/zfile ...

    The standard output of all those greps is a (shortened) list of filenames that match. That's piped to another xargs - it runs pr commands with the filenames that grep found.

    UNIX is weird and wonderful!

  • Sometimes you don't want xargs to run its command with as many arguments as it can fit on the command line. The - n option sets the maximum number of arguments xargs will give to each command. Another handy option, - p, prompts you before running each command.

    Here's a directory full of files with errors (whose names end with .bad) and corrected versions (named .fixed). I use ls to give the list of files to xargs; it reads two filenames at once, then asks whether I want to run diff - c to compare those two files. It keeps prompting me and running diff - c until it runs out of file pairs:

    % ls
    % ls | xargs -p -n2 diff -c
    diff -c chap1.bad chap1.fixed ?...y
    diff -c chap2.bad chap2.fixed ?...n
    diff -c chap3.bad chap3.fixed ?...y

As tomorrow's article explains, xargs can have trouble if an argument has white space inside a word. Luckily, the GNU xargs solves the problem.

Back More Unix Power Tools


Linux Online Certification

Linux/Unix System Administration Certificate Series
Linux/Unix System Administration Certificate Series — This course series targets both beginning and intermediate Linux/Unix users who want to acquire advanced system administration skills, and to back those skills up with a Certificate from the University of Illinois Office of Continuing Education.

Enroll today!

Linux Resources
  • Linux Online
  • The Linux FAQ
  • Linux Kernel Archives
  • Kernel Traffic

  • Sponsored by: