Search code examples
unixiopipe

Unix Pipes for Command Argument


I am looking for insight as to how pipes can be used to pass standard output as the arguments for other commands.

For example, consider this case:

ls | grep Hello

The structure of grep follows the pattern: grep SearchTerm PathOfFileToBeSearched. In the case I have illustrated, the word Hello is taken as the SearchTerm and the result of ls is used as the file to be searched. But what if I want to switch it around? What if I want the standard output of ls to be the SearchTerm, with the argument following grep being PathOfFileToBeSearched? In a general sense, I want to have control over which argument the pipe fills with the standard output of the previous command. Is this possible, or does it depend on how the script for the command (e.g., grep) was written?

Thank you so much for your help!


Solution

  • There are basically two options for this: shell command substitution and xargs. Brian Agnew has just written about the former. xargs is a utility which takes its stdin and turns it into arguments of a command to execute. So you could run

    ls | xargs -n1 -J % grep -- % PathOfFileToBeSearched
    

    and it would, for each file output by ls, run grep -e filename PathOfFileToBeSearched to grep for the filename output by ls within the other file you specify. This is an unusual xargs invocation; usually it's used to add one or more arguments at the end of a command, while here it should add exactly one argument in a specific place, so I've used -n and -J arguments to arrange that. The more common usage would be something like

    ls | xargs grep -- term
    

    to search all of the files output by ls for term. Although of course if you just want files in the current directory, you can this more simply without a pipeline:

    grep -- term *
    

    and likewise in your reversed arrangement,

    for filename in *; do
      grep -- "$@" PathOfFileToBeSearched
    done
    

    There's one important xargs caveat: whitespace characters in the filenames generated by ls won't be handled too well. To do that, provided you have GNU utilities, you can use find instead.

    find . -mindepth 1 -maxdepth 1 -print0 | xargs -0 -n1 -J % grep -- % PathOfFileToBeSearched
    

    to use NUL characters to separate filenames instead of whitespace