Search code examples
bashxargs

Saving arguments passed from xargs to bash as a variable for processing

I am trying to run many commands in parallel, and need to do some string manipulation on the input first. How can I make the below example work?

find . -mindepth 1 -type d | xargs -n 1 -P 20 -i sh -c "v={}; echo $v"

When I use this, $v is null. Why is it not being saved as the value of {}?

Solution

  • The parent shell is expanding $v before the string gets passed to xargs.

    Suppose your find command finds a subdirectory named ./stuff.

    First, the parent bash shell (the one you typed the find command into) will expand $v, because the string is in double quotes. You currently have no value set for variable v, so it expands to an empty string.

    Next, the arguments get passed to xargs, which will see this: v={}; echo

    Then, xargs will read ./stuff from the pipe, and replace {} with ./stuff

    Finally, the sh command is executed by xargs, and sh will see this: v=./stuff; echo

    To fix this, you need to either escape the $ so that the parent shell doesn't expand it, or use single quotes to avoid variable expansion. You should also probably quote the strings so that any directory names with spaces in them don't cause problems with the final sh command:

    find . -mindepth 1 -type d | xargs -n 1 -P 20 -i sh -c "v=\"{}\"; echo \"\$v\""

    OR

    find . -mindepth 1 -type d | xargs -n 1 -P 20 -i sh -c 'v="{}"; echo "$v"'

    With either command, the final sh process will see: v="./stuff"; echo "$v"

    By the way, one way to see for yourself that this is indeed what is happening would be to set a value for v in the parent shell, then run your original command. The shell will expand $v to whatever value you set, and you will see that value repeated for every directory found by find.

    $ v=foobar
$ find . -mindepth 1 -type d | xargs -n 1 -P 20 -i sh -c "v={}; echo $v"
foobar
foobar
foobar
foobar
foobar
foobar
foobar
foobar
foobar
foobar
foobar
...