How to use a loop to chain piped commands together?

Question

I would like to write a function with a loop to construct the command:

cat example.txt | sed ′s/A/1//' | sed ′s/B/2//' | sed ′s/C/3//' | sed ′s/D/4//' ...

while taking in a string of A B C D.

example.txt

A
B
C
D

Here is what I have come up with so far but I do not know the syntax to chain piped commands together. I was thinking that I could use echo to construct the string version of the command and then execute it that way but I am wondering if there is a better way to do this.

elements="A B C D"
n=1
for i in $elements ; do
   cat example.txt | sed "s/$i/$n/g"
   n=$(($n+1))
done

The output makes sense given the commands that I have generated: cat example.txt | sed "s/A/1/g" cat example.txt | sed "s/B/2/g" cat example.txt | sed "s/C/3/g" cat example.txt | sed "s/D/4/g"

but as stated above I would like to "chain" pipe them together.

Answer 1

Although it's not necessary for what you are trying to do (just pass multiple commands to a single sed process), it is possible to build pipelines of commands dynamically in Bash.

This Shellcheck-clean Bash code demonstrates one way to do it:

#! /bin/bash -p

function run_sed_pipeline
{
    local pipecmd='' i
    for ((i=1; i<=$#; i++)); do
        pipecmd+="${pipecmd:+ | }sed \"s/\${$i}/$i/\""
    done
    printf 'DEBUG: EVAL: %s\n' "$pipecmd" >&2
    eval "$pipecmd"
}

run_sed_pipeline A B C D <example.txt

When the code is run it produces output:

DEBUG: EVAL: sed "s/${1}/1/" | sed "s/${2}/2/" | sed "s/${3}/3/" | sed "s/${4}/4/"
1
2
3
4

• The basic idea is to build up a pipeline of commands in a string variable and use eval to run it.
• There are serious pitfalls associated with eval and it is best avoided. See Why should eval be avoided in Bash, and what should I use instead?. Also see BashFAQ/048 (Eval command and security issues).
• I think that the code here avoids significant eval pitfalls, but I could be wrong. The main thing that the code does to avoid problems is to refrain from putting the expanded function arguments (A B C D in this example) in the string to be evaled. Instead, the only expansions in the command string are (quoted) ${1}, ${2}, ${3}, and ${4}. This ensures that embedded expansions, or quotes etc., in the function arguments will not cause problems.

Another way to create a dynamic pipeline of commands is to use a recursive function, as with this Shellcheck-clean code:

#! /bin/bash -p

function run_sed_pipeline
{
    if (( $# < 2 )); then
        cat
    else
        sed "s/$2/$1/" | run_sed_pipeline "$(($1+1))" "${@:3}"
    fi
}

run_sed_pipeline 1 A B C D <example.txt

When the code is run it produces output:

1
2
3
4

• The first argument to the function is the number to substitute for the first (remaining) argument to be replaced. Each recursive call increments the first argument by one and removes the first of the following (remaining) arguments.
• This code features an unusual "useless use of cat" (UUoC). The last process in the pipeline is a useless cat. It's easy to avoid, but doing so makes the code a bit more complicated so I left it as it is for this (illustrative) example.