Search code examples
bashsplitio-redirectiondd

bash script to process stdin in chunks using dd and redirections


I came across an old bash script, which splits data from stdin.
When run as seq 1234 | ./test_dd_a.sh, the files temp.1 to temp.5 are created. As it progress, it shows:

processing part 1
...
processing part 5
processing part 6
extraneous temp.6 will be deleted...

Here's the code:

i=1
while true
do
    {
        echo "processing part $i" 1>&2
        dd bs=1k count=1 of="temp.$i" 2>&3
    } 3>&1 | grep -q '^0+0 ' && {
        echo "extraneous temp.$i will be deleted..."
        rm "temp.$i"
        break
    }
    i=`expr $i + 1`
done

I would like to use that as a basis for a script which copies data from stdin to stdout, something like the code below. However, when run in the same manner it does not yet give any output to stdout, but it does show progress and stops when expected.

i=1
while true
do
    {
        echo "processing part $i" 1>&2
        dd bs=1k count=1  2>&3
    } 3>&1 | grep -q '^0+0 ' && {
        break
    }
    i=`expr $i + 1`
done

I would like to get the above working with minimal changes and believe it can be done with some additional tinkering with the redirections. Assume pv, split or similar commands are not available.


Solution

  • #!/usr/bin/env bash
    i=1
    while :; do
      echo "processing part $i" 1>&2
      if { LC_ALL=POSIX dd bs=1k count=1 2>&3 >&4; } 3>&1 | grep -qe '^0[+]0 '; then
        break
      fi
      i=$(( i + 1 ))
    done 4>&1