Search code examples
arraysbashcsvwhile-loopprocess-substitution

read csv output into an array and process the variable in a loop using bash


assuming i have an output/file

1,a,info
2,b,inf
3,c,in

I want to run a while loop with read

while read r ; do 
   echo "$r";
   # extract line to $arr as array separated by ',' 
   # call some program (e.g. md5sum, echo ...) on one item of arr
done <<HEREDOC
1,a,info
2,b,inf
3,c,in   
HEREDOC

I would like to use readarray and while, but compelling alternatives are welcome too.

There is a specific way to have readarray (mapfile) behave correctly with process substitution, but i keep forgetting it. this is intended as a Q&A so an explanation would be nice


Solution

  • Since compelling alternatives are welcome too and assuming you're just trying to populate arr one line at a time:

    $ cat tst.sh
    #!/usr/bin/env bash
    
    while IFS=',' read -a arr ; do
        # extract line to $arr as array separated by ','
        # echo the first item of arr
        echo "${arr[0]}"
    done <<HEREDOC
    1,a,info
    2,b,inf
    3,c,in
    HEREDOC
    

    $ ./tst.sh
    1
    2
    3
    

    or if you also need each whole input line in a separate variable r:

    $ cat tst.sh
    #!/usr/bin/env bash
    
    while IFS= read -r r ; do
        # extract line to $arr as array separated by ','
        # echo the first item of arr
        IFS=',' read -r -a arr <<< "$r"
        echo "${arr[0]}"
    done <<HEREDOC
    1,a,info
    2,b,inf
    3,c,in
    HEREDOC
    

    $ ./tst.sh
    1
    2
    3
    

    but bear in mind why-is-using-a-shell-loop-to-process-text-considered-bad-practice anyway.