How to delete line from CSV as they are read in Bash

Question

I am currently doing this:

while read l
do
  echo git add $l/
  git add $l/
  # sed -i -e '1,1d' data/commit-folders.csv
  # echo git commit -am "'Autocommit'"
  # git commit -uno -am "'Autocommit'"
  # echo git push origin master
  # git push origin master
done < data/commit-folders.csv

Essentially just git add <folder> for a list of folders in a CSV file. I would like for this to be more robust, in that every time it restarts it restarts from where it left off. So I added that commented out line which does an in-place delete sed -i -e '1,1d' data/commit-folders.csv. However, with while read line, it messes up with the current line if they are being deleted. So I'm wondering how to do this properly.

How to iterate through a CSV file with <path> on each line, and delete the path once it is successfully git added. It seems like you need to have a loop that selects the first line from a file, and then deletes it from the file afterwards, rather than using while read line.

Answer 1

Here a solution without sed.

#!/bin/bash
csv="data/commit-folders.csv"
done="$(mktemp)"

# autoremove tempfile at exit
trap 'rm "$done"' EXIT

# loop over all lines in csv
while read -r file; do
   printf "git add %s\n" "$file"
   git add "$file"

   # write processed files in tempfile
   printf "%s\n" "$file" >> "$done"

   #...
done < "$csv"

# create tempfile for merge result
newfile="$(mktemp)"

# sort: merge and sort $cvs with $done
# uniq -u: write only unique files into tempfile
sort "$csv" "$done" | uniq -u > "$newfile" 

# override $csv with tempfile
mv "$newfile" "$csv"