Search code examples
linuxshellawksed

Need to Update the data of a ASOF_DATE column in a CSV file using shell script


I am trying to update the existing CSV file using shell script. There is a column called ASOF_DATE where am trying to replace the value of it with the current date. The shell script I am using is not helping me, it's returning blank values for the ASOF_DATE column; can you figure out what's wrong?

Sample Input Data:

ASOF_DATE,CUSIP,Current Face,Market Value
'04/08/2024',BENRDUZU0,-400000000
'04/08/2024',BENRDUZR7,-300000000
'04/08/2024',BENRE4H37,-225000000
'04/08/2024',BENRDUYW7,-250000000

Below is the shell script I am using:

#!/bin/bash
if [ "$#" -lt 1 ]; then
    echo "Argument is missing"
    exit 1
fi

asOnDate=$1
FILE_PATH=/awsdatalake/data/inbound/file/SRC_TPGCF/
if [ ! -f ${FILE_PATH}CFG_ANALYTICS_*.csv ]
then
    echo "File does not exist"
    exit 2
else
    echo "File found."
    for file in ${FILE_PATH}CFG_ANALYTICS_*.csv 
    do
        echo "$file"
        awk -v date="$asof_date" -F, 'BEGIN {OFS=","} {if (NR==1) {print $0} else {if (NR>1) {$1=date; print}}}' "$file" > temp_file && mv temp_file "$file"
    
        echo "Current date $asof_date added to the ASOF_DATE column in $file_path"
    done
fi

Solution

  • asof_date is just an uninitialized variable, so you are replacing the field with ... nothing. But it's easy to add the variable:

            :
            awk -v date="$(date +%F)" -F, 'BEGIN {OFS=","}
                NR>1 {$1=date } 1' "$file" > temp_file &&
            mv temp_file "$file"
            :
    

    You'll notice that I also refactored your Awk script slightly. There are many other problems with your shell script but I'll defer to https://shellcheck.net/ for diagnostics.

    If you meant to use (the otherwise unused variable) asOnDate then you might as well use $1 directly.