I'm writing a script to process a data file, and started with these two command lines:
proc(){ echo $1 ; }
while read -r line ; do proc "$line"; done <data.csv | less
To my surprise, I found a listing of the current directory in the middle of the output, and realised it appears where the data file contains an asterisk character.
The expansion occurs before proc is called. Could someone please explain why this is happening. Is it happening in the command line processor or read or the file stream?
I can do this:
cat data.csv | tr '*' '_' | while read -r line ; do proc "$line"; done | less
but I'd like to know if there's a way to prevent this expansion.
Alternatively, is there a way to filter input using the redirection form, rather than the pipe, and what other special characters, eg: [,],?,- would I need to replace?
Cheers, bitrat
PS: An example data file:
gfh fg hfgh fgh fg hgh
7 4 674 547767 56 7 56756
ghdghh gh fg h fgh fg hf gh
8 678 678 * 67 867 8 678
gfh fg hfgh fgh fg hgh
7 4 674 547767 56 7 56756
ghdghh gh fg h fgh fg hf gh
8 678 678 67 867 8 678
And example output:
gfh fg hfgh fgh fg hgh
7 4 674 547767 56 7 56756
ghdghh gh fg h fgh fg hf gh
8 678 678 data.csv file1.txt file2.txt file3.txt 67 867 8 678
gfh fg hfgh fgh fg hgh
7 4 674 547767 56 7 56756
ghdghh gh fg h fgh fg hf gh
8 678 678 67 867 8 678
UPDATE:
The asterisk is only treated as a wildcard if it has spaces on either side, so must be getting tokenised at some point.
UPDATE2:
The expansion doesn't occur if echo is called directly..
while read -r line ; do echo "$line"; done <data.csv
Why: Pathname expansion happens later than variable expansion. Search for "order of expansions" in man bash.
The expansion does not occur before proc is called, it happens inside proc.
Solution: Add double quotes:
echo "$1"