Search code examples
bashshellposixglobls

How can I confirm whether whitespace or special characters are escaped in a wildcard pattern?


I know that when you use a for loop in Bash, the items that you loop through are separated using the $IFS variable.

However, if I run the following commands, I correctly show the two files I have created - even though they have spaces:

touch file\ {1..2}.txt
for file in *.txt; do
    echo "Found: ${file}"
done

The output is:

Found: file 1.txt
Found: file 2.txt

I am assuming that this is because when the shell sees the wildcard pattern, it expands it and escapes any special characters or whitespace. This is in contrast to if I run:

touch file\ {1..2}.txt
files=$(ls *.txt)
for file in $files; do
    echo "Found: ${file}"
done

This results in:

Found: file
Found: 1.txt
Found: file
Found: 2.txt

Which makes sense - by default $IFS contains whitespace, so the file names are split.

What I want to understand is:

  1. Am I correct that wildcard expansion results in a set of strings that contain escaped special characters
  2. Where is it documented that this is the case, if I am correct?
  3. Is there any way to show that this is happening?

I was hoping I could use something like set -x to show what the wildcard expands to and actually see the escaped characters, because I really want to be able to understand what is going on here.

I am writing a long series of articles on effective shell usage (effective-shell.com) and I'm struggling to find a way to explain the differences of behaviour here, I'm assuming that the shell is escaping characters but I'd like to know if this is the case and how to see it if possible!

Thanks in advance. done


Solution

    1. Am I correct that wildcard expansion results in a set of strings that contain escaped special characters

    No. There is no need for the shell to escape special characters at that point, because filename expansion is the last word expansion to be performed; strings resulting from it are not subjected to word splitting or any other expansion; they stay as-is. This is documented in the manual as follows:

    The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and filename expansion.