Search code examples
bashtarglob

How do bash scripts expand file globs?


In this code, the excludespec variable is expected to match all it substring, but it appears to evaluate the to the actual files matching it when it is executed, rather than pass its exact representation to the tar command using it.

excludespec=${PWD##*/}\_$USER\_`hostname`.bkcd_backup*

The end result is the archive being created does not match the exclusion list so tar outputs:

tar: .: file changed as we read it

Is the string defining it doing something I don't know about?

Here is the code:

#!/bin/sh
# bkcd - backup current directory in situ

DMY_M() {
  date +%Y%m%d_%H%M
}

timestring=$(DMY_M)
echo `pwd` - $timestring > .bkcdspec  
filename=${PWD##*/}\_$USER\_`hostname`.bkcd_backup.$timestring.tar.gz    
excludespec=${PWD##*/}\_$USER\_`hostname`.bkcd_backup*
fullexclude="$excludespec"tar.gz    
echo excludespec - $excludespec
echo filename - $filename
echo fullexclude - $fullexclude    
tar -cpzf $filename --exclude=$fullexclude .
rm .bkcdspec

Solution

  • The usual trick to suppress globbing is to enclose the string in double quotes (when you want backticks or variables to be expanded) or single quotes (when you don't). So here, you'd eliminate $excludespec and set $fullexclude:

    fullexclude="${PWD##*/}_${USER}_$(hostname).bkcd_backup*.tar.gz"
    

    and use it as:

    tar -cpzf "$filename" --exclude="$fullexclude" .
    

    (Without the double quotes around "$fullexclude", if you happened to have some files named --exclude=...value-of-$fullexclude around, you would get a surprise. You're unlikely to have such files, but you may as well head off the trouble before it arises.)

    Note that when you're echoing for debugging, it is also crucial to stop the globbing; again, double quotes:

    echo "filename=$filename"
    echo "fullexclude=$fullexclude"   
    

    Otherwise, echo does glob expansion on the names, confusing you once more.

    Putting those changes together leads to:

    #!/bin/sh
    # bkcd - backup current directory in situ
    
    timestring=$(date +%Y%m%d_%H%M)
    echo "$(pwd) - $timestring" > .bkcdspec
    prefix="${PWD##*/}_${USER}_$(hostname).bkcd_backup"
    filename="$prefix.$timestring.tar.gz"   
    fullexclude="$prefix.*.tar.gz"
    echo "filename - $filename"
    echo "fullexclude - $fullexclude"   
    tar -cpzf "$filename" --exclude="$fullexclude" .
    rm .bkcdspec