Search code examples
pythonsubprocessquotesrsync

Python: rsync exclusions not working in script, works in bash shell

Below is a script I am using to test a problem.

Running an rsync command via subprocess.check_call does not work in excluding files obtained from an exclude variable.

I am printing the result of the command from Python, editing it and then running it in a bash shell directly as a comparison and it fails to exclude my exclusions when using Python.

#!/usr/bin/env python3.1

import subprocess

exclude = 'exclude_me_dir, exclude_me_file.txt'
source_path = '/tmp/source'
path_to_backup_file_name = '/tmp/destination'
engine_options = '-axh --delete --delete-excluded'

def rsync_backup(source_path, path_to_backup_file_name, exclude, engine_options):
    exclusions = ['--exclude="%s"' % x.strip() for x in exclude.split(',')]
    rsync_command = ['rsync'] + exclusions + engine_options.split() + [source_path + '/', path_to_backup_file_name]
    print(rsync_command)
    return subprocess.check_call(rsync_command)


rsync_backup(source_path, path_to_backup_file_name, exclude, engine_options)

This is the output of the Python script and running the rsync command directly.

> pwd
/root
> ls /tmp/source/
exclude_me_dir/  exclude_me_file.txt  file1.txt  folder1/
> /tmp/rsynctest.py
['rsync', '--exclude="exclude_me_dir"', '--exclude="exclude_me_file.txt"', '-axh', '--delete', '--delete-excluded', '/tmp/source/', '/tmp/destination']
> ls /tmp/destination/
exclude_me_dir/  exclude_me_file.txt  file1.txt  folder1/
> rsync --exclude="exclude_me_dir" --exclude="exclude_me_file.txt" -axh --delete --delete-excluded /tmp/source/ /tmp/destination
> ls /tmp/destination/
file1.txt  folder1/

N.B. As I was about to post this I discovered the problem appears to be the double quotes in '--exclude="file"' as if I remove them it works. I tried escaping them like so '--exclude=\"file\"'. but this does not work either. I require the double quotes for occasions when a space occurs in a file name or directory.

What am I missing?

Solution

  • Yes the double quotes are the problem, don't escape them, just drop them.

    They are just required on the shell to hinder the shell from expanding.

    Moreover: If you escape them the way you've shown, they are only escaped on the python level where it makes no sense, because double quotes are automatically escaped inside of single quotes

    In [2]: '\"foo\"'
Out[2]: u'"foo"'

    it should be

    In [3]: '\\"foo\\"'
Out[3]: u'\\"foo\\"'