Python subprocess script failing

Question

Have written the below script to delete files in a folder not matching the dates in the "keep" period. Eg. Delete all except files partly matching this name.

The command works from the shell but fails with the subprocess call.

/bin/rm /home/backups/!(*"20170920"*|*"20170919"*|*"20170918"*|*"20170917"*|*"20170916"*|*"20170915"*|*"20170914"*)

#!/usr/bin/env python
from datetime import datetime
from datetime import timedelta
import subprocess

### Editable Variables
keepdays=7
location="/home/backups"

count=0
date_string=''
for count in range(0,keepdays):
    if(date_string!=""):
        date_string+="|"
    keepdate = (datetime.now() - timedelta(days=count)).strftime("%Y%m%d")
    date_string+="*\""+keepdate+"\"*"

full_cmd="/bin/rm "+location+"/!("+date_string+")"
subprocess.call([full_cmd], shell=True)

This is what the script returns:

#./test.py

/bin/rm /home/backups/!(*"20170920"*|*"20170919"*|*"20170918"*|*"20170917"*|*"20170916"*|*"20170915"*|*"20170914"*)
/bin/sh: 1: Syntax error: "(" unexpected

Python version is Python 2.7.12

Answer 1

Just as @hjpotter said, subprocess will use /bin/sh as default shell, which doesn't support the kind of globbing you want to do. See official documentation. You can change that using the executable parameter to subprocess.call() with a more appropriate shell (/bin/bash or /bin/zsh for example): subprocess.call([full_cmd], executable="/bin/bash", shell=True)

BUT you can be a lot better served by Python itself, you don't need to call a subprocess to delete a file:

#!/usr/bin/env python
from datetime import datetime
from datetime import timedelta
import re
import os
import os.path

### Editable Variables
keepdays=7
location="/home/backups"

now = datetime.now()
keeppatterns = set((now - timedelta(days=count)).strftime("%Y%m%d") for count in range(0, keepdays))

for filename in os.listdir(location):
    dates = set(re.findall(r"\d{8}", filename))
    if not dates or dates.isdisjoint(keeppatterns):
        abs_path = os.path.join(location, filename)
        print("I am about to remove", abs_path)
        # uncomment the line below when you are sure it won't delete any valuable file
        #os.path.delete(abs_path)