Search code examples
pythonglobshutil

Find and copy files with glob and shutil

I have the following folder structure

enter image description here

Inside the folders the folders, the files in there are like in this example

enter image description here

I am trying to use a the following to parse through the folders but its not returning anything and not spitting any error.

import glob
import shutil

filenames_i_want = ['70631','70632','test']
tra=['7063']
TargetFolder = r'C:\ELK\LOGS\ATH\DEST'
all_files = []
for directory in tra:
    files=glob.glob('C:\ELK\LOGS\ATH\{}\*\_data\*.txt'.format(directory))
    all_files.append(files)
for file in all_files:
    if file in filenames_i_want:
        shutil.copy2(file, TargetFolder)
        print("found")

The above is not working and not spitting any errors. Also, since the date folders are many, the more efficient way is if i can provide an array with the date e.g ['2021-07-19','2021-07-20','2021-07-21']. How can i pass this to the glob instead of using the * which would mean going through many folders

Solution

  • The glob.glob() returns a list of file paths per the documentation. As a result, none of the those file paths will match what you have configured in filenames_i_want.

    Below is an example based loosely on your code:

    import glob

target_files=["t2.json" "test.json"]
dir="/tmp"
all_files=glob.glob("/tmp/*.json")
for f in all_files:
    if f in target_files:
        print(f"found target file: {f}")
    else:
        print(f"NOT a target file: {f}")

    The output from a test run:

    NOT a target file: /tmp/test.json
NOT a target file: /tmp/test2.json
NOT a target file: /tmp/t2.json
NOT a target file: /tmp/test3.json

    You can try modifying your code to use only the filename in the check against the filenames_i_want list:

    import glob
from pathlib import Path

target_files=["t2.json", "test.json"]
dir="/tmp"
all_files=glob.glob("/tmp/*.json")
for f in all_files:
    current_path=Path(f)
    if current_path.name in target_files:
        print(f"found target file: {f}")
    else:
        print(f"X ---- NOT a target file: {f}")

    Output from the modified code:

    found target file: /tmp/test.json
X ---- NOT a target file: /tmp/test2.json
found target file: /tmp/t2.json
X ---- NOT a target file: /tmp/test3.json