Search code examples
python-3.xglob

Python + glob: get a list of specific files from the root of the project directory, while the script location is in one of the subdirectory


please note im a beginner in python, apologies if this is redundant.

So, in my current repo, I have the following folder structure:

repo-name
  ->folder1
     ->subfolder1
        ->script.py
  ->some-naming-convention-folder2
      ->subfolder2
         ->subfolder21
             ->subfolder211
                 ->file.txt
  -> more such "some-naming-convention-folders" here... 

As you might have guessed it, I am running my script.py from the location path-to-my-repo/folder1/subfolder1, and I want to fetch all such *.txt files from all the other directories except mine.

The reason being if you observe this some-naming-convention-folder2 are the only types of folders i want to focus on, and also since all such folders will be having these files.

Also, this script will be running on jenkins as a job.

So, here is what i was doing all_my_files = glob.glob("/full-path-here-to-my-repo/**/*.txt", recursive=True)

I am able to get all such files ONLY when I do this via the python interpreter.

However, when I add the logic such as all_my_files = glob.glob("/just-my-repo-name/**/file.txt", recursive=True) all I get is an empty list.

I even tried

all_my_files = glob.glob("../**/file.txt", recursive=True) --> returns empty list

all_my_files = glob.glob("./just-my-repo-name/*.txt", recursive=True) --> returns empty list

Can someone please suggest me what needs to be done, or do I also need to make use of pathlib or os.walk()?


Solution

  • im such a dumbo, I wasn't aware what the ".." in a glob.glob("../directory/file.extension") meant! So, those tell "glob" hey go "UP" to my first parent directory, and then look for file.extension under that, recursively.

    And, here I was under the assumption that if you see a ".." in a path, it meant to me "go DOWN" one level.Aaargh!

    And, in my case since since I am 2 folders deep, I tried doing this

    ../../**/file.txt and I am now able to list all the txt files!