This is definitely a first for me. Using the os.listdir() method, I'm able to view files / folders from a directory that doesn't seem to exist. Below is a lightly redacted snippet from the console showing the effect:
sh-4.2$ python
Python 3.6.11 | packaged by conda-forge | (default, Aug 5 2020, 20:09:42)
[GCC 7.5.0] on linuxType "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.listdir(<full_file_path>)
['file01', 'file02', 'file03', 'file04', 'file05']
>>> exit()
sh-4.2$ ls <full_file_path>
sh-4.2$ ls -a <full_file_path>
. ..
sh-4.2$
From the graphical file explorer, I am unable to see anything in the parent folder for the files I'm searching for. Python insists that the files are real and exist, but they cannot be accessed without using python to do so. They should not be hidden, or having special permissions to be able to view them. Any help is appreciated.
This issue has been solved.
The directory that I'm looking for was created with os.makedirs(). On closer inspection, I can see that it created the filepath from os.getcwd() exactly as it was entered.
>>> import os
>>> os.getcwd()
'/ec2-user/SageMaker/path/to/current/'
>>> os.listdir('./results') # Should show me 5 different folders
[]
>>> os.listdir('./~') # Uh oh
['SageMaker']
So what happened was that the full file path was created from the original working directory, contrary to what was expected.
sh-4.2 $ ls ~/SageMaker/path/to/current/~/SageMaker/path/to/current/results
folder01 folder02 folder03 folder04 folder05
TL;DR
I did not confirm the location of the directory was being created from root as expected, and it was created in the wrong location. os.listdir() still showed the files in the "correct location" because it wasn't starting in root, but in the current working directory.