Search code examples
pythonos.walkmacos-monterey

Python os.walk reports DIR as FILE walking external NTFS drive with macos on Macbook 2021


With Macbook 2021 (arm64).

uname -a
Darwin MacBook.local 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:01 PDT 2021; root:xnu-8019.41.5~1/RELEASE_ARM64_T6000 arm64

External drive, SSD2TB, is NTFS.

diskutil info disk4
   Device Identifier:         disk4
   Device Node:               /dev/disk4
   Whole:                     Yes
   Part of Whole:             disk4
   Device / Media Name:       External

   Volume Name:               SSD2TB
   Mounted:                   Yes
   Mount Point:               /Volumes/SSD2TB

   Content (IOContent):       None
   File System Personality:   NTFS
   Type (Bundle):             ntfs
   Name (User Visible):       Windows NT File System (NTFS)

The simplest of tests, is to simply report a directory name for each cycle, e.g.

python3
Python 3.10.1 (main, Dec 31 2021, 10:22:35) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, os.path
>>> os.chdir("/Volumes/SSD2TB/Photos")
>>> from glob import glob
>>> glob('*')
['Pictures']
>>> for d,dd,f in os.walk('.'): print(f"{dd}")
... 
[]
>>> os.path.isdir('Pictures')
True
>>> for d, dd, f in os.walk('.'):
...     print(f"{f}")
... 
['.DS_Store', 'Pictures']
>>> for d, dd, f in os.walk('.'):
...     print(f"{d}")
... 
.

Does anyone understand why the sub-directory in '.' is getting reported as a file in os.walk (by getting returned in the 'f' variable? And the 'dd' variable which should be the list of directories returns an empty list.

I thought at first this was some quirk in Spyder that I was using, running python in the console, but as you can see, I get exactly the same result running python directly.

One final note. If I try the same test in a path on the local drive, then everything behaves as expected. Directories are reported in variable 'dd' and files in 'f'.


Solution

  • This is something to do with the way os.walk interfaces with Apple's NTFS handler. If you use a proprietary NTFS tool for MacOS, then os.walk works as expected.

    Check out my alternative code for pathlib to walk with Apple NTFS or any other.

    https://superuser.com/questions/1701107/macos-with-external-ntfs-default-read-only-python-os-walk-will-not-recurse-th