read all files in sub folder with pandas

Question

My notebook is in the home folder where I also have another folder "test". In the test folder, I have 5 sub folders. Each of the folder contains a .shp file. I want to iterate in all sub folders within test and open all .shp files. It doesn't matter if they get overwritten.

data = gpd.read_file("./test/folder1/file1.shp")
data.head()

How can I do so? I tried this

path = os.getcwd()
files = glob.glob(os.path.join(path + "/test/", "*.shp"))
print(files)

but this would only go in 1 layer deep.

Answer 1

you can use the os.walk method in the os library.

import os
import pandas as pd
for root, dirs, files in os.walk("./test"):
    for name in files:
        fpath = os.path.join(root, name)
        data = pd.read_file(fpath)