Search code examples
pythonpython-requestsos.walkpathlib

Removing python path hardcode and passing list


**Update 1/8/2019 0945 EST

I have passed the script through the function given by bhakta0007 but received a path error "The system cannot find the path specified:".

After review, I added the below statement to the end of the script to pass the list through the function and the code works.

for f in fList: excel_csv(fList)

I have added an answer to the question below.

I have a small script that I run to convert excel files to .csv. Currently , I have to repeat the script with the paths hardcoded in. The current paths have the exact same structure with the exceptions of a 3 digit identifier which I would like to create a list that I can call from. Below is my code. You will see I have variables that have the paths and I pass these variables where needed.I have looked into os.path, glob, and pathlib, but I can't find a good solution for the problem.

Original Code

import os
import glob
import pandas as pd
import shutil


Target_Path     =   os.path.join(os.path.dirname('//fs/Unprocessed/261/Edlog/Working/'))
Move_Path       =   os.path.join(os.path.dirname('//fs/Unprocessed/261/Edlog/ToProcess/'))
Process_Path    =   os.path.join(os.path.dirname('//fs/Unprocessed/261/Edlog/Processed/'))
os.chdir(Target_Path)

try:
    for f in glob.glob('*.xls'):
        out = f.split('.')[0]+'.csv'
        df = pd.read_excel(f,)
        df.to_csv(out, index=False)

finally:
    for f in glob.glob('*.xlsx'):
        out = f.split('.')[0]+'.csv'
        df = pd.read_excel(f,)
        df.to_csv(out, index=False)

xlsCounter = len(glob.glob1(Target_Path,"*.xls"))
xlsxCounter = len(glob.glob1(Target_Path,"*.xlsx"))
csvcounter = len(glob.glob1(Target_Path,"*.csv"))

if csvcounter == xlsCounter + xlsxCounter :
    print('Complete Convert')

else: 
    print('Failed Convert')   

for files in glob.glob('*.csv'):
    shutil.move(files, Move_Path)

for files in glob.glob('*.xls'):
    shutil.move(files, Process_Path)

for files in glob.glob('*.xlsx'):
    shutil.move(files, Process_Path)



if len(os.listdir(Target_Path) ) == 0:
    print('Complete Move')

else: 
    print('Failed Move')

Solution

  • import os
    import glob
    import pandas as pd
    import shutil
    
    def process(folders):
        for f in folders:
            Target_Path     =   os.path.join(os.path.dirname('//fs/Unprocessed/{}/Edlog/Working/').format(folder))
            Move_Path       =   os.path.join(os.path.dirname('//fs/Unprocessed/{}/Edlog/ToProcess/').format(folder))
            Process_Path    =   os.path.join(os.path.dirname('//fs/Unprocessed/{}/Edlog/Processed/').format(folder))
            os.chdir(Target_Path)
    
            <Rest of our code>
    
    fList = [261, 262, 278, 300]
    process(fList)