Search code examples
pythonpandasanacondapyinstallerglob

Pyinstaller exe fails execution and is slow


I used pyinstaller to pack the following script in an exe file with pyinstaller -F. For clarity the script concat csv files in one file and export them to a new csv file.

# import necessary libraries
import pandas as pd
import os
import glob
from datetime import datetime

#Set filename
file_name = 'daglig_LCR_RLI'

# in the folder
path = os.path.dirname(os.path.abspath(__file__))

# Delete CSV file 
# first check whether file exists or not
# calling remove method to delete the csv file
# in remove method you need to pass file name and type

del_file = path+"\\" + file_name +'.csv'

## If file exists, delete it ##
if os.path.isfile(del_file):
    os.remove(del_file)
    print("File deleted")
else:    ## Show an error ##
    print("File not found: " +del_file)

# use glob to get all the csv files
csv_files = glob.glob(os.path.join(path, "*.csv"))
df_list= list()

#format columns
dict_conv={'line_item': lambda x: str(x),
           'column_item': lambda x: str(x)}

# loop over the list of csv files
for f in csv_files:     
    # read the csv file
    df = pd.read_csv(f, sep=";", converters = dict_conv, encoding='latin1') #test latin1
    df_list.append(df)
    #print the location and filename
    print('Location:', f)
    print('File Name:', f.split("\\")[-1])

#add data frames to a list 
RLI_combined = pd.concat(df_list, axis=0)

#Write date to approval_text
now = datetime.now()
# dd/mm/YY
print_date = now.strftime("%d/%m/%Y")
RLI_combined.loc[:, 'approval_text'] = print_date

#replace value_text with n/a
RLI_combined.loc[:, 'value_text'] = "n/a"

#Sum columns
m = RLI_combined['column_item'].isin(['0030', '0050', '0080'])
RLI_combined_sum = RLI_combined[~m].copy()
RLI_combined_sum['amount'] = RLI_combined_sum.groupby(['report_name', 'line_item', 'column_item'])['amount'].transform('sum')
RLI_combined_sum = RLI_combined_sum.drop_duplicates(['report_name', 'line_item', 'column_item'])

RLI_combined = pd.concat([RLI_combined_sum, RLI_combined[m]])

#export to csv
RLI_combined.to_csv(path + "//" + file_name + '.csv', index=False, sep=";", encoding='latin1')

#Make log    
# Create the directory 
directory = "Log"
parent_dir = path
# Path 
path_log = os.path.join(parent_dir, directory) 

try: 
    os.mkdir(path_log)
    print('Log folder dannet') 
except OSError as error: 
    print('Log folder eksisterer')  

#export to csv
log_name = now.strftime("%d-%m-%Y_%H-%M-%S")
print(log_name)
RLI_combined.to_csv(path + "//" + 'Log' +"//" + file_name+'_' + log_name + '.csv', index=False, sep=";", encoding='latin1')

Everything works as intended when don't use pyinstaller. If I run the exe file after 10 sec of nothing it writes the following:

Error Message

What am I doing wrong that causes the error? and could I improve performance so the exe file isn't that slow.

I hope you can point me in the right direction.


Solution

  • I believe I found the solution to the problem. I use anaconda and pyinstaller uses all the installed libaries on the machine. So using a clean install of python and only installling the nescecary libaries might fix the problem. The error seems to be a numpy error and the script isn't using that libary.