I am exporting a data frame to a csv file using pandas to_csv() function in pycharm. However, the exported file does not contain any delimiters. The overall goal of the script is to read two csv files (one from 2016 and one from 2021 that contain some similar information but the 2021 csv has added information), search for different rows in the csv files by using a field called 'FileName,' and write the rows that are different to a new csv file. This will allow me to see what has been added to the csv since 2016. The code works up until the final process in which I export the data frame to the final csv. I checked the output in notepad and commas do not appear.
I have tried to specify parameters to ensure the file is comma delimited, however I believe this should be the default anyway. Here is my code:
#the csv's are large, ~1000 rows so I set the display options manually
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.width', None)
pd.set_option('display.max_colwidth', -1)
# Opens csv files as dataframes
f2016 = pd.read_csv( 'G:\\Wildlife_V_ErikSimoneNico_Done.csv',)
f2021 = pd.read_csv('G:\\test_v_wildlife_output.csv')
# stores files that do not match among csv in a list using the FileName field of both csvs to
# compare csvs
diffList = [f2021[~f2021.FileName.isin(f2016.FileName)]]
#converts list to a dataframe
diffList_df = pd.DataFrame(diffList)
#converts data frame to a csv
diffList_df.to_csv('G:\\v_wildlife_diff.csv', sep=',', index=False, header=True)
If I understood your problem correctly, you want to save the lines of f2021 that have a FileName value different than the ones in f2016.
But I think you have a problem with the line
diffList = [f2021[~f2021.FileName.isin(f2016.FileName)]]
In this line, you are storing a dataframe with the new rows of f2021 in a list, and then you cast it into a DataFrame.
Instead, you should try export directly this dataframe to .csv.
#the csv's are large, ~1000 rows so I set the display options manually
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.width', None)
pd.set_option('display.max_colwidth', -1)
# Opens csv files as dataframes
f2016 = pd.read_csv( 'G:\\Wildlife_V_ErikSimoneNico_Done.csv',)
f2021 = pd.read_csv('G:\\test_v_wildlife_output.csv')
# Creates a DataFrame with rows that do not match among csvs using the FileName field of both csvs to compare csvs
diffDataFrame = f2021[~f2021.FileName.isin(f2016.FileName)]
#converts data frame to a csv
diffDataFrame.to_csv('G:\\v_wildlife_diff.csv', sep=',', index=False, header=True)