Search code examples
pythonpandascsvdictionaryseries

How to remove Freq: MS, Name: des, dtype: int64 from pandas series?


I have some pandas series like result_ses. I wanted to accumulate all the data into a dictionary and save the data in a csv. I am using Google Colab to work. But I am facing trouble removing some of the unnecessary information from the data. My code is following:

asd = {}

for prod in unique_products[:4]:
    asd[prod] = {}  # empty dictionary for each product
    asd[prod]['ses'] = result_ses 
    asd[prod]['des'] = result_des 

print(asd)

The output is following:

{'2-28-437': {'ses': 2021-05-01    16
                     2021-06-01    16
                     2021-07-01    16
                     Freq: MS, Name: ses, dtype: int64, 
              'des': 2021-05-01    14
                     2021-06-01    14
                     2021-07-01    13
                     Freq: MS, Name: des, dtype: int64}, 
  '2-2-329': {'ses': 2021-05-01    16
                     2021-06-01    16
                     2021-07-01    16
                     Freq: MS, Name: ses, dtype: int64, 
              'des': 2021-05-01    14
                     2021-06-01    14
                     2021-07-01    13
                     Freq: MS, Name: des, dtype: int64}, 
  '24-30-42-7400': {'ses': 2021-05-01    16
                           2021-06-01    16
                           2021-07-01    16
                           Freq: MS, Name: ses, dtype: int64, 
                    'des': 2021-05-01    14
                           2021-06-01    14
                           2021-07-01    13
                           Freq: MS, Name: des, dtype: int64}, 
  '2-53-1151': {'ses': 2021-05-01    16
                       2021-06-01    16
                       2021-07-01    16
                       Freq: MS, Name: ses, dtype: int64, 
                'des': 2021-05-01    14
                       2021-06-01    14
                       2021-07-01    13
                       Freq: MS, Name: des, dtype: int64}}

Where both the result_ses and result_des are pandas series and unique_products is a list of string.

# if I type
result_ses.info() 
# I get 
<class 'pandas.core.series.Series'>
DatetimeIndex: 3 entries, 2021-05-01 to 2021-07-01
Freq: MS
Series name: ses
Non-Null Count  Dtype
--------------  -----
3 non-null      int64
dtypes: int64(1)
memory usage: 48.0 bytes

To view the contents of the result_ses I type print(result_ses) and get:

2021-05-01    16
2021-06-01    16
2021-07-01    16
Freq: MS, Name: ses, dtype: int64 # I do not want this included in the csv

I do not want the dictionary asd to include this two extra information specifically Freq: MS, Name: des, dtype: int64 and I want only the rest as it is so that I can get the desired output in the csv. Using the following code, I tried to save the data in the csv but it is not in the format I want.

op_path = '/content/output/'
output_file_path = op_path + f'desired_output.csv'
ddf = pd.DataFrame.from_dict(asd, orient='index')
ddf.to_csv(output_file_path, index_label='Date')

I am looking forward to getting the final output to be a csv like the following. How can I fix this problem? desired output image in csv


Solution

  • You can create columns by DatetimeIndex and values for ses and des:

    asd = {}
    for prod in unique_products[:4]:
        asd[prod] = pd.DataFrame({'ses_date':result_ses.index, 
                                  'ses_val':result_ses.to_numpy(),
                                  'des_date':result_des.index, 
                                  'des_val':result_des.to_numpy()})
    df = pd.concat(asd)
    

    For separate values by space use:

    asd = {}
    for prod in unique_products[:4]:
        ses = (result_ses.index.astype(str) +' ' + result_ses.astype(str)).to_numpy()
        des = (result_des.index.astype(str)  +' ' + result_des.astype(str)).to_numpy()
        asd[prod] = pd.DataFrame({'ses':ses, 'des':des})
    
    df = pd.concat(asd)
    

    Testing and solution working nice:

    result_ses = pd.Series([1,2,3], pd.date_range('2000-01-01', periods=3))
    result_des = pd.Series([8,9,7], pd.date_range('2000-02-01', periods=3))
    
    unique_products = ['2-28-437','2-2-329', '24-30-42-7400', '2-53-1151']
    
    asd = {}
    for prod in unique_products[:4]:
        ses = (result_ses.index.astype(str) +' ' + result_ses.astype(str)).to_numpy()
        des = (result_des.index.astype(str)  +' ' + result_des.astype(str)).to_numpy()
        asd[prod] = pd.DataFrame({'ses':ses, 'des':des})
    
    df = pd.concat(asd)
    
    df.to_csv(f'modified_output.csv')
    
    ,,ses,des
    2-28-437,0,2000-01-01 1,2000-02-01 8
    2-28-437,1,2000-01-02 2,2000-02-02 9
    2-28-437,2,2000-01-03 3,2000-02-03 7
    2-2-329,0,2000-01-01 1,2000-02-01 8
    2-2-329,1,2000-01-02 2,2000-02-02 9
    2-2-329,2,2000-01-03 3,2000-02-03 7
    24-30-42-7400,0,2000-01-01 1,2000-02-01 8
    24-30-42-7400,1,2000-01-02 2,2000-02-02 9
    24-30-42-7400,2,2000-01-03 3,2000-02-03 7
    2-53-1151,0,2000-01-01 1,2000-02-01 8
    2-53-1151,1,2000-01-02 2,2000-02-02 9
    2-53-1151,2,2000-01-03 3,2000-02-03 7
    

    If need format with separate lines in Excel:

    result_ses = pd.Series([1,2,3], pd.date_range('2000-01-01', periods=3))
    result_des = pd.Series([8,9,7], pd.date_range('2000-02-01', periods=3))
    unique_products = ['2-28-437','2-2-329', '24-30-42-7400', '2-53-1151']
    
    
    asd = {}
    for prod in unique_products[:4]:
        ses = (result_ses.index.astype(str) +' ' + result_ses.astype(str)).to_numpy()
        des = (result_des.index.astype(str)  +' ' + result_des.astype(str)).to_numpy()
        asd[prod] = pd.DataFrame({'ses':ses, 'des':des})
    
    df = pd.concat(asd).groupby(level=0).agg('\n'.join).rename_axis('Date').reset_index()
    
    with pd.ExcelWriter('modified_output.xlsx', engine='xlsxwriter') as writer:
        df.to_excel(writer, sheet_name='Sheet1', index=False)
        workbook  = writer.book
        worksheet = writer.sheets['Sheet1']
        cell_format = workbook.add_format({'text_wrap': True})
        worksheet.set_column('A:Z', cell_format=cell_format)