Search code examples
pythonpandaslambdastrftime

Using lambda and strftime on dates when there are null values (Pandas)


I'm trying to change the format of a datetime column in my Dataframe using lambda and strftime like below

df['Date Column'] = df['Date Column'].map(lambda x: x.strftime('%m/%d/%Y'))

However, since I have null values in some of these fields, this is giving me an error. I cannot drop these null rows because I still need them for the data in the other columns. Is there a way around this error without dropping the nulls.

Perhaps something like

df['Date Column'].map(lambda x: x.strftime('%m/%d/%Y') if x != null else "")

?

The method I've used is to drop the nulls, format the column, then merge it back onto the original dataset, but this seems like a very inefficient method.


Solution

  • You should be not checking for nan/nat (un)equality, but .notnull() should work and it does for me:

    s = pd.date_range('2000-01-01', periods=5).to_series().reset_index(drop=True)
    s[2] = None
    s
    
    0   2000-01-01
    1   2000-01-02
    2          NaT
    3   2000-01-04
    4   2000-01-05
    dtype: datetime64[ns]
    
    s.map(lambda x: x.strftime('%m/%d/%Y') if pd.notnull(x) else '')
    
    0    01/01/2000
    1    01/02/2000
    2              
    3    01/04/2000
    4    01/05/2000
    dtype: object
    

    This returns the same that the answers by @Alexander and @Batman but is more explicit. It may also be slightly slower for large series.

    Alternatively you can use the .dt accesor. The null values will be formatted as NaT.

    s.dt.strftime('%m/%d/%Y')
    
    0    01/01/2000
    1    01/02/2000
    2           NaT
    3    01/04/2000
    4    01/05/2000
    dtype: object