Pandas Value-Error: "time data 'nan' does not match format", when using "read_csv" with "date_parser" and "comment"

Question

I am using: Python 3.7.2 & Pandas 0.24.2 And I try to read the following data (data.txt). Separated by whitespace, first column should be parsed as datetime objects:

       #00:00:00               col0       col1
       2019-03-28_08:58:00     1064      31965
       2019-03-28_09:08:00     1084      32565
       !2019-03-28_09:18:00    1104      33165
       2019-03-28_09:28:00     1124      33765

with pandas read_csv as:

import pandas as pd
import datetime 

def date_parser (s):
    return datetime.datetime.strptime(str(s),'%Y-%m-%d_%H:%M:%S')

df      = pd.read_csv(filepath_or_buffer='data.txt',
                      delim_whitespace = True,
                      index_col='#00:00:00',
                      parse_dates=True,
                      date_parser=date_parser,
                      comment='!',
                      )

All lines starting with a special character (here: !) should be skipped. It can be any other charakter. But with the commented line I always receive the error:

ValueError: time data 'nan' does not match format '%Y-%m-%d_%H:%M:%S'

I am thankful for any ideas

Answer 1

The example code you have provided is working fine for me. I'm using the same Pandas version as you and Python 3.7:

It's working...

I removed redundant whitespace from the input file you provided:

#00:00:00 col0 col1
2019-03-28_08:58:00 1064 31965
2019-03-28_09:08:00 1084 32565
!2019-03-28_09:18:00 1104 33165
2019-03-28_09:28:00 1124 33765