Search code examples
pythonpython-requestssdknoaa

Python requests not pulling correct date range of data


I've been using requests to get station data for a few years now. I'm using the following code:

url = 'https://www.ncdc.noaa.gov/cdo-web/api/v2/data?datasetid=GHCND&datatypeid=TMAX&limit=1000&stationid=GHCND:{StationID}&startdate={'2020-01-01'}&enddate={'2020-12-31'}
        dates_temp, temps = [], []
        r = requests.get(url, headers={'token': 'My_token'})
        d = json.loads(r.text)
        try:
            data_temps = [item for item in d['results'] if item['datatype'] == 'TMAX']
            dates_temp += [item['date'] for item in data_temps]
            temps += [item['value'] for item in data_temps]

It is still mostly working. However, no matter what station I try or what year, it will not pull January/February. The first row of data coming back is always either 2/28 or 3/1. This was not the case a few years ago. I checked the NOAA interface (downloading a report directly) and the data is there. I'm also testing it with stations which I've used before and which I know should have data.

What's the deal? Was there an update to the NOAA feed through requests? Is there a v3 of the API link? I'm reluctant to entirely rewrite all my code to go the SDK library route - partly because I already have this code that's been pretty stable (and I'm hoping I just need to fix something small), but also partly because I used to use the RNOAA library with R and there was significantly less data that would come through the API than I could get with requests.


Solution

  • I had the same problem a few days ago and I reported the failure to NOAA's help email. They have answered that V2 API is deprecated and the documentation for the new one is located in the "Tools for Developers" section of this web: https://www.ncei.noaa.gov/access

    They have also let me know this:

    "In the new API, you must download data by reporting station (no spatial queries are permitted for data access, only for metadata discovery)."

    Hope it helps.