Search code examples
node.jsspotify

Persistent Spotify 429 errors - with ridiculous retry-after suggestion of 76,000s (about 21hr)


I am working on a application that uses the Spotify Web API to build and maintain playlists for the user based on a given recipe (just a JSON that represents a logic-scheme basically). Currently the application is in development mode. I use delays between each API call I make, currently about 400ms. And I also had delays of 7.5s when I got the occasional 429 error (too many requests).

Anyway, I recently made it so that all of the playlist recipes get rebuilt in an infinite loop. So the process is just always running and making API calls about every 100ms, in order to keep all of the playlists up-to-date based on the recipes. However after letting this loop run for about 10 minutes, I started persistently getting 429s even after retrying after 7.5s and longer.

Apparently the 429 responses contain a header called 'retry-after' which is how long Spotify suggests waiting before making another call (as I said, before I was just using a fixed 7.5s delay on 429s). I am seeing that the value I am receiving for 'retry-after' is on the order of about 76,000s (21 hours).

But I thought that the rate limits are enforced over a 30s window... (see https://developer.spotify.com/documentation/web-api/guides/rate-limits/) So why is my 'retry-after' header so high?

This is mostly a design philosophy question so the code itself I think is mostly irrelevant but if you'd like to take a look it's available here: https://github.com/jakefoglia/Smart-Playlist-Manager

site/SPM-core/maintainer.js : contains the 'infinite loop'

site/SPM-core/spotify_api_hook.js : contains most of the API calls


Solution

  • The 30s window is presented in the documentation only as an example, not as an actual way in which the API works. As you correctly say, Retry-After header (value is seconds) is all the information you need to decide how long to wait before doing the next call.

    Each time your app "violates" the rate limit by making an early request, it gets "punished" by an increased delay period, — and since the app apparently never even consulted the header, and repeatedly violated the limit, the delay got this high. This however did not result in shutdown, or blocking, or rejection, or something similar, because the header only suggests the duration of a delay, rather than enforcing it.