Return status code when maximum retries attempted using backoff

Question

Using asyncio and aiohttp, I have implemented an async function that triggers an API get request whenever a new record is inserted into database. If the request is successful, the status code has to be updated in database, otherwise the request should be retried 4 times and if it still fails, then the status code has to be updated in database.

To raise the exception on 404 status code, I have added raise_for_status flag to the aiohttp client session. When exception arises, the backoff decorator will retry the API call 4 times and when it still fails, it doesn't return any status code. This is what I have done:

# backoff decorator to retry failed API calls by "max_tries"
@backoff.on_exception(backoff.expo, aiohttp.ClientResponseError, max_tries=4, logger=logger)
async def call_url(language: str, word:str, headers:dict) -> bytes:
      url = f"https://od-api.oxforddictionaries.com:443/api/v2/entries/{language}/{word.lower()}"
      print(f"Started: {url}")

      # Create aiohttp session to trigger 'get' dictionary API call with app_id, app_key as headers
      async with aiohttp.ClientSession(headers=headers) as session:   

          # raise_for_status is used to raise exception for status_codes other than 200      
          async with session.get(url, raise_for_status=True) as response:

              # Awaits response from dictionary API  
              content = await response.read()                              
              status = response.status
              print("Finished: ", status)

              # Returns API status code to be updated in db
              return status                                               

I can't add the try-except clause because once exception is raised, it's handled by the try-except clause and the backoff doesn't retry the failed API call. Is there a way to make backoff decorator return the status code after maximum retries are attempted?

Answer 1

You say that "when it still fails it doesn't return any status code." But it has to either return something or else raise an exception. How about wrapping the call_url and handling the error situation in the wrapper function? The wrapper function always returns a status code. For instance:

   async def my_call_url(language: str, word:str, headers:dict) -> bytes:
      try:
          return await call_url(language, word, headers)
      except WhateverExceptionAiohttpRaises:
          return some_status_code

If, instead of raising, the decorator returns None, you can make the appropriate changes.

Your code will now call this new function instead of the other one.