Search code examples
djangodjango-oscar

django-oscar-api conflict with Django's UpdateCacheMiddleware: {TypeError}cannot pickle '_io.BufferedReader' object

I implemented django's per site cache in a django-oscar project with django-oscar-api. I use LocMemCache as a cache backend.

Before adding django.middleware.cache.UpdateCacheMiddleware to my middlewares, everything worked fine and I could make a request to the "basket" endpoint like this and it returned a 200 response:

import requests

session = session.Session()
r_basket = session.get("http://127.0.0.1:8000/api/basket/")

After adding the per-site caching the response has the status code 500. I debugged it and it fails in the UpdateCacheMiddleware with the following error: {TypeError}cannot pickle '_io.BufferedReader' object.

Other endpoints seem to work fine, but I haven't tested them all yet. The error can also be reproduced on a freshly installed django-oscar sandbox. I pip-installed the django-oscar-api and added it to the installed apps, and the MIDDLEWARE setting looks like this:

MIDDLEWARE = [
    'debug_toolbar.middleware.DebugToolbarMiddleware',

    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',

    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',

    # Allow languages to be selected
    'django.middleware.locale.LocaleMiddleware',
    'django.middleware.http.ConditionalGetMiddleware',
    'django.middleware.common.CommonMiddleware',

    # per site caching
    'django.middleware.cache.UpdateCacheMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.cache.FetchFromCacheMiddleware',

    # Ensure a valid basket is added to the request instance for every request
    'oscar.apps.basket.middleware.BasketMiddleware',
]

My package versions are Django v.3.2.9, django-oscar v.2.2 or 3.0, django-oscar-api v.2.1.1.

What object can not be pickled? Or is the order of the middlewares possibly wrong?

Traceback:

[2021-11-13 08:43:33,256] Internal Server Error: /api/basket/
Traceback (most recent call last):
  File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/utils/deprecation.py", line 119, in __call__
    response = self.process_response(request, response)
  File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/middleware/cache.py", line 111, in process_response
    response.add_post_render_callback(
  File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/template/response.py", line 92, in add_post_render_callback
    callback(self)
  File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/middleware/cache.py", line 112, in <lambda>
    lambda r: self.cache.set(cache_key, r, timeout)
  File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/debug_toolbar/panels/cache.py", line 39, in wrapped
    value = method(self, *args, **kwargs)
  File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/debug_toolbar/panels/cache.py", line 94, in set
    return self.cache.set(*args, **kwargs)
  File "/home/victor/PycharmProjects/virtual_environments/django-oscar-sandbox/lib/python3.8/site-packages/django/core/cache/backends/locmem.py", line 56, in set
    pickled = pickle.dumps(value, self.pickle_protocol)
TypeError: cannot pickle '_io.BufferedReader' object

expected response

{'id': 1,
 'owner': None,
 'status': 'Open',
 'lines': 'http://127.0.0.1:8000/api/baskets/1/lines/',
 'url': 'http://127.0.0.1:8000/api/baskets/1/',
 'total_excl_tax': '0.00',
 'total_excl_tax_excl_discounts': '0.00',
 'total_incl_tax': '0.00',
 'total_incl_tax_excl_discounts': '0.00',
 'total_tax': '0.00',
 'currency': None,
 'voucher_discounts': [],
 'offer_discounts': [],
 'is_tax_known': True}

View: oscarapi.views.basket.BasketView

Solution

  • You need to ensure that UpdateCacheMiddleware is before SessionMiddleware and LocaleMiddleware in your settings. DOCS

    FetchFromCacheMiddleware needs to be after those middleware, can be left where it is

    MIDDLEWARE = [
    'debug_toolbar.middleware.DebugToolbarMiddleware',

    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',

    # Moved UpdateCacheMiddleware to here
    'django.middleware.cache.UpdateCacheMiddleware',

    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware',

    # Allow languages to be selected
    'django.middleware.locale.LocaleMiddleware',
    'django.middleware.http.ConditionalGetMiddleware',
    'django.middleware.common.CommonMiddleware',

    # per site caching
    'django.middleware.cache.FetchFromCacheMiddleware',

    # Ensure a valid basket is added to the request instance for every request
    'oscar.apps.basket.middleware.BasketMiddleware',
]