Invalidate browser cache for static files in Django

Question

In Django we have ManifestStaticFilesStorage for caching static files, but it works between Django and browser, but I want right cache between user and browser.

I want: every time static file is changed, hash of file is recalculated and browser cache is invalidated and user see new static file without F5 adn without running --collectstatic --no-input.

My code now isn't working: settings.py

STATICFILES_STORAGE = 'auth.utils.HashPathStaticFilesStorage'

CACHES = {
    'staticfiles': {
        'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
        'LOCATION': 'staticfiles',
        'TIMEOUT': 3600 * 24 * 7,
        'MAX_ENTRIES': 100,
    }
}

and auth.utils.py:

# -*- coding: utf-8 -*-

import time

from hashlib import sha384

from django.conf import settings
from django.core.cache import cache
from django.contrib.staticfiles.storage import ManifestStaticFilesStorage


try:
    ACCURACY = settings.STATICFILES_HASH_ACCURACY
except AttributeError:
    ACCURACY = 12

try:
    KEY_PREFIX = settings.STATICFILES_HASH_KEY_PREFIX
except AttributeError:
    KEY_PREFIX = 'staticfiles_hash'

class HashPathStaticFilesStorage(ManifestStaticFilesStorage):
    """A static file storage that returns a unique url based on the contents
    of the file. When a static file is changed the url will also change,
    forcing all browsers to download the new version of the file.

    The uniqueness of the url is a GET parameter added to the end of it. It
    contains the first 12 characters of the SHA3 sum of the contents of the
    file.

    Example: {% static "image.jpg" %} -> /static/image.jpg?4e1243

    The accuracy of the hash (number of characters used) can be set in
    settings.py with STATICFILES_HASH_ACCURACY. Setting this value too low
    (1 or 2) can cause different files to get the same hash and is not
    recommended. SHA3 hashes are 40 characters long so all accuracy values
    above 40 have the same effect as 40.

    The values can be cached for faster performance. All keys in the cache have
    the prefix specified in STATICFILES_HASH_KEY_PREFIX in setings.py. This
    value defaults to 'staticfiles_hash'
    """

    @property
    def prefix_key(self):
        return "%s:%s" % (KEY_PREFIX, 'prefix')

    def invalidate_cache(self, nocache=False):
        """Invalidates the cache. Run this when one or more static files change.
        If called with nocache=True the cache will not be used.
        """
        value = int(time.time())
        if nocache:
            value = None
        cache.set(self.prefix_key, value)

    def get_cache_key(self, name):
        hash_prefix = cache.get(self.prefix_key)
        if not hash_prefix:
            return None
        key = "%s:%s:%s" % (KEY_PREFIX, hash_prefix, name)
        return key

    def set_cached_hash(self, name, the_hash):
        key = self.get_cache_key(name)
        if key:
            cache.set(key, the_hash)

    def get_cached_hash(self, name):
        key = self.get_cache_key(name)
        if not key:
            return None
        the_hash = cache.get(key)
        return the_hash

    def calculate_hash(self, name):
        path = self.path(name)
        try:
            the_file = open(path, 'rb')
            the_hash = sha384(the_file.read()).hexdigest()[:ACCURACY]
            the_file.close()
        except IOError:
            return ""
        return the_hash

    def get_hash(self, name):
        the_hash = self.get_cached_hash(name)
        if the_hash:
            return the_hash
        the_hash = self.calculate_hash(name)
        self.set_cached_hash(name, the_hash)
        return the_hash

    def url(self, name):
        base_url = super(HashPathStaticFilesStorage, self).url(name)
        the_hash = self.get_hash(name)
        if "?" in base_url:
            return "%s&%s" % (base_url, the_hash)
        return "%s?%s" % (base_url, the_hash)

Answer 1

A common and simple approach to avoid users having to reload the page to get fresh static content is to append some mutable value in the inclusion of the static files in the HTML markup, something like this:

<script src="{% static 'js/library.js' %}?{{ version }}"></script>

In this way when the variable version assumes a different value, the browser is forced to download a new version of the static files from the server.

You can set version using a custom context processor, for instance reading the project version from settings. Something like this:

from django.conf import settings

def version(request):
    return {
        'version': settings.VERSION
    }

If you are using git as VCS, another approach would be writing the last commit hash of your project in a file, when you push your modifications to the server. The file should be in a format that is readable by Python. In this way you can use the git commit hash as the version variable mentioned before. You can do this using a GIT post-receive hook:

#!/bin/bash

WORKDIR=/path/to/project/
VERSION_MODULE=${WORKDIR}django_project/project/version.py

# for every branch which has been pushed
while read oldrev newrev ref
do
    # if branch pushed is master, update version.py file in the django project
    if [[ $ref =~ .*/master$ ]]; then
        GIT_WORK_TREE=$WORKDIR git checkout -f master
        echo "GIT_REF = 'master'" > $VERSION_MODULE
        echo "GIT_REV = '$newrev'" >> $VERSION_MODULE
    fi
done

Then your context processor could be:

from project.version import GIT_REV

def version(request):
    return {
        'version': GIT_REV[:7]
    }