I need to cache lists 10MB+ size, is it safe to do it in Django filesystem cache?
Or is there a better way to do it?
I would examine carefully the Python DiskCache.
Django is Python’s most popular web framework and ships with several caching backends. Unfortunately the file-based cache in Django is essentially broken. The culling method is random and large caches repeatedly scan a cache directory which slows linearly with growth. Can you really allow it to take sixty milliseconds to store a key in a cache with a thousand items?
In Python, we can do better. And we can do it in pure-Python!