what is the best method to initialize or store a lookup dictionary that will be used in django views

Question

I'm reviving an old django 1.2 app. most of the steps have been taken. I have views in my django app that will reference a simple dictionary of only 1300ish key-value pairs.

Basically the view will query the dictionary a few hunderd to a few thousand times for user supplied values.The dictionary data may change twice a year or so.

fwiw: django served by gunicorn, db=postgres, apache as proxy, no redis available yet on the server

I thought of a few options here:

1. a table in the database that will be queried and let caching do its job (at the expense of a few hundred sql queries)

2. Simply define the dictionary in the settings file (ugly, and how many time is it read? Every time you do an 'from django.conf import settings'? This was the situation how it was coded in the django 1.2 predecessor of this app many years ago

3. read a tab delimited file using Pandas in the django settings and make this available. the advantage is that I can do some pandas magic in the view. (How efficient is this, will the file be read many times for different users or just once during server startup?)

4. prepopulate a redis cache from a file as part of the startup process (complicates things on the server side and we want it to be simple, but its fast.

5. List items in a tab delimited file and read it in in the view (my least popular option since it seems to be rather slow)

What are your thoughts on this? Any other options?

Answer 1

Let me give a few - simple to more involved

1. Hold it in memory
2. Basic flat file
3. Sqlite file
4. Redis
5. DB

I wouldn't bring redis in for 1300 kv pairs that don't even get mutated all that much

I would put a file alongside the code that gets slurped in memory at startup or do a single sql query and grab the entire thing at startup and keep it in memory to use throughout the application