Search code examples
pythonmysqltwisted

Server-side Data model: In database or loaded as object?


I'm writing a mobile application using python and mysql on the server-side. My server-side data will be read/written rather frequently through connections from the python twisted platform. Should this data be loaded into python objects (I assume faster, but may be expensive memory wise) or just maintained in the mysql database?

The stored data will be lists of strings (count capped ~200)


Solution

  • Use a memory store like redis, and then write it out to the database for later use/storage.

    redis will provide you excellent write speed (crucial as writing is a blocking operation for most databases), and fast reading because its all in-memory. Your data set is tiny, and redis will be a great boost.

    You can achieve similar results with memcache; another in-memory store.

    After a period you specify, you can then write the contents of redis to your database, using it as a "permanent store" for archived data. If data is required beyond the period that is available in redis (in other words, you have a miss when you try to load from redis) you can load it at that time from the database into redis. Every subsequent request will then pull from the (must faster) redis store.

    This is common practice for quick changing data especially where read/write is critical.