peewee & threadpool got OperationalError: database is locked

Question

I was using peewee and threadpool for my project, my code is something like:

pool = threadpool.ThreadPool(10)
for city in city_list:
    request = threadpool.WorkRequest(
        clean_data_process, [city])
    pool.putRequest(request)
pool.wait()

def clean_data_process(city):
    data = Item.select().where(Item.city == city)
    for item in data:
        item.score = item.level + item.status
        item.save()

Where Item here is my model.

class BaseModel(Model):
    class Meta:
        database = SqliteDatabase("test.db")
class Item(BaseModel):
    city = CharField()
    level = IntegerField()
    status = IntegerField()
    score = IntegerField()

And I got following error:

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/threadpool-1.3.2-py2.6.egg/threadpool.py", line 158, in run
    result = request.callable(*request.args, **request.kwds)
  File "/home/data_clean_convert.py", line 50, in clean_data_process
    item.save()
  File "/usr/lib/python2.6/site-packages/peewee.py", line 5166, in save
    rows = self.update(**field_dict).where(self._pk_expr()).execute()
  File "/usr/lib/python2.6/site-packages/peewee.py", line 3459, in execute
    return self.database.rows_affected(self._execute())
  File "/usr/lib/python2.6/site-packages/peewee.py", line 2940, in _execute
    return self.database.execute_sql(sql, params, self.require_commit)
  File "/usr/lib/python2.6/site-packages/peewee.py", line 3838, in execute_sql
    self.commit()
  File "/usr/lib/python2.6/site-packages/peewee.py", line 3657, in __exit__
    reraise(new_type, new_type(*exc_args), traceback)
  File "/usr/lib/python2.6/site-packages/peewee.py", line 3831, in execute_sql
    cursor.execute(sql, params or ())
 OperationalError: database is locked

Answer 1

Check out http://charlesleifer.com/blog/multi-threaded-sqlite-without-the-operationalerrors/ for some ideas -- you might try using wal-mode as the journalling mode.