My Flask application needs to scrape another webpage in order to update some database information every minute-or-so. So I made a background task to do this job, but unfortunately it seems to not actually modify the database, failing silently.
I have a database connection that looks like this:
config.py:
class Configuration(object):
DATABASE = {
'name': 'database.db',
'engine': 'peewee.SqliteDatabase',
'check_same_thread': False,
}
__ init __.py:
import flask
app = flask.Flask(__name__)
from flask_peewee.db import Database
app.config.from_object('config.Configuration')
db = Database(app)
And then I read/write database entries within @app.route() functions. Here's the background task, scrape.py:
from myapp import *
db.connect_db()
while True:
#scrape and update the database with commands similar to:
user = User.get(id=5)
user.value += 1
user.save()
The code works when I put it in with the rest of my views, like:
@app.route("/scrape")
def scrape_update_db():
user = User.get(id=5)
user.value += 1
user.save()
return "Done"
and then periodically direct my webbrowser to /scrape. So I could instead make a background task that just connects to http://localhost:80/scrape every minute. But this seems rather convoluted, and I image it wouldn't be as computationally efficient (the server is running on weak hardware) or maintainable.
How can I update database entries in the background?
Ok, so my application contains code that automatically creates the database/tables if it's empty. It turns out that Apache/WSGI was running the Flask application from a different working directory than what I was running my background worker in. That caused the worker application to create its own database, so the two applications were actually working with separate databases.
So the solution was to make sure they each ran from the same directory.