Flask POST route becoming unresponsive

Question

I've just taken my first dip into Flask and am trying to use it to create an API which has GET, POST and DELETE routes to manage a blacklist of URLs.

After a period of time the POST route gives me an error: "Could not get any response." The strange part is that my GET and DELETE routes still work fine. I've tried to get some logging going but haven't been able to get much. I'd appreciate any help!

Here is my app:

from flask import Flask, request
from flask_cors import CORS, cross_origin
from flask_restful import Resource, Api
import pickle

app = Flask(__name__)
app.config['DEBUG'] = True
CORS(app)


@app.route("/get-blacklist")
def loadblacklist():
    blacklist = pickle.load( open( "blacklist.p", "rb" ) )
    return dumps(blacklist)


@app.route("/post-blacklist", methods=['POST'])
def puturl():
    # Get and clean URL
    url = request.form['url']
    url = sanitise(url)

    # Load the pickle
    blacklist = list(pickle.load( open( "blacklist.p", "rb" ) ))

    # Check if it's already in there
    if url not in blacklist:
        blacklist.append(url)

    # Save the pickle
    pickle.dump( blacklist, open( "blacklist.p", "wb" ) )

    return 'You posted {}'.format(url)


@app.route("/remove-blacklist", methods=['DELETE'])
def removeurl():
    # Get and clean the URL
    url = request.form['url']
    url = sanitise(url)

    # Load the pickle
    blacklist = list(pickle.load( open( "blacklist.p", "rb" ) ))

    # Remove if its in there
    if url in blacklist:
        blacklist.remove(url)

    # Save the pickle
    pickle.dump( blacklist, open( "blacklist.p", "wb" ) )

    return 'You removed {}'.format(url)


def sanitise(url):
    # Sanitise
    url = url.lower()
    url = url.replace('www.', '')
    url = url.replace('http://', '')
    url = url.replace('https://', '')
    if url[-1] == '/':
        url = url[:-1]
    return url


if __name__ == '__main__':
    app.run()

Here is my config file:

import os

bind = '0.0.0.0:8000'
workers = 3
backlog = 2048
worker_class = "sync"
debug = False
proc_name = 'gunicorn.proc'
pidfile = './gunicorn.pid'
logfile = './debug.log'
loglevel = 'debug'
timeout = 1000

And here is the command I'm using to start the workers:

gunicorn -c app.conf -w3 --certfile=/etc/letsencrypt/live/mtfmu4.tk/cert.pem --keyfile=/etc/letsencrypt/live/mtfmu4.tk/privkey.pem api:app

I can't see anything particularly different in my POST route that would cause this to happen - especially since it seems to work for awhile.

I'm running Python 3 in a virtualenv, Flask, gunicorn on apache2 and Ubuntu 14.04.

Any help would be appreciated!

Answer 1

I found the solution was to use the with statement to open my pickle file. Seemed to help whatever open/close conflict was occurring.