Showing the progress of a tornado request

Question

I have a tornado app that uploads large text files (about 500,000 lines) then does some processing on it and finally inserts data to MySQL table.

I did this, but I want show the progress of the process. In other words, since the process takes a long time, I want show the user how many records of file were processed and how many records were inserted into the table.

I think maybe it's possible to make a json file and write progress into and then with AJAX read content and show in page.

Is there a better way?

Answer 1

Finally, I got an answer.

I used WebSocket to view the progress of the uploaded file.

That is, I first upload the file with handler and then, when web page rendered to browser, I open a socket connection and start process with a thread.

Something like this:

class MyProcessWebSocket(WebSocketHandler):
    def __init__(self, application, request, **kwargs):
        super(ProcessBillWebSocket, self).__init__(application, request, **kwargs)
        self.fid = None

    def do_process(self, z, name):
        res = dict(percent=0)
        self.write_message(res)

        # do some process and self.write_message(res) in each loop.

        self.close()

    def open(self, *args, **kwargs):
        try:
            self.fid = args[0] if args[0] else None
        except:
            pass
        if not self.fid:
            self.close()
            return

        # Open file with self.fid that sent from rendered webpage.

        filename = os.path.join(sh.web['static_address'], 'tmp', '%s.txt' % self.fid)

        try:
            z = ProcessClass.read_file(filename)
            t = threading.Thread(target=self.do_process, args=(z, self.fid))
            t.start()
        finally:
            os.remove(filename)


    def on_message(self, message):
        # nothing to do
        pass

    def on_close(self):
        print("connection closed.")

I don't know this is good way or not, but my problem has been solved.

Is Any other idea???