Tracking Long-Running Task Status in Luigi's Central Scheduler Web Interface

Question

In the Luigi framework, I am trying to show a progress bar of a long-running task in the central scheduler's web interface using set_tracking_url, set_progress_bar and set_status in the run() method, like this:

def run(self):
    self.set_tracking_url("127.0.0.1:8082")
    for i in range(100):
        self.do_long_calculation(i)
        self.set_status_message("Analyzing Id %d" % i)
        self.set_progress_percentage(i)

and I'm running the task using

PYTHONPATH='.' luigi --module AnalysisTasks LongTask --workers=5

where AnalysisTasks is the python source file and LongTask is the task to which the run() method belongs and luigid running in the background. However I do not see any progress bar or status report. I haven't found any answers or examples to this anywhere. Is it at all possible?

Answer 1

class MyTestTask(Task):
    name = "MyTestTask"
    target = ["test"]

    def run(self):
        for i in range(100):
            time.sleep(1)
            self.set_progress_percentage(i / 10)
            print i
        return {self.target[0]: i}