Implementing A Generic Progress Tracker For Batched Tasks

Question

I am currently working on a project which has a number of background tasks that get executed. Each task created and then sent to a concurrent service which manages the execution process of all tasks. Each task is stored in a database table during the length of its execution.

My dilemma is that each task performs very specific functions, and in general most delegate to a service in another part of the system, the brunt of the work is done, and then the task returns.

Currently I have a very simple system implemented which tracks the progress of task, it works well, however each task that is executed needs to add a lot of extra code to accommodate the functionality on the services it delegates to.

So to as an example my task would have a method:

@Override
public void execute() {
    service.calculateAverage();
}

And then correspondingly in the service:

public float calculateAverage() {
    float total = 0.0f; 
    for (i = 0; i < 20; i++) {
        total += i;
    }
    return total / 20;
}

Tracking the progress of this is fairly simple, I just update my task in the database after it has gone past a certain threshold of iterations. However, to generify this is proving to be quite a task, as each task that is executed might delegate to a different service entirely. This means in each service I need to add code specific to the implementation for that service.

I have done a bit of searching and I can't seem to find any good patterns that can help with creating a generic system for tracking the progress of each task. Any pointers or even just places to look or read-up on would be good.

Answer 1

If you make your service use your own Iterator rather than let it create the loop.

class SumService {

    private float calculateSum(Iterable<Integer> i) {
        float total = 0.0f;
        for (Integer x : i) {
            total += x;
        }
        return total;
    }

}

You can then create an Iterable that keep track of progress and reports it to the progress tracker.

/**
 * State of progress - returns a double result between 0 and 1.
 *
 * Will be called repeatedly by the progress tracker.
 */
interface Progress {

    public double getProgress();
}

/**
 * The progress tracker.
 */
static class ProgressTracker {

    // All processes are registered.
    static void registerProgressor(Progress p) {
        // Add it to mmy list of people to watch.
    }
}

/**
 * An Iterable that reports its progress.
 */
class ProgressingIterable<T> implements Iterable<T>, Progress {

    // The iterable we are hosting.
    final Iterable<T> it;
    // How far we are to go.
    final int steps;
    // Where we're at now.
    volatile int at = 0;

    public ProgressingIterable(Iterable<T> it, int steps) {
        this.it = it;
        this.steps = steps;
    }

    @Override
    public Iterator<T> iterator() {
        return new Iterator<T>() {
            // Grab an Iterator from the Iterable.
            Iterator<T> i = it.iterator();

            @Override
            public boolean hasNext() {
                // Delegate.
                return i.hasNext();
            }

            @Override
            public T next() {
                // Keep track of the steps.
                at++;
                return i.next();
            }

        };
    }

    @Override
    public double getProgress() {
        // How are we doing?
        return (double) at / (double) steps;
    }

}

/**
 * A range (from http://stackoverflow.com/a/6828887/823393).
 *
 * @param begin inclusive
 * @param end exclusive
 * @return list of integers from begin to end
 */
public static List<Integer> range(final int begin, final int end) {
    return new AbstractList<Integer>() {
        @Override
        public Integer get(int index) {
            return begin + index;
        }

        @Override
        public int size() {
            return end - begin;
        }
    };
}

/**
 * A process.
 */
class Process {

    ProgressingIterable<Integer> progress = new ProgressingIterable<>(range(0, 20), 20);

    public void execute() {
        // Register the Progress
        ProgressTracker.registerProgressor(progress);
        // Make the service use my progress object.
        service.calculateSum(progress);
    }

}

// The service it uses.
SumService service = new SumService();

This manages the separation of responsibilities. To the service is just an Iterable while to the progress tracker it will deliver current progress whenever asked.

I call this the Janus pattern because you have one object that does exactly two things. It allows you to bind two processes together in one object.

I have chosen the simplest of progress indicators - a double between 0 and 1. I am sure you could do a much better job.