Separating Progress Tracking and Loop Logic

Question

Suppose i want to track the progress of a loop using the progress bar printer ProgressMeter (as described in this recipe).

def bigIteration(collection):
    for element in collection:
        doWork(element)

I would like to be able to switch the progress bar on and off. I also want to update it only every x steps for performance reasons. My naive way to do this is

def bigIteration(collection, progressbar=True):
    if progressBar:
        pm = progress.ProgressMeter(total=len(collection))
        pc = 0
    for element in collection:
        if progressBar:
            pc += 1
            if pc % 100 = 0:
                pm.update(pc)
        doWork(element)

However, I am not satisfied. From an "aesthetic" point of view, the functional code of the loop is now "contaminated" with generic progress-tracking code.

Can you think of a way to cleanly separate progress-tracking code and functional code? (Can there be a progress-tracking decorator or something?)

Answer 1

It seems like this code would benefit from the null object pattern.

# a progress bar that uses ProgressMeter
class RealProgressBar:
     pm = Nothing
     def setMaximum(self, max):
         pm = progress.ProgressMeter(total=max)
         pc = 0
     def progress(self):
        pc += 1
        if pc % 100 = 0:
            pm.update(pc)

# a fake progress bar that does nothing
class NoProgressBar:
    def setMaximum(self, max):
         pass 
    def progress(self):
         pass

# Iterate with a given progress bar
def bigIteration(collection, progressBar=NoProgressBar()):
    progressBar.setMaximum(len(collection))
    for element in collection:
        progressBar.progress()
        doWork(element)

bigIteration(collection, RealProgressBar())

(Pardon my French, er, Python, it's not my native language ;) Hope you get the idea, though.)

This lets you move the progress update logic from the loop, but you still have some progress related calls in there.

You can remove this part if you create a generator from the collection that automatically tracks progress as you iterate it.

 # turn a collection into one that shows progress when iterated
 def withProgress(collection, progressBar=NoProgressBar()):
      progressBar.setMaximum(len(collection))
      for element in collection:
           progressBar.progress();
           yield element

 # simple iteration function
 def bigIteration(collection):
    for element in collection:
        doWork(element)

 # let's iterate with progress reports
 bigIteration(withProgress(collection, RealProgressBar()))

This approach leaves your bigIteration function as is and is highly composable. For example, let's say you also want to add cancellation this big iteration of yours. Just create another generator that happens to be cancellable.

# highly simplified cancellation token
# probably needs synchronization
class CancellationToken:
     cancelled = False
     def isCancelled(self):
         return cancelled
     def cancel(self):
         cancelled = True

# iterates a collection with cancellation support
def withCancellation(collection, cancelToken):
     for element in collection:
         if cancelToken.isCancelled():
             break
         yield element

progressCollection = withProgress(collection, RealProgressBar())
cancellableCollection = withCancellation(progressCollection, cancelToken)
bigIteration(cancellableCollection)

# meanwhile, on another thread...
cancelToken.cancel()