Search code examples
pythonmatplotlibpython-itertools

Python: Printing figures from file is getting slower and slower


I have a large file (~10 GB) which contains t = 1000 (n x n)-matrices where n = 1000. The data file contains float numbers. I wrote a program which prints every matrix as a .png image. The problem is, that it gets slower and slower. The first image needs less than a second to print. Image 350 needs more than 2 minutes! The plot below shows the problem for t = 27 matrices / images.

enter image description here

Here is my code I use. Does someone see why it is getting slower and slower?

import itertools as it
import matplotlib.pyplot as plt
import time

def printImage(M, k):
    fig = plt.imshow(M, interpolation='nearest')
    fig.set_cmap('hot')
    fig.axes.get_xaxis().set_visible(False)
    fig.axes.get_yaxis().set_visible(False)
    plt.axis('off')
    plt.tight_layout()
    plt.savefig(str(k) + '.png', bbox_inches='tight', pad_inches=0, dpi=300)

    plt.ioff()

n = 1000
t = 1000

with open('data', 'r') as f:
    for i in range(t):
        t0 = time.clock()
        try:
            items = [list(map(float, i.split())) for i in it.islice(f, n)]
        except:
            raise
        else:
            printImage(items,i)
        t1 = time.clock()
        print(str(i) + '/' + str(t) + ' ' + str(t1-t0))

Solution

  • The issue is probably because you're not closing your figures, which consumes more and more memory.

    Modify your plotting to read

    import itertools as it
    import matplotlib.pyplot as plt
    import time
    
    def printImage(M, k):
        figobj = plt.figure()
        fig = plt.imshow(M, interpolation='nearest')
        fig.set_cmap('hot')
        fig.axes.get_xaxis().set_visible(False)
        fig.axes.get_yaxis().set_visible(False)
        plt.axis('off')
        plt.tight_layout()
        plt.savefig(str(k) + '.png', bbox_inches='tight', pad_inches=0, dpi=300)
        plt.close(figobj)
    

    You probably don't need the call to plt.ioff either.