Python NamedTemporaryFile - ValueError When Reading

Question

I am having an issue writing to a NamedTemporaryFile in Python and then reading it back. The function downloads a file via tftpy to the temp file, reads it, hashes the contents, and then compares the hash digest to the original file. The function in question is below:

def verify_upload(self, image, destination):
    # create a tftp client
    client = TftpClient(ip, 69, localip=self.binding_ip)
    # generate a temp file to hold the download info
    if not os.path.exists("temp"):
        os.makedirs("temp")
    with NamedTemporaryFile(dir="temp") as tempfile, open(image, 'r') as original:
        try:
            # attempt to download the target image
            client.download(destination, tempfile, timeout=self.download_timeout)
        except TftpTimeout:
            raise RuntimeError("Could not download {0} from {1} for verification".format(destination, self.target_ip))
        # hash the original file and the downloaded version
        original_digest = hashlib.sha256(original.read()).hexdigest()
        uploaded_digest = hashlib.sha256(tempfile.read()).hexdigest()
        if self.verbose:
            print "Original SHA-256: {0}\nUploaded SHA-256: {1}".format(original_digest, uploaded_digest)
        # return the hash comparison
        return original_digest == uploaded_digest

The problem is that every time I try to execute the line uploaded_digest = hashlib.sha256(tempfile.read()).hexdigest() the application errors out with a ValueError - I/O Operation on a closed file. Since the with block is not complete I am struggling to understand why the temp file would be closed. The only possibility I can think of is that tftpy is closing the file after doing the download, but I cannot find any point in the tftpy source where this would be happening. Note, I have also tried inserting the line tempfile.seek(0) in order to put the file back in a proper state for reading, however this also gives me the ValueError.

Is tftpy closing the file possibly? I read that there is possibly a bug in NamedTemporaryFile causing this problem? Why is the file closed before the reference defined by the with block goes out of scope?

Answer 1

TFTPy is closing the file. When you were looking at the source, you missed the following code path:

class TftpClient(TftpSession):
    ...
    def download(self, filename, output, packethook=None, timeout=SOCK_TIMEOUT):
        ...
        self.context = TftpContextClientDownload(self.host,
                                                 self.iport,
                                                 filename,
                                                 output,
                                                 self.options,
                                                 packethook,
                                                 timeout,
                                                 localip = self.localip)
        self.context.start()
        # Download happens here
        self.context.end()  # <--

TftpClient.download calls TftpContextClientDownload.end:

class TftpContextClientDownload(TftpContext):
    ...
    def end(self):
        """Finish up the context."""
        TftpContext.end(self)  # <--
        self.metrics.end_time = time.time()
        log.debug("Set metrics.end_time to %s", self.metrics.end_time)
        self.metrics.compute()

TftpContextClientDownload.end calls TftpContext.end:

class TftpContext(object):
    ...
    def end(self):
        """Perform session cleanup, since the end method should always be
        called explicitely by the calling code, this works better than the
        destructor."""
        log.debug("in TftpContext.end")
        self.sock.close()
        if self.fileobj is not None and not self.fileobj.closed:
            log.debug("self.fileobj is open - closing")
            self.fileobj.close()  # <--

and TftpContext.end closes the file.