Why is tempfile.TemporaryFile showing invalid path on AWS ECS/docker?

Question

I am surprised to see very different behavior of tempfile.TemporaryFile() on Windows vs. Linux.

As is best pratice, I am using Python's tempfile module, as also suggested here.

Intent: I want to create a temporary file, and pass its name/path to some function (not the file object). The following code works fine on Windows (Win 10, Python 3.8.6), but fails in a docker container on Amazon ECS (python:3.8.6-slim-buster).

import tempfile
import boto3

with tempfile.TemporaryFile() as tmpfile:
    s3 = boto3.client('s3')
    s3.download_file("my_bucket", "my_object", tmpfile.name)

This code fails not for S3 access rights/roles (that's all ok), but indeed due to the file name being invalid! A valid file name is produced on Windows (see output below), but is clearly invalid in the docker container on ECS:

• Windows: tmpfile.name: C:\Users\micha\AppData\Local\Temp\tmppky63n59
• Docker/ECS: tmpfile.name: 6

Why does tempfile.TemporaryFile().name produce valid output in one case, but no the other? How can I fix it?

Answer 1

This is because for tempfile.TemporaryFile():

The returned object is a true file object on POSIX platforms. On other platforms, it is a file-like object whose file attribute is the underlying true file object.

instead, you should use tempfile.NamedTemporaryFile():

This function operates exactly as TemporaryFile() does, except that the file is guaranteed to have a visible name in the file system (on Unix, the directory entry is not unlinked). That name can be retrieved from the name attribute of the returned file-like object.

so:

import tempfile
import boto3

with tempfile.NamedTemporaryFile() as tmpfile:
    s3 = boto3.client('s3')
    s3.download_file("my_bucket", "my_object", tmpfile.name)