Search code examples
pythonunit-testingboto3moto

Mock download file from s3 with actual file

I would like to write a test to mock the download of a function from s3 and replace it locally with an actual file that exists of my machine. I took inspiration from this post. The idea is the following:

from moto import mock_s3
import boto3

def dl(src_f, dest_f):
  s3 = boto3.resource('s3')
  s3.Bucket('fake_bucket').download_file(src_f, dest_f)

@mock_s3
def _create_and_mock_bucket():

    # Create fake bucket and mock it
    bucket = "fake_bucket"

    # We need to create the bucket since this is all in Moto's 'virtual' AWS account
    file_path = "some_real_file.txt"
    s3 = boto3.client("s3", region_name="us-east-1")
    s3.create_bucket(Bucket=bucket)
    s3.put_object(Bucket=bucket, Key=file_path, Body="")

    dl(file_path, 'some_other_real_file.txt')

_create_and_mock_bucket()

Now some_other_real_file.txt exists, but it is not a copy of some_real_file.txt. Any idea on how to do that?

Solution

  • If 'some_real_file.txt' already exists on your system, you should use upload_file instead: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.upload_file

    For your example:

    file_path = "some_real_file.txt"
s3 = boto3.client("s3", region_name="us-east-1")
s3.create_bucket(Bucket=bucket)
s3_resource = boto3.resource('s3')
s3_resource.meta.client.upload_file(file_path, bucket, file_path)

    Your code currently creates an empty file in S3 (since Body=""), and that is exactly what is being downloaded to 'some_other_real_file.txt'. Notice that, if you change the Body-parameter to have some text in it, that exact content will be downloaded to 'some_other_real_file.txt'.