Search code examples
pythonmockingbotoboto3botocore

Mocking boto3 S3 client method Python


I'm trying to mock a singluar method from the boto3 s3 client object to throw an exception. But I need all other methods for this class to work as normal.

This is so I can test a singular Exception test when and error occurs performing a upload_part_copy

1st Attempt

import boto3
from mock import patch

with patch('botocore.client.S3.upload_part_copy', side_effect=Exception('Error Uploading')) as mock:
    client = boto3.client('s3')
    # Should return actual result
    o = client.get_object(Bucket='my-bucket', Key='my-key')
    # Should return mocked exception
    e = client.upload_part_copy()

However this gives the following error:

ImportError: No module named S3

2nd Attempt

After looking at the botocore.client.py source code I found that it is doing something clever and the method upload_part_copy does not exist. I found that it seems to call BaseClient._make_api_call instead so I tried to mock that

import boto3
from mock import patch

with patch('botocore.client.BaseClient._make_api_call', side_effect=Exception('Error Uploading')) as mock:
    client = boto3.client('s3')
    # Should return actual result
    o = client.get_object(Bucket='my-bucket', Key='my-key')
    # Should return mocked exception
    e = client.upload_part_copy()

This throws an exception... but on the get_object which I want to avoid.

Any ideas about how I can only throw the exception on the upload_part_copy method?


Solution

  • As soon as I posted on here I managed to come up with a solution. Here it is hope it helps :)

    import botocore
    from botocore.exceptions import ClientError
    from mock import patch
    import boto3
    
    orig = botocore.client.BaseClient._make_api_call
    
    def mock_make_api_call(self, operation_name, kwarg):
        if operation_name == 'UploadPartCopy':
            parsed_response = {'Error': {'Code': '500', 'Message': 'Error Uploading'}}
            raise ClientError(parsed_response, operation_name)
        return orig(self, operation_name, kwarg)
    
    with patch('botocore.client.BaseClient._make_api_call', new=mock_make_api_call):
        client = boto3.client('s3')
        # Should return actual result
        o = client.get_object(Bucket='my-bucket', Key='my-key')
        # Should return mocked exception
        e = client.upload_part_copy()
    

    Jordan Philips also posted a great solution using the the botocore.stub.Stubber class. Whilst a cleaner solution I was un-able to mock specific operations.