Boto3 file upload to S3 Bucket is failing SHA256 checksum check

Question

I am trying to upload a file to S3 Bucket. I am calculating the SHA256 checksum in my code and passing that to the put_object method to ensure the integrity of the file uploaded.

However, I am getting an error botocore.exceptions.ClientError: An error occurred (InvalidRequest) when calling the PutObject operation: Value for x-amz-checksum-sha256 header is invalid. when I am doing this.

The following code snippet is how I am calculating the checksum in my python code:

def checksum_sha256(filename):
        """ Calculates SHA256 Checksum for specified filename
        """
        sha256_hash = hashlib.sha256()
        with open(filename, "rb") as f:
            for byte_block in iter(lambda: f.read(8192), b""):
                sha256_hash.update(byte_block)
            return str(b64encode(bytes(sha256_hash.hexdigest(), "utf-8")))

The following code snippet shows how the file is being uploaded to S3 using Boto3 S3 Client:

checksum = checksum_sha256(file_location)
client = boto3.client("s3")
with open(file_location, 'rb') as file_obj:
    response = client.put_object(Body=file_obj, Bucket=bucket, Key=target_file_name,
                                 ChecksumAlgorithm='SHA256',
                                 ChecksumSHA256=checksum)

This is leading to the following error:

botocore.exceptions.ClientError: An error occurred (InvalidRequest) when calling the PutObject operation: Value for x-amz-checksum-sha256 header is invalid.

I can't pinpoint where I am going wrong with this. Any help is appreciated. Thanks.

Answer 1

I found the following changes to the checksum calculation fixed the error:

def checksum_sha256(filename):
    """ Calculates SHA256 Checksum for specified filename
    """
    sha256_hash = hashlib.sha256()
    with open(filename, "rb") as f:
        # Read and update hash string value in blocks of 8K
        for byte_block in iter(lambda: f.read(8192), b""):
            sha256_hash.update(byte_block)
        return b64encode(sha256_hash.digest()).decode()