Boto3 upload to S3: cut off last few rows of data from a .csv file

Question

When I upload a .csv file using boto3 (Python), the last few rows of data get cut off. The size of the file is 268KB which should not to be too big for non-multipart upload. Here is my code:

import boto3 
s3 = boto3.client('s3') 
s3 = boto3.resource('s3')
s3.meta.client.upload_file(report_file.name, 'raw-data-bucket', 'Reports/report.csv') 

*These are not exact bucket and path I've used but they suould be irrelevant in this case

Any help would be appriciated.

Answer 1

Have you closed the file you are uploading to S3 before using .upload_file() ? I had the exact same problem with my *.CSV file upoads, and solved it by explicitly closing each file before uploading it, and the problem was solved. No more truncated *.CSV files.