Search code examples
python-3.xamazon-web-servicesamazon-s3io

Measure consumed in and out bandwidth from s3 bucket in Python

I am using cloud computing with Amazon S3 bucket. Recently the cost for outbounds usage from S3 exploded. I made various changes to my code, hence I am not able to pinpoint the source of the extra cost. Is there a python package that could help me measure my io with S3

Solution

  • You should Log requests using server access logging - Amazon Simple Storage Service.

    Server Access Logging will create (many) log files in Amazon S3 where you can monitor requests and the size (Bytes Sent) of responses.

    See: Amazon S3 server access log format - Amazon Simple Storage Service

    The number of log files, and the amount of content within each log file, can make it difficult to analyze. You might want to use Amazon Athena to query these access logs.

    See: Analyze S3 server access logs using Athena