Search code examples
amazon-web-servicesamazon-glacieraws-billing

Large Number of Requests While uploading a single file in AWS Glacier


I am new to AWS Glacier. I am trying to store some of my data into Amazon Glacier. I have uploaded a single file(archive) in a glacier, Now when I go to see the Billing Dashboard, It shows the Number of Requests is 20 and more.

I am unaware of what is happening. How the Number of Request hit will occur when I upload a single file. Is Number of Request changes according to the size of file?


Solution

  • Yes, the number of requests changes with the size of the file. The Glacier API supports multipart (with or without parallelism) uploads, and it requires that the part size be specified by the uploading code and constrained within some boundaries... so usually, your uploaded file will have multiple parts.

    When you initiate a multipart upload, you specify the part size in number of bytes. The part size must be a megabyte (1024 KB) multiplied by a power of 2—for example, 1048576 (1 MB), 2097152 (2 MB), 4194304 (4 MB), 8388608 (8 MB), and so on. The minimum allowable part size is 1 MB, and the maximum is 4 GB.

    https://docs.aws.amazon.com/amazonglacier/latest/dev/api-multipart-initiate-upload.html

    A complete API cycle for one small object requires at 3 requests -- initiate, (one) part, and complete. Larger objects will require multiple "part" uploads depending on the part size. The SDK you are using may be choosing some default values here.