Python Swift Client Object count mismatch with the actutal objects in cloud

Question

I am using following command to list all the objects in the container and dumping into the log file.

swift -A url -U username -K password list container_name> log.txt

I have checked the number of lines in the output log file and it showing the count 1076857 . But when I have checked the cloud manually and found the the count which is showing 1081756. What might be the reason for this mismatch? Is the above command excludes the empty folder?

Please suggest.

Answer 1

Large files (greater than 5GB) are stored on Swift partially(?) which are called segments.

For example:
Let's assume we have a file with size 13 GB and name log.txt.
We want to upload it to container container1.

This was the scenario, and Swift does the following:

1. Swift takes this file and divides into segments which are:

    log.txt_segment1 ---> size 5GB   
    log.txt_segment2 ---> size 5GB   
    log.txt_segment3 ---> size 3GB
   

2. Swift creates an object with name log.txt in container1 with size 0MB. Actually this one is not a real object, it is only the alias name.
3. Then Swift creates another container with name container1_segments which stores the real object (segmented object) of container1.
4. Then Swift uploads the segments of the log.txt to container1_segments.

So, what do we have now?
- we have 2 containers [container1, container1_segments]
- we have 4 objects [log.txt, log.txt_segment1, log.txt_segment2, log.txt_segment3]

Theoretically:
We should have 1 container and 1 object, but instead we have 2 containers and 4 objects; maybe this causes the object count mismatch for your Swift.