Is there a way to tracking and upload file and folder to S3 with c#?

Question

I want to upload a folder with subfolder and files to S3. At the same, I want to track the process. I can get the files to upload and track the process. However, it only uploads the files, no subfolder if the files are in subfolder.

For example:

C:\backup
  backup\testfile1.txt
  backup\test\
  Backup\test\testfile2.txt

The result after I uploaded to AWS:

 Testfile1.txt
 Testfile2.txt

What I want is:

Testfile1.txt
Test\testfile2.txt

     static void Main(string[] args)
        {
            var credentials = new BasicAWSCredentials("keyID", "secrectKey");
            s3Client = new AmazonS3Client(credentials, bucketRegion);
            TrackMPUAsync().Wait();

        }

        private static async Task TrackMPUAsync()
        {
            string[] files = Directory.GetFileSystemEntries(sourcePath, "*", SearchOption.AllDirectories);
           // string[] directories = Directory.GetDirectories(sourcePath, "*", SearchOption.AllDirectories);


            try
            {
                var fileTransferUtility = new TransferUtility(s3Client);

                // Use TransferUtilityUploadRequest to configure options.
                // In this example we subscribe to an event.
                foreach(string ss in files)
                {
                    Console.WriteLine(ss);

                        var uploadRequest =
                    new TransferUtilityUploadRequest
                    {
                        BucketName = bucketName,
                        FilePath = ss,

                    };

                        uploadRequest.UploadProgressEvent +=
                            new EventHandler<UploadProgressArgs>
                                (uploadRequest_UploadPartProgressEvent);

                        await fileTransferUtility.UploadAsync(uploadRequest);
                        Console.WriteLine("Upload " + ss + " completed");

                }


            }
            catch (AmazonS3Exception e)
            {
                Console.WriteLine("Error encountered on server. Message:'{0}' when writing an object", e.Message);
            }
            catch (Exception e)
            {
                Console.WriteLine("Unknown encountered on server. Message:'{0}' when writing an object", e.Message);
            }
        }

        static void uploadRequest_UploadPartProgressEvent(object sender, UploadProgressArgs e)
        {
            // Process event.
            Console.WriteLine("{0}/{1}", e.TransferredBytes, e.TotalBytes);
        }

Answer 1

Amazon S3 does not actually use folders or directories.

Instead, the Key (effectively the filename) of an object contains contain the full path of the object.

Therefore, the file should be called Test/testfile2.txt when it is stored in S3.

The Amazon S3 management console will make it 'look' like there are folders, but they aren't actually there.

For example, you could upload an object to folder1/folder2/foo.txt and it will 'appear' as though S3 has created the two folders, but if the object is deleted, the folders will also disappear.

Therefore, you should change your code so that the Key contains the full path of the object as you would like it in the Amazon S3 bucket.