Search code examples
c#dropbox-api

Getting an error when uploading to Dropbox

I am attempting to upload a file to dropbox.

Here is what I have said so far:

    using (var stream = new MemoryStream(File.ReadAllBytes(fileName)))
    {
      var numChunks = (int) Math.Ceiling((double) stream.Length / chunkSize);
      var buffer = new byte[chunkSize];
      string sessionId = null;

      if (numChunks == 1)
      {
        using (var memStream = new MemoryStream(buffer, 0, chunkSize))
        {
          Console.WriteLine($"Sending file: {path}");
          var tst = await client.Files.UploadAsync(path, WriteMode.Overwrite.Instance, body: memStream);
        }
      }
      else
      {
        for (var idx = 0; idx < numChunks; idx++)
        {
          Console.WriteLine($"Uploading chunk {idx + 1} / {numChunks}.");
          var byteRead = stream.Read(buffer, 0, chunkSize);

          using (var memStream = new MemoryStream(buffer, 0, byteRead))
          {
            if (idx == 0)
            {
              var result = await client.Files.UploadSessionStartAsync(body: memStream);
              sessionId = result.SessionId;
            }

            var cursor = new UploadSessionCursor(sessionId, (ulong) (chunkSize * idx));

            if (idx == numChunks - 1)
            {
              Console.WriteLine($"Finalizing file: {path}");
              var response = await client.Files.UploadSessionFinishAsync(cursor, new CommitInfo(path), memStream);
            }
            else
            {
              await client.Files.UploadSessionAppendV2Async(cursor, body: memStream);
            }
          }
        }
      }

      var url = string.Empty;
      var link = await client.Sharing.ListSharedLinksAsync(path);

      if (link.Links.Count == 0)
      {
        var result = await client.Sharing.CreateSharedLinkWithSettingsAsync(path);
        url = result.Url;
      }
      else
      {
        url = link.Links[0].Url;
      }

      Console.WriteLine();
      Console.WriteLine("Dropbox Download Link:");
      Console.WriteLine(url);
      Console.WriteLine();
      //Console.ReadKey();
    }

When I try to send up a large file, I'm getting this message:

enter image description here

I'm accounting for small and large files. If the file is bigger than the chunksize. I cannot get it to send up the file, I just get an error.

It's happening at this line:

var response = await client.Files.UploadSessionFinishAsync(cursor, new CommitInfo(path), memStream);

Any suggestions?

Solution

  • Didn't run it locally and didn't use Dropbox's client but I see the following...

    1. Probably, while slow manual debugs you can miss the issue.
    2. Expect, that the error message is slightly misleading. In general, it is right. But there should be an additional check to catch a case when your original Stream was changed.
    3. Let me add more sync/async chaos data about 'what the hell is happening here?!.'. At first, your for runs synchronously - all upload sessions start in parallel in a short period of time. At second, each of fors has its own block-level scope that has the using (...) {...} syntax sugar in the sync context. All var memStreams are covering each other in the upper sync context. Then, every thread does its work in its own async context. Take a look at the var response. It is a loop variable and it is not in use in your main SYNC context. It is not declared out of the using closure. So this line can be not awaitable in the sync context. In this case using will finish when the memStream will be consumed but UploadSessionFinishAsync will be working still.
    4. Why it happens only at var response = await client.Files.UploadSessionFinishAsync(cursor, new CommitInfo(path), memStream);? Think, it could be the issue in each await client.Files... call but only client.Files.UploadSessionFinishAsync tries to do something with the passed memStream. For example, close it as a final step. Or process the memStream, abandoned the stream, do another work such as a request to Dropbox to get a final upload result.
    5. Idea about the fix. I think you need to declare the final result var response above the for scope. Then just set the result at the last chunk processing. In this case using will have to wait for its result, only then it could dispose memStream. Just like with sessionId = result.SessionId;. Or just explicitly wait for its result before the using end.
    6. Code optimization. For a short file you don't need buffer and sessionId. They are for multiple uploading threads. For a big file - return the UploadSessionFinishAsync result from your loop. Maybe, declare multiple memStreams as different vars. Not sure about UploadSessionAppendV2Async result. If it is void then it is OK. Otherwise, such a result also should be awaited. It could be like this:
    //outer vars
SomeDropboxFileUploadResultOrElse uploadResult;
string sessionId = null;

//original full file read
using (var stream = new MemoryStream(File.ReadAllBytes(fileName)))
{
   var numChunks = (int) Math.Ceiling((double) stream.Length / chunkSize);
   if (numChunks == 1)
   {
       Console.WriteLine($"Sending file: {path}");
       uploadResult = await client.Files.UploadAsync(path, WriteMode.Overwrite.Instance, body: stream);
   }
   else
   {
       var buffer = new byte[chunkSize];
       for (var idx = 0; idx < numChunks; idx++)
       {
           Console.WriteLine($"Uploading chunk {idx + 1} / {numChunks}.");
           var byteRead = stream.Read(buffer, 0, chunkSize);

           using (var memStream = new MemoryStream(buffer, 0, byteRead))
           {
               if (idx == 0)
               {
                   var result = await client.Files.UploadSessionStartAsync(body: memStream);
                   sessionId = result.SessionId;
               }
               //prevent 2+ parts processing on idx == 0 iteration, start thread in the sync scope but wait for sessionId value in the async thread
               else
               {
                   var cursor = new UploadSessionCursor(sessionId, (ulong)(chunkSize * idx));
                   if (idx == numChunks - 1)
                   {
                       Console.WriteLine($"Finalizing file: {path}");
                       //explicitly pass result to outer scope, then the execution could be continued, memStream could be closed below
                       uploadResult = await client.Files.UploadSessionFinishAsync(cursor, new CommitInfo(path), memStream);
                   }
                   else
                   {
                       await client.Files.UploadSessionAppendV2Async(cursor, body: memStream);
                   }
               }
           } //memStream?.Dispose(); 
       }
   }
...
}

    Hope, this will help.