Search code examples
iosnode.jsswiftuploadlarge-files

URLSession dataTask / uploadTask splitting large uploads into multiple requests

Hoping someone can shed some light on this behavior.

  1. Large file (say 50mb+) needs to be uploaded to server
  2. Uploading is split into multiple requests, not chunks in a single request ( see Charles image )
  3. Using any of these all have the same behavior.
  • URLSession.shared.dataTask
  • URLSession.shared.uploadTask
  • URLSession.shared.uploadTask(withStreamedRequest:)
  • URLSession.shared.dataTaskPublisher
  1. Server is node / express, testing locally.
// Express server route
app.post('/save', (req, res, next) => {

   // Receives twice for the same upload
}

Example Attempt

    var request = URLRequest(url: url)
    request.httpMethod = "POST"
    request.setValue("application/octet-stream", forHTTPHeaderField: "content-type")
    request.setValue("chunked", forHTTPHeaderField: "transfer-encoding")
    request.httpBodyStream = InputStream(data: package)
    let task = self.session.uploadTask(withStreamedRequest: request)
    self.streamingTask = task
    task.resume()

Is this normal? It seems odd to me, and makes it difficult to put together the data at the end. I've set various headers to see if that's what I'm missing, such as Content-Length, tried multiple encodings ( multipart/form, json ).

Charles Image

Edit: Just discovered that Content-Length is a reserved header and cannot be manually set. But the content length is not being set to the size of the httpBody.

https://developer.apple.com/documentation/foundation/nsurlrequest

Solution

  • Well, as I suspected this was a question ripe for a face palm. The url system works as expected, but I was triggering through

    PHImageManager.default().requestImage

    which for anyone looking in the future might have its block called more than once if the first one it returns has the PHImageResultIsDegradedKey set to 1.

    Apple Doc Reference

    This is what was causing the requests to appear like they were being split apart, but in reality it was submitting a thumbnail of an image and then the full size once it loaded.

    The goose is caught, but it was a silly chase. Hopefully this will help someone in the future.