Search code examples
rsharepointmicrosoft-graph-apihttr

Using httr + graph API for large files


I'm trying to upload large files to SharePoint Online directly through R using the Microsoft Graph API. To this end, I'm using the API's createUploadSession, documentation here.

My code looks like this:

httr::PUT(url = "https://graph.microsoft.com/v1.0/sites/.../createUploadSession", 
          headers   = add_headers(.headers = headers), 
          body      = httr::upload_file(file),
          encode    = mime::guess_type(file), 
          verbose())

(where 'headers' include authentication and host name, here graph.microsoft.com)

And the resultant request looks like this:

-> PUT /v1.0/sites/.../createUploadSession HTTP/1.1
-> Host: graph.microsoft.com
-> User-Agent: libcurl/7.64.1 r-curl/4.3 httr/1.4.2
-> Accept-Encoding: deflate, gzip
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Type: text/plain
-> Content-Length: 4543954542

Of course, this fails:

<- HTTP/1.1 413 Request Entity Too Large
<- Content-Type: text/html
<- Server: Microsoft-IIS/10.0
<- Strict-Transport-Security: max-age=31536000
<- Date: Fri, 02 Oct 2020 12:32:29 GMT
<- Connection: close
<- Content-Length: 67
<- 

since as the documentation says, we need to upload in 327,680 byte chunks. However, I was under the assumption that httr's upload_file allows for streaming. This is where I'm stuck: it looks like my request is still trying to upload this all at once, so how do I 'invoke' this streaming behavior? And is some kind of while loop required to continue sending the next chunk of data?


Solution

  • This functionality is now available in the Microsoft365R package. Full disclosure: I'm the author of this package.

    site <- get_sharepoint_site("sitename")
    site$get_drive()$upload_file("path/to/file", "destfilename")