Lua - Post multipart/form-data and a file via http.request

I’m trying to use the REST APi for Paperless-ngx to upload documents to a http server, their instructions are as follows..

POSTing documents

The API provides a special endpoint for file uploads:

/api/documents/post_document/

POST a multipart form to this endpoint, where the form field document contains the document that you want to upload to paperless. The filename is sanitized and then used to store the document in a temporary directory, and the consumer will be instructed to consume the document from there.

The endpoint supports the following optional form fields:

title: Specify a title that the consumer should use for the document.

created: Specify a DateTime document was created (e.g. “2016-04-19” or “2016-04-19 06:15:00+02:00”).

correspondent: Specify the ID of a correspondent that the consumer should use for the document.

document_type: Similar to correspondent.

tags: Similar to correspondent. Specify this multiple times to have multiple tags added to the document.

The endpoint will immediately return “OK” if the document consumption process was started successfully. No additional status information about the consumption process itself is available, since that happens in a different process

While I’ve been able to achieve what I needed with curl (see below), I’d like to achieve the same result with Lua.

curl -H "Authorization: Basic Y2hyaXM62tgbsgjunotmeY2hyaXNob3N0aW5n" -F "title=Companies House File 10" -F "correspondent=12" -F "document=@/mnt/nas/10.pdf" http://192.168.102.134:8777/api/documents/post_document/

On the Lua side, I’ve tried various ways to get this to work, but all have been unsuccessful, at best it just times out and returns nil.

Update: I’ve progressed from a nil timeout, to a 400 table: 0x1593c00 HTTP/1.1 400 Bad Request {"document":["No file was submitted."]} error message

Please could someone help ..

local http = require("socket.http")
local ltn12 = require("ltn12")
local mime = require("mime")
local lfs = require("lfs")

local username = "username"
local password = "password"

local httpendpoint = 'http://192.168.102.134:8777/api/documents/post_document/'
local filepath = "/mnt/nas/10.pdf"
local file = io.open(filepath, "rb")
local contents = file:read( "*a" )

-- https://stackoverflow.com/questions/3508338/what-is-the-boundary-in-multipart-form-data

local boundary = "somerndstring"
local send = "--"..boundary..
            "\r\nContent-Disposition: form-data; "..
            "title='testdoc'; document="..filepath..
            --"\r\nContent-type: image/png"..
            "\r\n\r\n"..contents..
            "\r\n--"..boundary.."--\r\n";

-- Execute request (returns response body, response code, response header)

local resp = {}
local body, code, headers, status = http.request {
    url = httpendpoint,
    method = 'POST',
    headers = {
        -- ['Content-Length'] = lfs.attributes(filepath, 'size') + string.len(send),
        -- ["Content-Length"] = fileContent:len(), 
        -- ["Content-Length"] = string.len(fileContent), 
        ["Content-Length"] = lfs.attributes(filepath, 'size'),
        ['Content-Type'] = "multipart/form-data; boundary="..boundary,
        ["Authorization"] = "Basic " .. (mime.b64(username ..":" .. password)),
        --body = send
    },
    source = ltn12.source.file( io.open(filepath,"rb") ),
    sink = ltn12.sink.table(resp)
}

print(body, code, headers, status)
print(table.concat(resp))

if headers then 
    for k,v in pairs(headers) do 
        print(k,v) 
    end
end

Solution

Huge thanks to a person on GitHub who helped me with this, and also has their own module to do it - https://github.com/catwell/lua-multipart-post .

local http = require("socket.http")
local ltn12 = require("ltn12")
local lfs = require "lfs"
http.TIMEOUT = 5

local function upload_file ( url, filename )
    local fileHandle = io.open( filename,"rb")
    local fileContent = fileHandle:read( "*a" )
    fileHandle:close()

    local boundary = 'abcd'

    local header_b = 'Content-Disposition: form-data; name="document"; filename="' .. filename .. '"\r\nContent-Type: application/pdf'
    local header_c = 'Content-Disposition: form-data; name="title"\r\n\r\nCompanies House File'
    local header_d = 'Content-Disposition: form-data; name="correspondent"\r\n\r\n12'

    local MP_b = '--'..boundary..'\r\n'..header_b..'\r\n\r\n'..fileContent..'\r\n'
    local MP_c = '--'..boundary..'\r\n'..header_c..'\r\n'
    local MP_d = '--'..boundary..'\r\n'..header_d..'\r\n'

    local MPCombined = MP_b..MP_c..MP_d..'--'..boundary..'--\r\n'

    local   response_body = { }
    local   _, code = http.request {
            url = url ,
            method = "POST",
            headers = {    ["Content-Length"] =  MPCombined:len(),
                           ['Content-Type'] = 'multipart/form-data; boundary=' .. boundary
                         },
            source = ltn12.source.string(MPCombined) ,
            sink = ltn12.sink.table(response_body),
                }
     return code, table.concat(response_body)
end

 local rc,content = upload_file ('http://httpbin.org/post', '/mnt/nas/10.pdf' )
 print(rc,content)