I'm currently working on a module in Erlang that sends an HTTP POST request with a multipart/related message containing both JSON and binary data. However, I'm facing issues with concatenating the JSON message and binary data correctly.
-module(MIME_post).
-export([send_post_request/0, get_modify_req/0]).
send_post_request() ->
% ... (existing code)
Url = "http://localhost:8666",
JsonData = #{<<"some_val">> => <<"imsi-460886666660006">>},
JsonBody = jsx:encode(JsonData),
Headers = [{"Content-Type", "multipart/related;boundary=-21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5"}],
MimeBinary = <<"2e0a00d1">>,
Request = {Url, Headers, "multipart/related", JsonBody},
%here i want to include that MIME binary the Json body should get recognised as json and MIME binary should get recognized as another content type
case httpc:request(post, Request, [stream], []) of
{ok, {{_, 201, _}, _Headers, ResponseBody}} ->
io:format("HTTP Response Header:~p~n", [Headers]),
io:format("HTTP Response: ~s~n", [ResponseBody]),
{ok, ResponseBody};
{error, Reason} ->
io:format("HTTP Request failed with reason: ~p~n", [Reason]),
{error, Reason}
end.
request(Method, Request, HttpOptions, Options)
case httpc:request(post, Request, [stream], [] ) of
stream is not one of the available HttpOptions:
HttpOption =
{timeout, timeout()} |
{connect_timeout, timeout()} |
{ssl, [ssl:tls_option()]} |
{autoredirect, boolean()} |
{proxy_auth, {string(), string()}} |
{version, HttpVersion} |
{relaxed, boolean()}
Rather stream is one of the allowed Options for the fourth argument to httpc:request/4, and it should be in a 2-tuple along with a value.
In your Content-Type header, you probably want to change:
-21ba....
to:
21ba...
Also, add:
;type=\"application/json\"
to the end. Required.
You may also need to specify a Content-Length header that specifies the number of bytes in the body.
This is what you want the body of the request to look like:
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5
Content-Type: application/json
{"some_key":"imsi-460886666660006"}
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5
Content-Type: application/octet-stream
2e0a00d1
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5--
Note the trailing "--" on the last boundary. You will have to assemble all that by hand.
The actual boundary that appears in the body of the request is made up of the following parts:
An initial Carriage Return + Line Feed, which is referred to as CRLF, and consists of the characters "\r\n" (which is two octets). Omit the initial CRLF for the first boundary in the body.
The two characters "--".
The boundary specified in the Content-Type header.
A terminating CRLF, or if it's the last boundary, then a terminating "--".
See RFC 2406, Section 5.1.1 and RFC 822, Section 3.2.
Probably the easiest way to assemble the body is using an iolist, which is a list that contains the integers 0..255 or binaries or ascii-strings (or sublists containing any of those types). Here, the useful trait of an iolist is that erlang automatically concatenates everything together. Below, the code uses a list of strings to take advantage of that concatenation effect:
send_post_request() ->
% ... (existing code)
Url = "http://localhost:8080/print_request",
JsonData = #{<<"some_key">> => <<"imsi-460886666660006">>},
JsonBody = jsx:encode(JsonData),
MimeBinary = <<"2e0a00d1">>,
Boundary = "21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5",
CRLF = "\r\n",
ContentTypeHeader = [ % this is an iolist()
"multipart/related; ",
"boundary=", Boundary,
"; type=\"application/json\""
],
RequestBody = [ % this is an iolist()
"--", Boundary, CRLF,
"Content-Type: application/json", CRLF, CRLF,
JsonBody,
CRLF, "--", Boundary, CRLF,
"Content-Type: application/octet-stream", CRLF, CRLF,
MimeBinary,
CRLF, "--", Boundary, "--"
],
BodyBinary = iolist_to_binary(RequestBody),
ContentLength = byte_size(BodyBinary),
AdditionalHeaders = [
{
"Content-Length",
integer_to_list(ContentLength)
}
],
Request = {Url, AdditionalHeaders, ContentTypeHeader, BodyBinary},
Response = httpc:request(post, Request, [], []),
io:format("Response:~n~p~n", [Response]).
Using this express server (https://www.derpturkey.com/node-multipart-form-data-explained/):
let express = require('express');
let app = express();
const port = 8080;
app.post('/print_request', (req, res) => {
// output the headers
console.log(req.headers);
// capture the encoded form data
req.on('data', (data) => {
console.log(data.toString());
});
// send a response when finished reading
// the encoded form data
req.on('end', () => {
res.send('ok');
});
});
// start server on port 8080
app.listen(port, () => {
console.log(`Listening on port ${port}`)
});
...this is the output I see in the server window:
Listening on port 8080
{
'content-type': 'multipart/related; boundary=21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5; type="application/json"',
'content-length': '315',
te: '',
host: 'localhost:8080',
connection: 'keep-alive'
}
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5
Content-Type: application/json
{"some_key":"imsi-460886666660006"}
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5
Content-Type: application/octet-stream
2e0a00d1
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5--
Here's the output in the erlang shell:
1> my_http:send_post_request().
Response:
{ok,{{"HTTP/1.1",200,"OK"},
[{"connection","keep-alive"},
{"date","Sun, 21 Jan 2024 02:19:35 GMT"},
{"etag","W/\"2-eoX0dku9ba8cNUXvu/DyeabcC+s\""},
{"content-length","2"},
{"content-type","text/html; charset=utf-8"},
{"x-powered-by","Express"},
{"keep-alive","timeout=5"}],
"ok"}}
ok
Good luck. Are you sure your server knows how to parse the request body? If not, you probably want to use a Content-Type of multipart/form-data, with a Content-Type header like this:
"multipart/form-data;boundary=21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5"
And the body of the request will need to look something like this:
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5
Content-Disposition: form-data; name="my_json"
Content-Type: application/json <followed by two newlines>
<JSON HERE>
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5
Content-Disposition: form-data; name="my_binary_data"
Content-Type: application/octet-stream <followed by two newlines>
<BINARY DATA HERE>
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5--
A typical server will place the json in params["my_json"] and the binary data in params["my_binary_data"].
If your binary data was a jpeg file, you might have a section like this:
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5
Content-Disposition: form-data; name="my_image"; filename="red_square.jpg"
Content-Type: image/jpeg
r"?>��Adobed����''2&&2.&&&&.>55555>DAAAAAADDDDDDDDDDDDDDDDDDDDDDDDDDDDD &&6& &6D6++6DDDB5BDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD�}�"��a !S1A�r��5���
?����-4�f�fbb��a������� �|�G��>v��\��:>��f��)[�֍��6��P�n��yo�� 3��-�C�������#|�@I>�W�������_'�ol���Z�:�_&+ֻ/�Ԙ����
--21ba7406cff5fa6c192d6b78fe58e16c5fd0cde4111e10f11c2bf55b45a5
Response to comment =====:
In your handler for your route, look for a multipart request:
-module(hello_handler).
-behavior(cowboy_handler).
-export([init/2, get_json_part/1]).
init(Req, State) ->
case cowboy_req:parse_header(<<"content-type">>, Req) of
{<<"multipart">>, <<"related">>, _} ->
io:format("Found multipart request~n"),
get_json_part(Req);
_ ->
io:format("do something else for requests that aren't multipart~n")
end,
NewReq = cowboy_req:reply(200,
#{<<"content-type">> => <<"text/plain">>},
<<"Hello Erlang">>,
Req),
{ok, NewReq, State}.
Then process the multipart request with get_json_part(Req):
get_json_part(Req) ->
case cowboy_req:read_part(Req) of
{ok, Header=#{<<"content-type">> := <<"application/json">>}, Req1} ->
{ok, Body, _Req2} = cowboy_req:read_part_body(Req1),
io:format("Header: ~p~n", [Header]),
io:format("Body: ~p~n", [Body]);
{ok, _, Req1} ->
get_json_part(Req1);
{done, _Req1} ->
io:format("json part not found")
end.
This expression:
Header=#{<<"content-type">> := <<"application/json">>}
matches against a Content-Type header in one of the parts of the multipart request returned by cowboy_req:read_part(Req). This part:
#{<<"content-type">> := <<"application/json">>}
is a map, and to match it against another map, instead of the normal => that you see between a key and a value in a map:
#{<<"content-type">> => <<"application/json">>}
you have to use :=. You can also match just part of a map: you don't have to match the whole map. So any map that contains the key-value pair:
<<"content-type">> => <<"application/json">>
will match the following:
#{<<"content-type">> := <<"application/json">>}
The matching map is then assigned to the variable Header, so that the matching map can be used later in the function:
Header=#{<<"content-type">> := <<"application/json">>}
Here's a simpler example doing the same thing:
1> Map = #{x => 1, y =>2}.
#{y => 2,x => 1}
2> Result = #{x := 1} = Map.
#{y => 2,x => 1}
3> Result.
#{y => 2,x => 1}