Search code examples
socketserlanggen-tcp

Receiving unknown strings lengths?

So I'm converting a Python program I wrote to Erlang, and it's been a long time since I used Erlang. So I guest I'm moved back to beginner level. Anyways from experience every language I use when dealing with sockets have send/recv functions that always return the length of data sent/receive. In Erlangs gen_tcp case however doesn't seem to do that.

So when I call send/recv/or inet:setopts it knows when the packet has ended? Will I need to write a looping recvAll/sendAll function so I can find the escape or \n in the packet(string) I wish to receive?

http://erlang.org/doc/man/gen_tcp.html#recv-2

Example code I'm using:

server(LS) ->
    case gen_tcp:accept(LS) of
        {ok,S} ->
            loop(S),
            server(LS);
        Other ->
            io:format("accept returned ~w - goodbye!~n",[Other]),
            ok
    end.

loop(S) ->
    inet:setopts(S,[{active,once}]),
    receive
        {tcp,S,Data} ->
            Answer = process(Data), % Not implemented in this example
            gen_tcp:send(S,Answer),
            loop(S);
        {tcp_closed,S} ->
            io:format("Socket ~w closed [~w]~n",[S,self()]),
            ok
    end.

Just from looking at examples and documentation it seems like Erlang just knows. And I want to confirm, because the length of data being received can be anywhere between to 20 bytes to 9216 bytes and or could be sent in chunks since the client is a PHP socket library I'm writing.

Thank you,

Ajm.

Solution

  • TL;DR

    So when I call send/recv/or inet:setopts it knows when the packet has ended?

    No, it doesn't.

    Will I need to write a looping recvAll/sendAll function so I can find the escape or \n in the packet(string) I wish to receive?

    Yes, generally, you will. But erlang can do this job for you.

    HOW?

    Actually, you couldn't rely on TCP in sense of splitting messages into packets. In general, TCP will split your stream to arbitrary sized chunks, and you program have to assemble this chunks and parse this stream by own. So, first, your protocol must be "self delimiting". For example you can:

    1. In binary protocol - precede each packet with its length (fixed-size field). So, protocol frame will looks like this: <<PacketLength:2/big-unsigned-integer, Packet/binary>>
    2. In text protocol - terminate each line with line feed symbol.

    Erlang can help you with this deal. Take a look here http://erlang.org/doc/man/gen_tcp.html#type-option. There is important option:

    {packet, PacketType}(TCP/IP sockets)

Defines the type of packets to use for a socket. The following values are valid:

raw | 0

    No packaging is done.
1 | 2 | 4

    Packets consist of a header specifying the number of bytes in the packet, followed by that number of bytes. The length of header can be one, two, or four bytes; containing an unsigned integer in big-endian byte order. Each send operation will generate the header, and the header will be stripped off on each receive operation.

    In current implementation the 4-byte header is limited to 2Gb.

line

    Line mode, a packet is a line terminated with newline, lines longer than the receive buffer are truncated.

    Last option (line) is most interesting for you. If you'll set this option, erlang will parse input stream internally and yeld packets splitted by lines.