How do I prevent message flooding with open_port?

Question

I'm trying to read a 100gb file through stdin one line at a time using

Port = open_port({fd, 0, 1}, [in, binary, {line, 4096}]),

but this floods my system with messages until I run out of ram. Is there a away to make it like {active, once} with ports? There is also io:get_line() but I was wondering if this could work.

Answer 1

No, there is not flow control over ports so if you can't process fast enough you should use another method of processing. You can set binary mode on STDIN using

ok = io:setopts(standard_io, [binary]),

and then you can read it using file:read_line(standard_io) if you are using version 17 or newer (there was performance impacting bug).