[erlang-questions] input too fast

[Ulf Wiger (TN/EAB)]

I submitted a sum-file entry to the shootout, which worked
nicely in my environment(*), but failed miserably in the 
official benchmark.

http://shootout.alioth.debian.org/gp4/benchmark.php?test=sumcol&lang=hip
e&id=2


It uses the (admittedly undocumented) command-line flag for
installing a custom user process, and opens stdin in line-
oriented mode.

The problem is that it runs out of memory. As far as I can make
out, it's because the emulator chops up lines and sends them
to the process at such a high rate that, even though the 
process is in a tight loop and doing minimal work on each item,
it can't stop the message queue from building up.

This has disastrous effects when the input file is large enough.

I realise that the feature is undocumented, but perhaps it's still
a valid point - some sort of generic flow-control on ports, 
similar to the {active, bool()} on sockets, would be just the
thing here.

(*) I realise that I tested it in an NFS-mounted disk (on a clearcase-
enabled file system at that). This might have given the port 
sufficient flow control that the program lasted a bit longer, at least.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070626/10e376ea/attachment.htm>