[erlang-questions] input too fast

[Ulf Wiger (TN/EAB)]

Per Hedeland wrote:
> 
> I didn't see any lasting memory buildup in this test, with or 
> without 'binary', but this just goes to show that "really 
> really really fast"
> can be somewhat quantified, not that the port version can in 
> general handle arbitrarily large files: In Ulf's case, using 
> 'line' mode probably turned the 64k chunk into more than 1000 
> line-list messages, and it may well be that just processing 
> those with zero "useful" work exceeded the number of 
> reductions that the VM will do before polling for input again 

Indeed. I inserted a debug statement which periodically 
sampled the message queue length, and it was in excess of 
12000 messages and growing fast (of course the debug 
statement is likely to have made the condition worse...) 

The lines in the sum-file test had only one number each, 
so it was pretty much the worst possible scenario given 
that there was no way to apply back-pressure to the 
port input.

I did manage to make it complete in constant space, by
giving the process max priority, a heap of a million words,
and adding the +T 9 flag (which messes with the timing).

BR,
Ulf W