[erlang-questions] I/O performance when reading from stdin

[Fredrik Svahn]

It is a known problem that reading from stdin is slow. I believe it
has been discussed several times on the mailing list.

Most erlang applications do not read from stdin, at least not huge
amounts of data. Erlang was originally built and optimized for
applications that are supposed to run for years with thousands of
concurrent processes, not for shortlived scripts running just a single
process.

However, there has been some small improvements in later releases. You
do not mention what release you are running, but my guess would be
R12B-4 or earlier? I get pretty much the same results as you get when
running R12B-3. With R12B-5, however, the stdio variant is just as
fast as file io with get_line (but still many times slower than a
corresponding c/c++ program). There is no improvement in memory
consumption in R12B-5, though.

By the way, if the file is small enough to fit in memory then
file:read_file/1 is a lot faster!

BR /Fredrik
PS, How would you like to improve the string handling?


On Sat, Jan 10, 2009 at 6:36 PM, Hynek Schlawack <hs@REDACTED> wrote:
> Hi,
>
> I'm starting to learn and love Erlang and wanted as a small exercise
> implement a word occurrence counter for http://ptrace.fefe.de/wp/ .
>
> To say it kindly, Erlang isn't built for string operations. :) My
> (mostly approved by #erlang) version[1] is twice as slow as the slowest
> already submitted[2].
>
> Anyway, I encountered a rather strange effect I'd like to ask about:
> When I read from stdin, the program takes twice the time and memory
> compared to the case when I open the file myself. Any hints? Is it a
> bug? Isn't it just the "intended use" of Erlang and nobody cares? The
> effect occurs under Linux, haven't tested it elsewhere.
>
> Thanks!
> Hynek
> --
> [1] http://github.com/hynek/wp-erlang/tree/master/wp.erl
> [2] http://ptrace.fefe.de/wp/timings.txt
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>