[erlang-questions] os:cmd performance

[Erik Søe Sørensen]

Could it be because of a different  environment? In particular, a different
PATH could make a difference of that scale.
Try running "env" in both ways, and compare.
Den 29/07/2013 16.31 skrev "Michał Ptaszek" <erlang@REDACTED>:

> Hey,
>
> I am trying to run some shell commands from Erlang, using os:cmd/1:
>
>   os:cmd("cat /tmp/huge_file | grep 'some_regexp' | sort -u >
> /tmp/output_file").
>
> Where huge_file is ~4GB big, the output_file is expected to occupy ~150MB
> on disk. When running this command from Erlang it takes ~45-50 minutes to
> create the output file, however when ran directly from the shell (bash)
> everything happens within 10 minutes.
> It's not a matter of FS cache (I "echo 3 > /proc/sys/vm/drop_caches" right
> before each test), or external load (it's a physical machine, completely
> idle during the test). Erlang version: R16B.
>
> My question here is not "how to do it?", but rather "why os:cmd/1 is so
> inefficient?", even though the output returned to the caller is empty
> (stdout is redirected to the disk file, there are no stderr printouts). Or
> maybe I'm doing something entirely stupid that should be avoided...
>
> Any clues/hints?
> Thanks,
> Michal
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20130729/1378302a/attachment.htm>