How to make the Erlang VM predictible when spawning processes?

[Ovidiu Deac]

I'm back with some tests that I've just ran on an Intel Core i5/Linux
2.6.32/erts v5.7.4.

The results look more credible since 1000 processes are no longer
spawned in 1microsecond. Also the variance between separate test
executions is much smaller.

But the problem remains if I spawn 1 milion processes I come across
spawns which take up to 18milliseconds. So again, any idea about how I
could make the VM to be predictible?

I assume that I should find the level of load that it can take.
Reusing processes as I would have done in other languages sounds like
a bad idea from start. Also I'm thinking I could increase the work
done by one process such that it's not too small but still it fits
inside the initial heap size so garbage collection is not needed.

Also another thing that I observed is that during the test only about
50% out of each processor seems to be used. At least this is what the
system monitor says. Any VM flags that would help with this? I tried
+P and +S but without results.

Ovidiu



On Thu, Jun 3, 2010 at 9:00 PM, Ovidiu Deac <ovidiudeac@REDACTED> wrote:
> Hello everybody,
>
> I'm doing some research in order to figure out if Erlang is suitable
> for an application that we need to implement.
>
> In a few words it's about a server which has a requirement saying "no
> request should take more then 10ms"', measured on the client's
> computer.
>
> So I started a couple of tests for various erlang components to figure
> out how much some operations would take. Partly because I wanted to
> get some figures and partly to exercise my Erlang skills.
>
> First I wrote a profiler module which can measure the execution time
> for various modules which implement set_up,tear_down and test_main.
> Also this profiler can repeat a test a number of times and returns a
> list with all the execution times, the minimum, the maximum and the
> average time. Nothing fancy.
>
> See the attachment for the profiler code and I post below some results
> for a test that I ran on a machine with Intel Core2 Duo/Win7/erlang
> v5.7.5. One test consists in 1000 spawns and is repeated 100 times.
> The times displayed are microseconds.
>
> $ make -C .. && erl +P 200000 -noshell -s profiler run spawn_test 100
> silent 0 1000 -s init stop
> make: Entering directory `/cygdrive/d/work/erlang/finn'
> make: Leaving directory `/cygdrive/d/work/erlang/finn'
> Module to test:spawn_test, Repeats:100, Params=[silent,'0','1000']
> Times elapsed = [1,15998,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,15991,1,1,1,1,
>                 14991,1,1,1,1,15991,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,
>                 15991,1,1,1,1,1,14989,1,1,1,1,15991,1,1,1,1,15991,1,1,1,
>                 14993,1,1,1,15993,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,15991,
>                 1,1,1,1,14991,1,1,1,15993,1,1,1,1,15991,1,1,1,1,14991]
> Max us/execution=15998
> Min us/execution=1
> Variation=15997
> Average us/execution=3279.01
>
> The average time for spawning a process is around 3us which is very
> good. It's interesting to see that spawning 1000 processes took
> 1microsecond (!!!) Looks like some bug it the profiler. Also my main
> concern is that from time to time spawning a process takes around 15ms
> which is way too much.
>
> I tried to run erl +P 200000 but the behaviour is the same as without
> this parameter.
>
> Do you have any idea how to make the VM predictible in order to
> satisfy the 10ms requirement?
>
> Thanks in advance,
> Ovidiu
>