[erlang-questions] Re: How to make the Erlang VM predictible when spawning processes?

[Hendrik Visage]

On 6/4/10, Ovidiu Deac <ovidiudeac@REDACTED> wrote:
> I was hoping that the latency introduced by the OS would be lower and
> we would have enough time to satisfy the 10ms requirement. If you are
> right then I guess 10ms response time is not realistic on a standard
> system.

Especially not on Windows ;)

> Anyway we will do some tests first and then try to see what is doable
> and what would be acceptable.

You might be able to handle bursts, but as been mentioned before, you
will have to also look at specialized OSs for real time needs like
that. Which brings us to the question: Is it for things that
somebody's life depends on, or is it for production/etc. where you
will have damages if the 10ms is missed.

In either of those two cases, you are better of with proper real time
OS and software.

However, if you have something that could handle a mis or three every
so often (like  webserver/OLTP/ATM/etc.) then these spikes should not
worry you ;)


>
> On Fri, Jun 4, 2010 at 9:04 AM, Raimo Niskanen
> <raimo+erlang-questions@REDACTED> wrote:
>> Just a naive thought.
>>
>> On any contemporary OS (Windows, Linux, MacOS X, BSD...) you should
>> expect that the OS scheduler might throw you out about 10..20 ms
>> for some arbitrary OS reason...
>>
>> Unless you run some OS with a more real-timeish scheduler,
>> i think there are e.g Linux flavours for this.
>>
>> On Fri, Jun 04, 2010 at 01:45:26AM +0300, Ovidiu Deac wrote:
>>> I'm back with some tests that I've just ran on an Intel Core i5/Linux
>>> 2.6.32/erts v5.7.4.
>>>
>>> The results look more credible since 1000 processes are no longer
>>> spawned in 1microsecond. Also the variance between separate test
>>> executions is much smaller.
>>>
>>> But the problem remains if I spawn 1 milion processes I come across
>>> spawns which take up to 18milliseconds. So again, any idea about how I
>>> could make the VM to be predictible?
>>>
>>> I assume that I should find the level of load that it can take.
>>> Reusing processes as I would have done in other languages sounds like
>>> a bad idea from start. Also I'm thinking I could increase the work
>>> done by one process such that it's not too small but still it fits
>>> inside the initial heap size so garbage collection is not needed.
>>>
>>> Also another thing that I observed is that during the test only about
>>> 50% out of each processor seems to be used. At least this is what the
>>> system monitor says. Any VM flags that would help with this? I tried
>>> +P and +S but without results.
>>>
>>> Ovidiu
>>>
>>>
>>>
>>> On Thu, Jun 3, 2010 at 9:00 PM, Ovidiu Deac <ovidiudeac@REDACTED> wrote:
>>> > Hello everybody,
>>> >
>>> > I'm doing some research in order to figure out if Erlang is suitable
>>> > for an application that we need to implement.
>>> >
>>> > In a few words it's about a server which has a requirement saying "no
>>> > request should take more then 10ms"', measured on the client's
>>> > computer.
>>> >
>>> > So I started a couple of tests for various erlang components to figure
>>> > out how much some operations would take. Partly because I wanted to
>>> > get some figures and partly to exercise my Erlang skills.
>>> >
>>> > First I wrote a profiler module which can measure the execution time
>>> > for various modules which implement set_up,tear_down and test_main.
>>> > Also this profiler can repeat a test a number of times and returns a
>>> > list with all the execution times, the minimum, the maximum and the
>>> > average time. Nothing fancy.
>>> >
>>> > See the attachment for the profiler code and I post below some results
>>> > for a test that I ran on a machine with Intel Core2 Duo/Win7/erlang
>>> > v5.7.5. One test consists in 1000 spawns and is repeated 100 times.
>>> > The times displayed are microseconds.
>>> >
>>> > $ make -C .. && erl +P 200000 -noshell -s profiler run spawn_test 100
>>> > silent 0 1000 -s init stop
>>> > make: Entering directory `/cygdrive/d/work/erlang/finn'
>>> > make: Leaving directory `/cygdrive/d/work/erlang/finn'
>>> > Module to test:spawn_test, Repeats:100, Params=[silent,'0','1000']
>>> > Times elapsed =
>>> > [1,15998,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,15991,1,1,1,1,
>>> >
>>> > 14991,1,1,1,1,15991,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,
>>> >
>>> > 15991,1,1,1,1,1,14989,1,1,1,1,15991,1,1,1,1,15991,1,1,1,
>>> >
>>> > 14993,1,1,1,15993,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,15991,
>>> >                 1,1,1,1,14991,1,1,1,15993,1,1,1,1,15991,1,1,1,1,14991]
>>> > Max us/execution=15998
>>> > Min us/execution=1
>>> > Variation=15997
>>> > Average us/execution=3279.01
>>> >
>>> > The average time for spawning a process is around 3us which is very
>>> > good. It's interesting to see that spawning 1000 processes took
>>> > 1microsecond (!!!) Looks like some bug it the profiler. Also my main
>>> > concern is that from time to time spawning a process takes around 15ms
>>> > which is way too much.
>>> >
>>> > I tried to run erl +P 200000 but the behaviour is the same as without
>>> > this parameter.
>>> >
>>> > Do you have any idea how to make the VM predictible in order to
>>> > satisfy the 10ms requirement?
>>> >
>>> > Thanks in advance,
>>> > Ovidiu
>>> >
>>>
>>> ________________________________________________________________
>>> erlang-questions (at) erlang.org mailing list.
>>> See http://www.erlang.org/faq.html
>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>>
>>
>> --
>>
>> / Raimo Niskanen, Erlang/OTP, Ericsson AB
>>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>