[erlang-questions] Re: How to make the Erlang VM predictible when spawning processes?

Mon Jun 7 00:13:16 CEST 2010

After the tests we did we have two options. Either to reconsider the
requirements and accept something like <10ms in 99% of the calls or to
do some tests with a more RT-capable os. We'll think about it next
week.
Thanks for your answers.

On Sat, Jun 5, 2010 at 4:43 PM, Hendrik Visage <hvjunk@REDACTED> wrote:
> On 6/4/10, Ovidiu Deac <ovidiudeac@REDACTED> wrote:
>> I was hoping that the latency introduced by the OS would be lower and
>> we would have enough time to satisfy the 10ms requirement. If you are
>> right then I guess 10ms response time is not realistic on a standard
>> system.
>
> Especially not on Windows ;)
>
>> Anyway we will do some tests first and then try to see what is doable
>> and what would be acceptable.
>
> You might be able to handle bursts, but as been mentioned before, you
> will have to also look at specialized OSs for real time needs like
> that. Which brings us to the question: Is it for things that
> somebody's life depends on, or is it for production/etc. where you
> will have damages if the 10ms is missed.
>
> In either of those two cases, you are better of with proper real time
> OS and software.
>
> However, if you have something that could handle a mis or three every
> so often (like  webserver/OLTP/ATM/etc.) then these spikes should not
> worry you ;)
>
>
>>
>> On Fri, Jun 4, 2010 at 9:04 AM, Raimo Niskanen
>> <raimo+erlang-questions@REDACTED> wrote:
>>> Just a naive thought.
>>>
>>> On any contemporary OS (Windows, Linux, MacOS X, BSD...) you should
>>> expect that the OS scheduler might throw you out about 10..20 ms
>>> for some arbitrary OS reason...
>>>
>>> Unless you run some OS with a more real-timeish scheduler,
>>> i think there are e.g Linux flavours for this.
>>>
>>> On Fri, Jun 04, 2010 at 01:45:26AM +0300, Ovidiu Deac wrote:
>>>> I'm back with some tests that I've just ran on an Intel Core i5/Linux
>>>> 2.6.32/erts v5.7.4.
>>>>
>>>> The results look more credible since 1000 processes are no longer
>>>> spawned in 1microsecond. Also the variance between separate test
>>>> executions is much smaller.
>>>>
>>>> But the problem remains if I spawn 1 milion processes I come across
>>>> spawns which take up to 18milliseconds. So again, any idea about how I
>>>> could make the VM to be predictible?
>>>>
>>>> I assume that I should find the level of load that it can take.
>>>> Reusing processes as I would have done in other languages sounds like
>>>> a bad idea from start. Also I'm thinking I could increase the work
>>>> done by one process such that it's not too small but still it fits
>>>> inside the initial heap size so garbage collection is not needed.
>>>>
>>>> Also another thing that I observed is that during the test only about
>>>> 50% out of each processor seems to be used. At least this is what the
>>>> system monitor says. Any VM flags that would help with this? I tried
>>>> +P and +S but without results.
>>>>
>>>> Ovidiu
>>>>
>>>>
>>>>
>>>> On Thu, Jun 3, 2010 at 9:00 PM, Ovidiu Deac <ovidiudeac@REDACTED> wrote:
>>>> > Hello everybody,
>>>> >
>>>> > I'm doing some research in order to figure out if Erlang is suitable
>>>> > for an application that we need to implement.
>>>> >
>>>> > In a few words it's about a server which has a requirement saying "no
>>>> > request should take more then 10ms"', measured on the client's
>>>> > computer.
>>>> >
>>>> > So I started a couple of tests for various erlang components to figure
>>>> > out how much some operations would take. Partly because I wanted to
>>>> > get some figures and partly to exercise my Erlang skills.
>>>> >
>>>> > First I wrote a profiler module which can measure the execution time
>>>> > for various modules which implement set_up,tear_down and test_main.
>>>> > Also this profiler can repeat a test a number of times and returns a
>>>> > list with all the execution times, the minimum, the maximum and the
>>>> > average time. Nothing fancy.
>>>> >
>>>> > See the attachment for the profiler code and I post below some results
>>>> > for a test that I ran on a machine with Intel Core2 Duo/Win7/erlang
>>>> > v5.7.5. One test consists in 1000 spawns and is repeated 100 times.
>>>> > The times displayed are microseconds.
>>>> >
>>>> > $ make -C .. && erl +P 200000 -noshell -s profiler run spawn_test 100
>>>> > silent 0 1000 -s init stop
>>>> > make: Entering directory `/cygdrive/d/work/erlang/finn'
>>>> > make: Leaving directory `/cygdrive/d/work/erlang/finn'
>>>> > Module to test:spawn_test, Repeats:100, Params=[silent,'0','1000']
>>>> > Times elapsed =
>>>> > [1,15998,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,15991,1,1,1,1,
>>>> >
>>>> > 14991,1,1,1,1,15991,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,
>>>> >
>>>> > 15991,1,1,1,1,1,14989,1,1,1,1,15991,1,1,1,1,15991,1,1,1,
>>>> >
>>>> > 14993,1,1,1,15993,1,1,1,1,14991,1,1,1,1,15991,1,1,1,1,15991,
>>>> >                 1,1,1,1,14991,1,1,1,15993,1,1,1,1,15991,1,1,1,1,14991]
>>>> > Max us/execution=15998
>>>> > Min us/execution=1
>>>> > Variation=15997
>>>> > Average us/execution=3279.01
>>>> >
>>>> > The average time for spawning a process is around 3us which is very
>>>> > good. It's interesting to see that spawning 1000 processes took
>>>> > 1microsecond (!!!) Looks like some bug it the profiler. Also my main
>>>> > concern is that from time to time spawning a process takes around 15ms
>>>> > which is way too much.
>>>> >
>>>> > I tried to run erl +P 200000 but the behaviour is the same as without
>>>> > this parameter.
>>>> >
>>>> > Do you have any idea how to make the VM predictible in order to
>>>> > satisfy the 10ms requirement?
>>>> >
>>>> > Thanks in advance,
>>>> > Ovidiu
>>>> >
>>>>
>>>> ________________________________________________________________
>>>> erlang-questions (at) erlang.org mailing list.
>>>> See http://www.erlang.org/faq.html
>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>>>
>>>
>>> --
>>>
>>> / Raimo Niskanen, Erlang/OTP, Ericsson AB
>>>
>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>
>>
>