[erlang-questions] gen_server bottleneck

[Garrett Smith]

On Fri, Dec 14, 2012 at 8:22 PM, Saravanan Vijayakumaran
<sarva.v@REDACTED> wrote:
> Dear Max,
>                 There are in fact 10,000 client and server processes. Each
> of them is a gen_server [1][2]. These are just simulation entities which are
> connected by point-to-point channels [3] which are also a simulation
> entities.
>
> The purpose of network simulation is to guide decision making with respect
> to choice of network protocols and parameters in a particular scenario. The
> scenarios are typically too expensive to try out in real networks. Imagine a
> thousand node network placed in different cities.
>
> Regarding discrete event simulation, suppose we want to figure out how many
> runways to have in a new airport. The more the runways the more costly it is
> but it is also will result in less waiting time for arriving/departing
> flights. One way to characterize the runway count vs waiting time tradeoff
> is to use queueing theory. An alternative is to do simulate flight arrivals
> and departures. At any point of time, the simulated world will consists of
> flights in transit, flights waiting to depart, flights waiting to land,
> flights departing and flights landing. In discrete time simulation, the
> state of the simulated world is updated at a regular time step (say every
> minute). Note that this is simulated time and not wall-clock time. An
> alternative which is much faster to execute is discrete event simulation.
> Here there is a queue of simulation events with the earliest events at the
> head of the queue. The events are executed one by one by picking the event
> at the head of the  queue. When a flight departure event is executed, a
> waiting to land event is inserted into the queue having a timestamp equal to
> current time + the transit time of the flight. When the waiting to land
> event is executed, if a runway is free a land event is inserted into the
> queue having timestamp equal to the current time + landing time. If a runway
> is not free, a circle the airport and check again event is inserted into the
> queue having timestamp equal to the current time + circling time. The
> purpose of the simulation queue is to provide causal ordering of the
> simulation events.
>
> In my case, I think it is the simulation queue which is becoming the
> bottleneck. I must add that network simulation falls under the category of
> number-crunching applications which Erlang is not supposed to be suited for.
> But people have been trying to accelerate it in C++-based simulators using
> pthreads. I thought it was worth a try in Erlang given the amount of work
> which has already gone into Erlang SMP.

I've lost track of this thread -- are you still guessing, or have you
spent any time measuring? I looked back and couldn't see any
discussion about e.g. specific processes with high reduction counts,
growing queue sizes, etc.

Garrett