[erlang-questions] Load balancing/multiplexing rpc calls amongst Erlang Nodes

Tue Oct 23 11:52:45 CEST 2012

Thank you so much. let me try that. But is it not possible to have a
non-random method ? one that is intelligent and fair on all the nodes

  <http://www.linkedin.com/pub/muzaaya-joshua/39/2ba/202>
Designed with WiseStamp -
<http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350985854695&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10>Get
yours<http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350985854695&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10>

On Tue, Oct 23, 2012 at 10:45 AM, Paul Peregud <paulperegud@REDACTED>wrote:

> > find the erlang random generator skewing results making around 50/60
> requests hitting one Node while others are waiting
>
> This is unusual. I've seen PRNG from random module skewing the results,
> but never to such extent. Please check if it is properly seeded
> (random:seed/0 seeds with predefined constant).
>
> If seeding is done properly, then you may want to consider switching to
> crypto:rand_uniform/2. It's a bit slower, but it produces random numbers of
> quality better then enough for purpose of load balancing.
>
>
> On Tue, Oct 23, 2012 at 7:22 AM, Joshua Muzaaya <joshmuza@REDACTED>wrote:
>
>> Yes,i tries using the random method, but because requests are so
>> frequently many, you find the erlang random generator skewing results
>> making around 50/60 requests hitting one Node while others are waiting.
>> Another thing is that, i am not using gen_servers at the Web Server layer.
>> I am using yaws web server and for each connection, yaws spawns a process,
>> this process communicates with Mnesia Nodes to query for data. But the
>> connections are so many and i wanted to scale the application horizontally,
>> adding more web servers and more mnesia Nodes. I came to think of a load
>> balancing middle ware, abstracting my processes from knowing where the call
>> has hit ( i.e on what mnesia node the call has hit). This middle ware
>> ensures that requests are load balanced across my Mnesia DBs.
>>
>> That is the background of the problem. Its a real-time Web Notification
>> system, plugged into a major intranet Management System. However, clients
>> are many, and yaws is sustaining 30,000 concurrent connections at low
>> peaks. I am a software engineer in one of the telecommunications companies
>> in Africa. I keep running into a few memory problems on single node yaws
>> server, so i need ti add more web servers to assist. Also, mnesia sometime
>> will get *** Too many DB Tables ** when requests are too many and too
>> frequent. I changed everything to use dirty operations and when i by-passed
>> the transaction manager, things improved a bit.
>>
>> I need fellow erlangers to think of a load balancing algorithm in such a
>> situation. Do you think a process dictionary like GPROC would be so useful
>> ? i was kinda thinking about it last night, but i wonder how i would apply
>> it in this case.
>>
>> Having one gen_server to decide where the request may go, might alos slow
>> down the application as all requests will have to go through that
>> gen_server.
>>
>>   <http://www.linkedin.com/pub/muzaaya-joshua/39/2ba/202>
>> Designed with WiseStamp -
>> <http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350969121119&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10>Get
>> yours<http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350969121119&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10>
>>
>>
>>
>> On Tue, Oct 23, 2012 at 2:53 AM, Yogish Baliga <baliga@REDACTED> wrote:
>>
>>> One option is to run proxy gen_server on each Mnesia box and register
>>> these gen_server pids with pg2. Now you can do load balancing on pg2
>>> processes based on message queue length as described here
>>>
>>> http://dev.lethain.com/load-balancing-across-erlang-process-groups/
>>>
>>> When I used this method in my last project in Erlang, it gave better
>>> result than normal round robin. Under very low load, all requests were
>>> redirected to single Mnesia instance.
>>>
>>> http://dev.lethain.com/load-balancing-across-erlang-process-groups/
>>>
>>> -- baliga
>>>
>>>
>>> On Mon, Oct 22, 2012 at 2:22 PM, Paul Peregud <paulperegud@REDACTED>wrote:
>>>
>>>> May you specify why load balancing should be based on time and can not
>>>> be random? Have you implemented random load balancing? Has it proved
>>>> to be insufficient?
>>>>
>>>> On Mon, Oct 22, 2012 at 9:51 AM, muzaaya_joshua <joshmuza@REDACTED>
>>>> wrote:
>>>> > Building from this question (
>>>> http://stackoverflow.com/q/5339329/431620 ),
>>>> > imagine an application with N Erlang Web Servers, and N/2 Mnesia
>>>> Database
>>>> > Nodes. The set up is such that the Web Servers, each, runs on its own
>>>> > hardware server (say HP DL385), and each Mnesia Instance, runs on its
>>>> own
>>>> > hardware Server as well.
>>>> >
>>>> > Web Servers make rpc:call/4 calls to the back end (the Mnesia DB
>>>> Servers).
>>>> > The Data is all replicated across all the Mnesia instances. Now, you
>>>> want to
>>>> > have the calls being made to the Database servers, MULTIPLEXED, more
>>>> > precisely ( by TIME), on each Web Server, so that some kind of LOAD
>>>> > BALANCING is attained.
>>>> >
>>>> > If Web Server A makes a connection to Mnesia Instance 3, it cannot
>>>> make the
>>>> > next connection to the same Instance. All Database Nodes need to be
>>>> kept
>>>> > busy and not having any one of them idle while the others are
>>>> working. The
>>>> > Load balancing Algorithm should not be random, but should be aimed at
>>>> > balancing the load on the Database Servers.
>>>> >
>>>> > Qn 1: Come up with your load balancing strategy, in such a situation.
>>>> Also,
>>>> > please show with some sample illustrative code, how you would
>>>> implement this
>>>> > strategy.
>>>> >
>>>> > Qn 2: If a Mnesia Instance goes down, how would your load balance
>>>> Algorithm
>>>> > adapt to the changes in the cluster ?
>>>> >
>>>> > Qn 3: Is there any Erlang library aimed at load balancing of Erlang
>>>> Servers
>>>> > working within the same system, and calling each other via rpc:call/4
>>>> ?
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > View this message in context:
>>>> http://erlang.2086793.n4.nabble.com/Load-balancing-multiplexing-rpc-calls-amongst-Erlang-Nodes-tp4655158.html
>>>> > Sent from the Erlang Questions mailing list archive at Nabble.com.
>>>> > _______________________________________________
>>>> > erlang-questions mailing list
>>>> > erlang-questions@REDACTED
>>>> > http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Paul Peregud
>>>> +48602112091
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>
>>>
>>
>>
>> --
>> *Muzaaya Joshua
>> Systems Engineer
>> +256774115170*
>> *"Through it all, i have learned to trust in Jesus. To depend upon His
>> Word"
>> *
>>
>>
>>
>>
>
>
> --
> Best regards,
> Paul Peregud
> +48602112091
>

-- 
*Muzaaya Joshua
Systems Engineer
+256774115170*
*"Through it all, i have learned to trust in Jesus. To depend upon His Word"
*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121023/4b8431fa/attachment.htm>