> find the erlang random generator skewing results making around 50/60 requests hitting one Node while others are waiting<br><br>This is unusual. I've seen PRNG from random module skewing the results, but never to such extent. Please check if it is properly seeded (random:seed/0 seeds with predefined constant).<br>
<br>If seeding is done properly, then you may want to consider switching to crypto:rand_uniform/2. It's a bit slower, but it produces random numbers of quality better then enough for purpose of load balancing.<br><br>
<div class="gmail_quote">On Tue, Oct 23, 2012 at 7:22 AM, Joshua Muzaaya <span dir="ltr"><<a href="mailto:joshmuza@gmail.com" target="_blank">joshmuza@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Yes,i tries using the random method, but because requests are so frequently many, you find the erlang random generator skewing results making around 50/60 requests hitting one Node while others are waiting. Another thing is that, i am not using gen_servers at the Web Server layer. I am using yaws web server and for each connection, yaws spawns a process, this process communicates with Mnesia Nodes to query for data. But the connections are so many and i wanted to scale the application horizontally, adding more web servers and more mnesia Nodes. I came to think of a load balancing middle ware, abstracting my processes from knowing where the call has hit ( i.e on what mnesia node the call has hit). This middle ware ensures that requests are load balanced across my Mnesia DBs.<br>
<br>That is the background of the problem. Its a real-time Web Notification system, plugged into a major intranet Management System. However, clients are many, and yaws is sustaining 30,000 concurrent connections at low peaks. I am a software engineer in one of the telecommunications companies in Africa. I keep running into a few memory problems on single node yaws server, so i need ti add more web servers to assist. Also, mnesia sometime will get *** Too many DB Tables ** when requests are too many and too frequent. I changed everything to use dirty operations and when i by-passed the transaction manager, things improved a bit. <br>
<br>I need fellow erlangers to think of a load balancing algorithm in such a situation. Do you think a process dictionary like GPROC would be so useful ? i was kinda thinking about it last night, but i wonder how i would apply it in this case. <br>
<br>Having one gen_server to decide where the request may go, might alos slow down the application as all requests will have to go through that gen_server. <br><br><div><div style="font-size:13.3px;font-family:Verdana,Arial,Helvetica,sans-serif">
<div style="max-width:469px;padding:0.5em 0 0.5em"> <a href="http://www.linkedin.com/pub/muzaaya-joshua/39/2ba/202" target="_blank"> <img src="" style="border:none" border="0"> </a></div>
<div style="border-top:1px solid #eeeeee;margin-top:17px;padding-top:2px;font-size:75%"><a style="color:#6f6f6f;text-decoration:none" href="http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350969121119&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10" target="_blank"><span style="color:#6f6f6f">Designed with WiseStamp - </span></a><a style="color:#3f48cc;text-decoration:underline" href="http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350969121119&promo=10&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_10" target="_blank"><span style="color:#3f48cc">Get yours</span></a> <img src="" height="1" width="1"></div>
<img src="" height="1" width="1"></div></div><div class="HOEnZb"><div class="h5"><br><br><div class="gmail_quote">On Tue, Oct 23, 2012 at 2:53 AM, Yogish Baliga <span dir="ltr"><<a href="mailto:baliga@gmail.com" target="_blank">baliga@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">One option is to run proxy gen_server on each Mnesia box and register these gen_server pids with pg2. Now you can do load balancing on pg2 processes based on message queue length as described here<div>
<br></div><div><a href="http://dev.lethain.com/load-balancing-across-erlang-process-groups/" target="_blank">http://dev.lethain.com/load-balancing-across-erlang-process-groups/</a></div>
<div><br></div><div>When I used this method in my last project in Erlang, it gave better result than normal round robin. Under very low load, all requests were redirected to single Mnesia instance.</div><div><br></div><div>
<a href="http://dev.lethain.com/load-balancing-across-erlang-process-groups/" target="_blank">http://dev.lethain.com/load-balancing-across-erlang-process-groups/</a></div><span><font color="#888888"><div><br>
</div><div>-- baliga</div></font></span><div><div><div><br><div><br><div class="gmail_quote">
On Mon, Oct 22, 2012 at 2:22 PM, Paul Peregud <span dir="ltr"><<a href="mailto:paulperegud@gmail.com" target="_blank">paulperegud@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
May you specify why load balancing should be based on time and can not<br>
be random? Have you implemented random load balancing? Has it proved<br>
to be insufficient?<br>
<br>
On Mon, Oct 22, 2012 at 9:51 AM, muzaaya_joshua <<a href="mailto:joshmuza@gmail.com" target="_blank">joshmuza@gmail.com</a>> wrote:<br>
> Building from this question ( <a href="http://stackoverflow.com/q/5339329/431620" target="_blank">http://stackoverflow.com/q/5339329/431620</a> ),<br>
> imagine an application with N Erlang Web Servers, and N/2 Mnesia Database<br>
> Nodes. The set up is such that the Web Servers, each, runs on its own<br>
> hardware server (say HP DL385), and each Mnesia Instance, runs on its own<br>
> hardware Server as well.<br>
><br>
> Web Servers make rpc:call/4 calls to the back end (the Mnesia DB Servers).<br>
> The Data is all replicated across all the Mnesia instances. Now, you want to<br>
> have the calls being made to the Database servers, MULTIPLEXED, more<br>
> precisely ( by TIME), on each Web Server, so that some kind of LOAD<br>
> BALANCING is attained.<br>
><br>
> If Web Server A makes a connection to Mnesia Instance 3, it cannot make the<br>
> next connection to the same Instance. All Database Nodes need to be kept<br>
> busy and not having any one of them idle while the others are working. The<br>
> Load balancing Algorithm should not be random, but should be aimed at<br>
> balancing the load on the Database Servers.<br>
><br>
> Qn 1: Come up with your load balancing strategy, in such a situation. Also,<br>
> please show with some sample illustrative code, how you would implement this<br>
> strategy.<br>
><br>
> Qn 2: If a Mnesia Instance goes down, how would your load balance Algorithm<br>
> adapt to the changes in the cluster ?<br>
><br>
> Qn 3: Is there any Erlang library aimed at load balancing of Erlang Servers<br>
> working within the same system, and calling each other via rpc:call/4 ?<br>
><br>
><br>
<span><font color="#888888">><br>
> --<br>
> View this message in context: <a href="http://erlang.2086793.n4.nabble.com/Load-balancing-multiplexing-rpc-calls-amongst-Erlang-Nodes-tp4655158.html" target="_blank">http://erlang.2086793.n4.nabble.com/Load-balancing-multiplexing-rpc-calls-amongst-Erlang-Nodes-tp4655158.html</a><br>
> Sent from the Erlang Questions mailing list archive at Nabble.com.<br>
> _______________________________________________<br>
> erlang-questions mailing list<br>
> <a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
> <a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
<br>
<br>
--<br>
Best regards,<br>
Paul Peregud<br>
<a href="tel:%2B48602112091" value="+48602112091" target="_blank">+48602112091</a><br>
_______________________________________________<br>
erlang-questions mailing list<br>
<a href="mailto:erlang-questions@erlang.org" target="_blank">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
</font></span></blockquote></div><br></div></div>
</div></div></blockquote></div><br><br clear="all"><br></div></div><span class="HOEnZb"><font color="#888888">-- <br>*Muzaaya Joshua<br>Systems Engineer<br><a href="tel:%2B256774115170" value="+256774115170" target="_blank">+256774115170</a>*<br>
*"Through it all, i have learned to trust in Jesus. To depend upon His Word"<br>*<br><br><br>
<br>
</font></span></blockquote></div><br><br clear="all"><br>-- <br>Best regards,<br>Paul Peregud<br>+48602112091<br>