[erlang-questions] Slow when using chained gen_server:call's, redesign or optimize?

[Matthew Evans]

Of course you need to  run a profiler such as fprof to see what's going on.
Sounds like a classic head of line blocking problem. Many requests, possibly from processes on different schedulers/cores, all getting serialized on a single gen_server.
The obvious, and maybe non-OTP, answer is to hold some of this state information in a public or protected named ETS table that your clients read from directly. A single gen_server can still own and write to that ETS table. 
Another obvious answer is to provide back-pressure of some kind to prevent clients from requesting data when it is under load.

You might find that a particular infrequent  gen_server:call operation is taking a long time to complete causing a message queue to suddenly grow. You might want to change such an operation from:
handle_call({long_operation,Data},From,State) ->    Rsp = do_lengthy_operation(Data),    {reply, Rsp, State};
to:
handle_call({long_operation,Data},From,State) ->    spawn(fun() ->            Rsp = do_lengthy_operation(Data),            gen_server:reply(Rsp,From)     end),       {noreply, State};

From: goddang@REDACTED
To: erlang-questions@REDACTED
Date: Sat, 28 Jan 2012 00:06:04 +0000
Subject: [erlang-questions] Slow when using chained gen_server:call's,	redesign or optimize?







I'm creating a system where I've ended up with alot of gen_servers that provides a clean interface. When I run this under load I see that the gen_server:call's is becoming a bottleneck.For instance, In a handle_request I might ask an other gen_server to get me a cached object, then ask the database something, then etc...and in some cases I have my-gen_server->cache-gen_server->memcache-client-gen_server as you see it stacks up to alot of steps. I've tried to optimize with deferring gen_server responses and that has given a slight performance improvement but not as drastical as if I for instance bypass one gen_server instance.
Is there a better way to go about this or some smart optimization to do? And FYI, I use gen_server when I need to keep a state of a connection or something so if the answer is to scrap  or reduce the number of gen_servers I will need to keep those connections somewhere else.
Thanks, Dang
 		 	   		  

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120128/74dca894/attachment.htm>