[erlang-questions] optimizing an asynchronous architecture

[Cliff Moon]

One technique that I was successful in using was timing exit and entry 
points for the various servers I was interested in using a one off 
profiling server to collect the statistics.  Code for the profiling 
server is here:

http://github.com/cliffmoon/dynomite/blob/b621762c1d052f8e4e85f5f4e057ed04389014d2/elibs/dynomite_prof.erl 


And profiling is turned off or on via a macro during the build:

http://github.com/cliffmoon/dynomite/blob/cfe6dd6768d39a268ff2d49203bb2b70e19cba16/include/profile.hrl 


So long as your exit and entry points have unique labels, you can get an 
idea of where time is going.  This helped me figure out that a massive 
amount of time was going to run queue contention in R12B-5, for instance.

Joel Reymont wrote:
> Suppose I'm tasked with building a server that can broadcast a message 
> to 20K subscribers to a topic. No, I do not want to use RabbitMQ since 
> my implementation fits into 2-3 of pages of code.
>
> I'm using gen_server:cast all around and the trip from publishing a 
> message to gen_tcp:send is through 5 gen_servers.
>
> #1 is the "transport process" that receives the message and forwards 
> it to #2, a locally registered "topic manager".
>
> I need the topic manager intermediary because the names of locally 
> registered servers must be atoms and so I cannot locally register a 
> server for each topic. The topic manager gen_server keeps track of 
> locally registered "subscriber servers" (#3) which keep track of 
> subscribers for each topic. Both #2 and #3 use dicts to map topics to 
> processes and keep track of subscriber processes respectively.
>
> #4 is a client proxy that forwards the message back to the transport 
> (#1) which pushes it out to the client socket.
>
> How do I go about cutting the message trip time in half?
>
> I tried using fprof but the output is mostly gen_server:loop, 
> proc_lib:sync_wait, etc. I cannot disable certain functions since 
> fprof always calls erlang:trace(PidSpec, true, ...), that is I can't 
> disable tracing for unwanted.
>
> What I would like, ideally, is to timestamp the message as it goes 
> through the 1-2-3-4-1 pipeline and calculate the deltas once I'm back 
> at #1. Any other suggestions?
>
>     Thanks, Joel
>
> ---
> Mac hacker with a performance bent
> http://www.linkedin.com/in/joelreymont
>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>