[erlang-questions] slow message processing in cluster

[Dustin Sallings]

	Hi *,

	I've got a cluster of five nodes across two machines and I'm finding  
my message queues are getting really full on my remote nodes.

	I'm pushing some rather large data around.  This application  
consists of a process that reads a file and translates them into http  
requests (many are posts with binary contents), a process to issue  
the requests, and a process to handle the responses.

	I'm using http:request (async) to issue my requests from a process  
whose job is to time and track requests and send the responses back  
to the ``master'' process for dispatch.

	The problem I'm experiencing is in this process.  The source code is  
available here:

http://hg.west.spy.net/hg/erlang/httpload/file/fc80675fc138/src/ 
http_requestor.erl

	When this process is running on a remote node, the message queue  
grows fairly rapidly, but it's fine on the local node.  I assumed  
this might be because of the response, so I pushed that out into yet  
another process (locally queue the outbound message so I can quickly  
get back to my input queue), but it had no effect.

	This is what ni() in an interactive session shows me:

<5253.14588.0>        net_kernel:spawn_func/6            8024355  
23848914 7332
http_requestor        gen:wait_resp_mon/3                     35
<5253.14589.0>        erlang:apply/2                        2584    
156940    0
http_response_sender  http_requestor:response_sender_lo        1

	I named the processes for clarity (note that this is showing the  
version I altered to separate local vs. remote node sending -- that  
detail seemed to not matter).

	Examining the message queue, I see good mix of request and response  
messages.  I am observing responses coming back from every node, even  
though the queues are always bigger each time I look.

	Any clues as to how I might get stuff moving along a bit more smoothly?

-- 
Dustin Sallings


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070814/ab08ee1d/attachment.htm>