[erlang-questions] xmerl troubles

[Alex Shneyderman]

Hi, there!

I am using xmerl (via xmlrpc) and it seems to be having problems under
heavy load.

I have a system that tries to simulate load on our system. I have to
shoot a lot of little
XML RPC-like messages towards the system and parse responses from it.
Each simulation
runs in its own process. Parsing of the payload is done with xmlrpc
which in turn uses xmerl.

When load on the system is small around 250 agents, things work well.
When I up the number
pseudo-agents to around 290 xmerl starts throwing a hissy fit. A
message that is similar in
all respects but the data inside elements to the previously parsed
messages become problematic.

Code is nothing extra-ordinary and looks like this:

case xmlrpc_decode:payload(XmlStr) of
	{ok, {response, [{struct, ResponseParams}]}}->
		lager:debug("XML-RPC payload parsed:~n {response, [{struct, ~p}]}
~n", [ResponseParams]),
		{ok,{login_response,ResponseParams},[RestOfBinary]};
	{error, Reason} ->
		lager:debug("XML-RPC payload could not be parsed:~n~p~n", [Reason]),
		{error,Reason}
end;

the exception I get:

2012-06-09 23:02:25.037 [debug]
<0.1877.0>@cc_agent_utils:parse_login_response_xml:105 XML-RPC payload
could not be parsed:
{{badmatch,{error,emfile}},[{xmerl_scan,int_string,3,[{file,"xmerl_scan.erl"},{line,291}]},{xmerl_scan,string,2,[{file,"xmerl_scan.erl"},{line,286}]},{xmlrpc_decode,payload,1,[{file,"src/xmlrpc_decode.erl"},{line,39}]},{cc_agent_utils,parse_login_response_xml,2,[{file,"src/cc_agent_utils.erl"},{line,100}]},{cc_agent_utils,parse,2,[{file,"src/cc_agent_utils.erl"},{line,46}]},{cc_agent_srv,process_login_response,2,[{file,"src/cc_agent_srv.erl"},{line,192}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}
2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_srv:terminate:147
Terminate was called for agent '9ba8de71-6821-4b76-b2a9-c4e0d1b2ae82'
talking on local port 64001

Anyone had similar experience with xmerl? Is there any setting I can
tweak to make xmerl behave nicely?

Thanks,
Alex.