Are you using Mac OS X? The ulimit probably needs to be adjusted to accommodate the number of sockets you want to open. The default is notoriously low.<div><br></div><div>In any case, the error that's happening is of course not in the code you've shared. <span></span><br>
<br>On Saturday, June 9, 2012, Alex Shneyderman wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I am logging problematic XML (there are no files involved). And diffing it<br>
with the XML received in one of the first parses (that actually parse fine)<br>
does not reveal significant difference in structure, all match except for but<br>
internal data within the elements.<br>
<br>
If I leave the number of agents at around 290 system works for hours. Parsing<br>
messages back and forth. 290 seems to be my threshold which is too low for my<br>
experiments :-(<br>
<br>
Thanks.<br>
<br>
PS: CGS, sorry for the duplicate my first reply was not to all :-(<br>
<br>
On Sun, Jun 10, 2012 at 12:04 AM, CGS <<a href="javascript:;" onclick="_e(event, 'cvml', 'cgsmcmlxxv@gmail.com')">cgsmcmlxxv@gmail.com</a>> wrote:<br>
> Hi Alex,<br>
><br>
> The problem may be also coming from the network transfer, not only from<br>
> xmlerl/xmlrpc. From the error you provided, {{badmatch,{error,emfile}}, it<br>
> looks like the XML file wasn't correctly transmitted/received and that may<br>
> come from traffic congestion as well. Of course, that if the XML file exists<br>
> and it contains no error.<br>
><br>
> CGS<br>
><br>
><br>
><br>
><br>
> On Sat, Jun 9, 2012 at 11:42 PM, Alex Shneyderman <<a href="javascript:;" onclick="_e(event, 'cvml', 'a.shneyderman@gmail.com')">a.shneyderman@gmail.com</a>><br>
> wrote:<br>
>><br>
>> Hi, there!<br>
>><br>
>> I am using xmerl (via xmlrpc) and it seems to be having problems under<br>
>> heavy load.<br>
>><br>
>> I have a system that tries to simulate load on our system. I have to<br>
>> shoot a lot of little<br>
>> XML RPC-like messages towards the system and parse responses from it.<br>
>> Each simulation<br>
>> runs in its own process. Parsing of the payload is done with xmlrpc<br>
>> which in turn uses xmerl.<br>
>><br>
>> When load on the system is small around 250 agents, things work well.<br>
>> When I up the number<br>
>> pseudo-agents to around 290 xmerl starts throwing a hissy fit. A<br>
>> message that is similar in<br>
>> all respects but the data inside elements to the previously parsed<br>
>> messages become problematic.<br>
>><br>
>> Code is nothing extra-ordinary and looks like this:<br>
>><br>
>> case xmlrpc_decode:payload(XmlStr) of<br>
>> {ok, {response, [{struct, ResponseParams}]}}-><br>
>> lager:debug("XML-RPC payload parsed:~n {response, [{struct,<br>
>> ~p}]}<br>
>> ~n", [ResponseParams]),<br>
>> {ok,{login_response,ResponseParams},[RestOfBinary]};<br>
>> {error, Reason} -><br>
>> lager:debug("XML-RPC payload could not be parsed:~n~p~n",<br>
>> [Reason]),<br>
>> {error,Reason}<br>
>> end;<br>
>><br>
>> the exception I get:<br>
>><br>
>> 2012-06-09 23:02:25.037 [debug]<br>
>> <0.1877.0>@cc_agent_utils:parse_login_response_xml:105 XML-RPC payload<br>
>> could not be parsed:<br>
>><br>
>> {{badmatch,{error,emfile}},[{xmerl_scan,int_string,3,[{file,"xmerl_scan.erl"},{line,291}]},{xmerl_scan,string,2,[{file,"xmerl_scan.erl"},{line,286}]},{xmlrpc_decode,payload,1,[{file,"src/xmlrpc_decode.erl"},{line,39}]},{cc_agent_utils,parse_login_response_xml,2,[{file,"src/cc_agent_utils.erl"},{line,100}]},{cc_agent_utils,parse,2,[{file,"src/cc_agent_utils.erl"},{line,46}]},{cc_agent_srv,process_login_response,2,[{file,"src/cc_agent_srv.erl"},{line,192}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}<br>
>> 2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_srv:terminate:147<br>
>> Terminate was called for agent '9ba8de71-6821-4b76-b2a9-c4e0d1b2ae82'<br>
>> talking on local port 64001<br>
>><br>
>> Anyone had similar experience with xmerl? Is there any setting I can<br>
>> tweak to make xmerl behave nicely?<br>
>><br>
>> Thanks,<br>
>> Alex.<br>
>> _______________________________________________<br>
>> erlang-questions mailing list<br>
>> <a href="javascript:;" onclick="_e(event, 'cvml', 'erlang-questions@erlang.org')">erlang-questions@erlang.org</a><br>
>> <a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
><br>
><br>
_______________________________________________<br>
erlang-questions mailing list<br>
<a href="javascript:;" onclick="_e(event, 'cvml', 'erlang-questions@erlang.org')">erlang-questions@erlang.org</a><br>
<a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
</blockquote></div>