[erlang-questions] xmerl troubles

[Bob Ippolito]

Are you using Mac OS X? The ulimit probably needs to be adjusted to
accommodate the number of sockets you want to open. The default is
notoriously low.

In any case, the error that's happening is of course not in the code you've
shared.

On Saturday, June 9, 2012, Alex Shneyderman wrote:

> I am logging problematic XML (there are no files involved). And diffing it
> with the XML received in one of the first parses (that actually parse fine)
> does not reveal significant difference in structure, all match except for
> but
> internal data within the elements.
>
> If I leave the number of agents at around 290 system works for hours.
> Parsing
> messages back and forth. 290 seems to be my threshold which is too low for
> my
> experiments :-(
>
> Thanks.
>
> PS: CGS, sorry for the duplicate my first reply was not to all :-(
>
> On Sun, Jun 10, 2012 at 12:04 AM, CGS <cgsmcmlxxv@REDACTED <javascript:;>>
> wrote:
> > Hi Alex,
> >
> > The problem may be also coming from the network transfer, not only from
> > xmlerl/xmlrpc. From the error you provided, {{badmatch,{error,emfile}},
> it
> > looks like the XML file wasn't correctly transmitted/received and that
> may
> > come from traffic congestion as well. Of course, that if the XML file
> exists
> > and it contains no error.
> >
> > CGS
> >
> >
> >
> >
> > On Sat, Jun 9, 2012 at 11:42 PM, Alex Shneyderman <
> a.shneyderman@REDACTED <javascript:;>>
> > wrote:
> >>
> >> Hi, there!
> >>
> >> I am using xmerl (via xmlrpc) and it seems to be having problems under
> >> heavy load.
> >>
> >> I have a system that tries to simulate load on our system. I have to
> >> shoot a lot of little
> >> XML RPC-like messages towards the system and parse responses from it.
> >> Each simulation
> >> runs in its own process. Parsing of the payload is done with xmlrpc
> >> which in turn uses xmerl.
> >>
> >> When load on the system is small around 250 agents, things work well.
> >> When I up the number
> >> pseudo-agents to around 290 xmerl starts throwing a hissy fit. A
> >> message that is similar in
> >> all respects but the data inside elements to the previously parsed
> >> messages become problematic.
> >>
> >> Code is nothing extra-ordinary and looks like this:
> >>
> >> case xmlrpc_decode:payload(XmlStr) of
> >>        {ok, {response, [{struct, ResponseParams}]}}->
> >>                lager:debug("XML-RPC payload parsed:~n {response,
> [{struct,
> >> ~p}]}
> >> ~n", [ResponseParams]),
> >>                {ok,{login_response,ResponseParams},[RestOfBinary]};
> >>        {error, Reason} ->
> >>                lager:debug("XML-RPC payload could not be parsed:~n~p~n",
> >> [Reason]),
> >>                {error,Reason}
> >> end;
> >>
> >> the exception I get:
> >>
> >> 2012-06-09 23:02:25.037 [debug]
> >> <0.1877.0>@cc_agent_utils:parse_login_response_xml:105 XML-RPC payload
> >> could not be parsed:
> >>
> >>
> {{badmatch,{error,emfile}},[{xmerl_scan,int_string,3,[{file,"xmerl_scan.erl"},{line,291}]},{xmerl_scan,string,2,[{file,"xmerl_scan.erl"},{line,286}]},{xmlrpc_decode,payload,1,[{file,"src/xmlrpc_decode.erl"},{line,39}]},{cc_agent_utils,parse_login_response_xml,2,[{file,"src/cc_agent_utils.erl"},{line,100}]},{cc_agent_utils,parse,2,[{file,"src/cc_agent_utils.erl"},{line,46}]},{cc_agent_srv,process_login_response,2,[{file,"src/cc_agent_srv.erl"},{line,192}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}
> >> 2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_srv:terminate:147
> >> Terminate was called for agent '9ba8de71-6821-4b76-b2a9-c4e0d1b2ae82'
> >> talking on local port 64001
> >>
> >> Anyone had similar experience with xmerl? Is there any setting I can
> >> tweak to make xmerl behave nicely?
> >>
> >> Thanks,
> >> Alex.
> >> _______________________________________________
> >> erlang-questions mailing list
> >> erlang-questions@REDACTED <javascript:;>
> >> http://erlang.org/mailman/listinfo/erlang-questions
> >
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED <javascript:;>
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120609/58095c73/attachment.htm>