[erlang-questions] xmerl troubles

[Alex Shneyderman]

I am logging problematic XML (there are no files involved). And diffing it
with the XML received in one of the first parses (that actually parse fine)
does not reveal significant difference in structure, all match except for but
internal data within the elements.

If I leave the number of agents at around 290 system works for hours. Parsing
messages back and forth. 290 seems to be my threshold which is too low for my
experiments :-(

Thanks.

PS: CGS, sorry for the duplicate my first reply was not to all :-(

On Sun, Jun 10, 2012 at 12:04 AM, CGS <cgsmcmlxxv@REDACTED> wrote:
> Hi Alex,
>
> The problem may be also coming from the network transfer, not only from
> xmlerl/xmlrpc. From the error you provided, {{badmatch,{error,emfile}}, it
> looks like the XML file wasn't correctly transmitted/received and that may
> come from traffic congestion as well. Of course, that if the XML file exists
> and it contains no error.
>
> CGS
>
>
>
>
> On Sat, Jun 9, 2012 at 11:42 PM, Alex Shneyderman <a.shneyderman@REDACTED>
> wrote:
>>
>> Hi, there!
>>
>> I am using xmerl (via xmlrpc) and it seems to be having problems under
>> heavy load.
>>
>> I have a system that tries to simulate load on our system. I have to
>> shoot a lot of little
>> XML RPC-like messages towards the system and parse responses from it.
>> Each simulation
>> runs in its own process. Parsing of the payload is done with xmlrpc
>> which in turn uses xmerl.
>>
>> When load on the system is small around 250 agents, things work well.
>> When I up the number
>> pseudo-agents to around 290 xmerl starts throwing a hissy fit. A
>> message that is similar in
>> all respects but the data inside elements to the previously parsed
>> messages become problematic.
>>
>> Code is nothing extra-ordinary and looks like this:
>>
>> case xmlrpc_decode:payload(XmlStr) of
>>        {ok, {response, [{struct, ResponseParams}]}}->
>>                lager:debug("XML-RPC payload parsed:~n {response, [{struct,
>> ~p}]}
>> ~n", [ResponseParams]),
>>                {ok,{login_response,ResponseParams},[RestOfBinary]};
>>        {error, Reason} ->
>>                lager:debug("XML-RPC payload could not be parsed:~n~p~n",
>> [Reason]),
>>                {error,Reason}
>> end;
>>
>> the exception I get:
>>
>> 2012-06-09 23:02:25.037 [debug]
>> <0.1877.0>@cc_agent_utils:parse_login_response_xml:105 XML-RPC payload
>> could not be parsed:
>>
>> {{badmatch,{error,emfile}},[{xmerl_scan,int_string,3,[{file,"xmerl_scan.erl"},{line,291}]},{xmerl_scan,string,2,[{file,"xmerl_scan.erl"},{line,286}]},{xmlrpc_decode,payload,1,[{file,"src/xmlrpc_decode.erl"},{line,39}]},{cc_agent_utils,parse_login_response_xml,2,[{file,"src/cc_agent_utils.erl"},{line,100}]},{cc_agent_utils,parse,2,[{file,"src/cc_agent_utils.erl"},{line,46}]},{cc_agent_srv,process_login_response,2,[{file,"src/cc_agent_srv.erl"},{line,192}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,597}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}
>> 2012-06-09 23:02:25.037 [debug] <0.1877.0>@cc_agent_srv:terminate:147
>> Terminate was called for agent '9ba8de71-6821-4b76-b2a9-c4e0d1b2ae82'
>> talking on local port 64001
>>
>> Anyone had similar experience with xmerl? Is there any setting I can
>> tweak to make xmerl behave nicely?
>>
>> Thanks,
>> Alex.
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>
>