xmerl, ets,(mnisia) Too many db tables

[chandru]

Hi,

On 10/10/05, Ulf Wiger (AL/EAB) <ulf.wiger@REDACTED> wrote:
>
> xmerl creates an ets table to keep DTD rules etc.
> They should be deleted in the cleanup/1 function in xmerl_scan.erl.
>
> A possibility is that you're starting more XML processing than you
> can finish, and that you eventually get too many rules tables as a
> result of that.
>
> A quick fix is to set the environment variable ERL_MAX_DB_TABLES
> to a higher value (default is 1400). You can set this to a much higher
> value than that -- say 32000, or higher still. It will raise the ceiling, but
> you could still run out of tables, of course.

You can also specify an ETS table to use so that a new ETS table is
not created every time you parse some XML. This is what we did in our
app.

%% This is done in the init function of a gen_server. Create an ETS
table which xmerl can use.
ets:new(esaspub_xmerl_rules, [named_table, set, public]).

%% We also parse the DTD once in the init function.
catch xmerl_scan:string("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
 	                             "<!DOCTYPE APITK_Request SYSTEM
\"http://www.t-mobile.co.uk/dtd/apitk_V1.0.dtd\">\n"
 	                             "<APITK_Request></APITK_Request>",
 	                             [{fetch_fun, dtd_fetch_fun()},
 	                              {rules, esaspub_xmerl_rules}]),

dtd_fetch_fun() ->
 	     fun(_, State) ->
 	             {ok, {string, ?TPG_REQUEST_DTD}, State}
 	     end.

And then every time we wanted to parse some XML

xmerl_scan:string(Xml, [{rules, esaspub_xmerl_rules}, {fetch_fun,
dtd_fetch_fun(), undefined}])

That made sure only one ETS table was used. Did you forget about this
feature Uffe :-)

cheers
Chandru

>
>
> -----Original Message-----
> From: owner-erlang-questions@REDACTED [mailto:owner-erlang-questions@REDACTED]On Behalf Of Sanjaya Vitharana
> Sent: den 10 oktober 2005 06:03
> To: erlang-questions@REDACTED
> Subject: xmerl, ets,(mnisia) Too many db tables
>
>
> Hi ...!!!
>
> I got the below error after running my system about 30-45 min. But I'm sure that I'm not using any database in my code. But the error referes to ** Too many db tables **. I have try to search the community replies & found that another person get the similer error when using mnisia. But in my case I don't use mnesia at all.
>
> So ... I dig down to my xml server & parser. Found that in xmerl it self uses exacly the same code as error shows (i.e Tab = ets:new(rules, [set, public]), )
>
> This occurs when I try to increase the load of xml parssing.
>     i.e. I have set my dialout thread to check the dialout calls every second (earlier was 15 seconds), so each check should pass the xml file (or parse_error.xml ). Also each of that dialout call need to pass 5-6 xml files to initiate a call.
>
> Except to that dialout thing, all the call flow running on the xml & all xml files should pass. So the xml processing is high in the system.
>
> As I have wrote about, the system works properly about 30-45 min without any problem & later gives the below error.
>