[erlang-questions] HTML Entity Refs and xmerl

[Bertil Karlsson]

Hi,

whatever entity references you depend on when parsing an XML document 
with xmerl you have to provide a DTD with those entities defined. 
However I think the time difference in your example is unacceptable and 
maybe indicating a bug in xmerl. I will take a closer look at this.

/Bertil

Caoyuan wrote:
> According to http://www.erlang.org/pipermail/erlang-bugs/2007-May/000345.html,
> currently xmerl_scan only recognizes the very limited set of entity
> references. In brief, if you try to xmerl:scan xml text that includes
> standard HTML entity refs, such as nbsp, iexcl, pound, frac14, etc.
> you'll encounter something like:
>
> 16> edoc:file("exprecs.erl").
> 2670- fatal: {unknown_entity_ref,nbsp}
> 2580- fatal: error_scanning_entity_ref
> exprecs.erl, in module header: at line 28: error in XML parser: {fatal,
>                          {error_scanning_entity_ref,
>                              {file,file_name_unknown},
>                              {line,86},
>                              {col,18}}}.
> ** exited: error **
>
> Ulf Wiger said:
> ... I realize that xmerl can be customized with a rules function
> which, for example, can handle entity references...
>
> So I take a try by writing a piece of code (html_entity_refs.erl) to
> parse a HTML entity ref DTD file to ets rules, then:
>
> xmerl_scan:string(XmlText, [{rules, html_entity_refs:get_xmerl_rules()}]).
>
> Yes, this works. But for a 3MB testing file, the parsing took about 30 seconds.
>
> How about convert these entity refs to utf-8 chars first, then apply
> xmerl_scan to it?
> I wrote another piece of code, and now:
>
> xmerl_scan:string(html_entity_refs:decode_for_xml(XmlText)).
>
> This time, the decoding+parsing time is about 5 seconds, it's 6 times
> faster than ets rules solution.
>
> The html_entity_refs.erl can be got from:
> http://caoyuan.googlecode.com/svn/trunk/erlang/html_entity_refs.erl
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>