<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD><TITLE></TITLE>
<STYLE>
body, table, tr, td, p {margin: 0px 0px 0px 0px}
.bgtabl {BACKGROUND-REPEAT: no-repeat}
</STYLE>
</HEAD>
<BODY bgProperties="fixed" bgcolor="#FFFFFF">
Hi there!<br><br>Good advice Ke Han.<br>And here is a patch for Joe "www_tools" to fix some parsing problems.<br><br>cheers<br>Younès<br><br><br>----Message d'origine----
<br>>Copie à: Erlang-Questions Mailing List <erlang-questions@erlang.org>
<br>>De: ke han <ke.han@redstarling.com>
<br>>Sujet: Re: Screen scraping
<br>>Date: Wed, 30 Aug 2006 11:24:36 +0800
<br>>A: Joel Reymont <joelr1@gmail.com>
<br>>
<br>>Joel,
<br>>How about jungerl's www_tools ??
<br>>
<br>>Here is a snippet of its example code to show you how easy it is to
<br>>tokenize an HTML stream or file and harvest element of interest:
<br>>
<br>>%%********************************
<br>>file(File) ->
<br>> Toks = html_tokenise:file2toks(File),
<br>> analyse(Toks).
<br>>
<br>>analyse(Toks) ->
<br>> Hrefs = [H || {tagStart, "a", L} <- Toks, {"href", H} <- L],
<br>> Images1 = [S || {tagStart, "img", L} <- Toks, {"src", S} <- L],
<br>> Images2 = [S || {tagStart, "body", L} <- Toks, {"background", S}
<br>><- L],
<br>> {remove_duplicates(Hrefs), remove_duplicates(Images1++Images2)}.
<br>>%%********************************
<br>>
<br>>ke han
<br>>
<br>>
<br>>
<br>>On Aug 30, 2006, at 5:46 AM, Joel Reymont wrote:
<br>>
<br>>> Does anyone have tools for screen scraping with Erlang?
<br>>>
<br>>> It's a combination of HTTP client with parsing and regexp-ing
<br>>> through HTML. Ruby has nice tools for this like hpricot and scrAPI
<br>>> and they parse HTML into a structure and let you query for elements
<br>>> based on their class, id, name, etc.
<br>>>
<br>>> Thanks, Joel
<br>>>
<br>>> --
<br>>> http://wagerlabs.com/
<br>>>
<br>>>
<br>>>
<br>>>
<br>>>
<br>>
<br>>
</joelr1@gmail.com></ke.han@redstarling.com></erlang-questions@erlang.org><hints id="hah_hints"></hints>
</body></html>