Beginner Screen Scaping, and Auto-login/input on 3rd party web app?

JETkoten jetkoten@REDACTED
Fri Dec 24 15:38:31 CET 2010


Hi Everyone,

I'm (very) new to Erlang, and hoping to get some basic experience with it.

I've made some initial steps in learning Ruby, and was in the bookstore 
the other day and thought I'd look at what's new in Ruby books. They 
were listed in "Other Languages" and the bookstore didn't have anything 
I hadn't seen before for Ruby so I thought since it was such a small 
selection I'd just grab and browse all the books for the languages I 
wasn't familiar with.  After reading a little bit about Haskell, Common 
Lisp and Scala, I found Joe Armstrong's Erlang book and was very 
interested. I've since viewed hours of Erlang videos online, and am 
reading as many general tutorials as I can find.

I really learn best by doing something I'm interested in. I have a "pet 
project" that I've been working on in Ruby that I would also like to 
implement now in Erlang.

Here it is:

I have a large personal library of books and find that I don't need many 
of them anymore. I'd like to create a program that will help me manage 
my online sales on a marketplace site, by automatically checking 
competing sellers' prices at a set time interval an then logging into 
their website and adjusting my prices according to a formula I'd set 
based on the other prices.

I started it in Ruby already, and got the basic price lookup 
functionality working for one title at a time only by manually inputting 
the URL and then using Nokogiri to screen scrape the price info from 
parsed CSS3 tags on the page, and then Mechanize to login and change my 
prices for me.

I did a Google search on Erlang "screen scraping" and saw some options:

www_tools, Yaws parser, xmerl, mochiweb

However, none of the posts that suggest those are less than 2 years 
old... which is the best/easiest way, and/or are there newer, better 
options now?

Approaching Erlang, I have absolutely no idea how to get started at 
getting the same functionality working as I have in Ruby. I am hoping 
that someone here can point me in the right direction. If possible, I 
think it'd be neat to do it all concurrently too, but I don't want to 
make it too hard on myself if that'd bring up too much complexity for a 
beginner like me.

Any ideas?

Thanks in advance,
Jack


More information about the erlang-questions mailing list