Beginner Screen Scaping, and Auto-login/input on 3rd party web app?
Fri Dec 24 15:38:31 CET 2010
I'm (very) new to Erlang, and hoping to get some basic experience with it.
I've made some initial steps in learning Ruby, and was in the bookstore
the other day and thought I'd look at what's new in Ruby books. They
were listed in "Other Languages" and the bookstore didn't have anything
I hadn't seen before for Ruby so I thought since it was such a small
selection I'd just grab and browse all the books for the languages I
wasn't familiar with. After reading a little bit about Haskell, Common
Lisp and Scala, I found Joe Armstrong's Erlang book and was very
interested. I've since viewed hours of Erlang videos online, and am
reading as many general tutorials as I can find.
I really learn best by doing something I'm interested in. I have a "pet
project" that I've been working on in Ruby that I would also like to
implement now in Erlang.
Here it is:
I have a large personal library of books and find that I don't need many
of them anymore. I'd like to create a program that will help me manage
my online sales on a marketplace site, by automatically checking
competing sellers' prices at a set time interval an then logging into
their website and adjusting my prices according to a formula I'd set
based on the other prices.
I started it in Ruby already, and got the basic price lookup
functionality working for one title at a time only by manually inputting
the URL and then using Nokogiri to screen scrape the price info from
parsed CSS3 tags on the page, and then Mechanize to login and change my
prices for me.
I did a Google search on Erlang "screen scraping" and saw some options:
www_tools, Yaws parser, xmerl, mochiweb
However, none of the posts that suggest those are less than 2 years
old... which is the best/easiest way, and/or are there newer, better
Approaching Erlang, I have absolutely no idea how to get started at
getting the same functionality working as I have in Ruby. I am hoping
that someone here can point me in the right direction. If possible, I
think it'd be neat to do it all concurrently too, but I don't want to
make it too hard on myself if that'd bring up too much complexity for a
beginner like me.
Thanks in advance,
More information about the erlang-questions