Erlang, Strings, Mnesia?
Tue Apr 13 15:51:20 CEST 2004
On Thu, 8 Apr 2004, Jimmie Houchin wrote:
> I am trying to understand whether or not Erlang is right or a good
> option for a website I am building.
> Currently I am processing 422,000 files with 6.6gb of text to place in a
> database. I will have to regularly parse and process large volumes of
> textual data.
> The database will initially have 4-5 million rows/objects.
> This data in turn will be searchable. A prime purpose of the data.
> On this page of the Mnesia User Guide it says:
> #On the other hand, Mnesia may not be appropriate with the following
> #types of applications:
> # * Programs that process plain text or binary data files
> I am used to Squeak and Python and native string support.
> I understand well Erlang's strengths. I don't understand where it is weak.
> An example.
> Would Erlang be suitable for a Google type site?
> An Amazon? An eBay?
Based on what you have said so far there is no way of answering your
Before doing *any* major bit of programming I always try to do the
1) Identify the most difficult problem(s) to be solved
2) Prototype solutions the the problems raised in 1)
3) Measure performance of prototype
When this has converged you can go ahead and write the program.
Premature optimizations and pre-conceived ideas about what is and is
not efficient is a sure fired receipt for disaster.
I would suggest you write a little parser and build a prototype
*before* you make any decisions.
Personally (now I'm *very* biased here) I'd go for yaws - why?
Yaws got concurrency right.
I know nothing about your application but there is a heck of a
difference between having say 1000 users who just request a web page
and then break the connection and 100,000 users who are logged in to
chat systems and remain connected for hours at a time.
Yaws is great for handling large numbers of persistent connections -
I would also (counter to what some other people have said) be very
cautious about using *different* components in your system. If you
use Erlang AND mySQL you have to learn to use two things - not one.
The mySQL way of thinking (SQL) is NOT the Erlang way of thinking -
and you will get an impedance mismatch between the two ways of
Whenever I see projects that use A and B and C and D ... I judder in
horror. Each new thing causes an explosion in complexity << and yes any
problem no matter how trivial can be made nightmarishly difficult just
by using Java, mySQL, Cobra, ASN.1, SOAP, WSDL, ...>>
Projects that I complete in a small amount of time are almost
invariably monolingual. Do *everything* in Erlang or C or whatever.
IMHO - as soon as you start mixing things you get into a mess - as
Wirth said (paraphrased) - mastery of one language is better than
incomplete knowledge of several.
So here's what I'd do:
- Identify the HARD bits
- do small prototypes of the hard bits
- if fast enough HOORAY
if not fast enough ask this group why
> Any wisdom, advise welcome and appreciated.
> Jimmie Houchin
More information about the erlang-questions