Motorbikes does have problems (was Erlang does have problems)

[Mickael Remond]

Hello Guys,

You know what ?
I have heard that motorbikeis are difficult to drive. I even happen to
know a Nobel Prize that cannot manage to drive his brand new motorbike. 
He his now seriously thinking about buy a Lincoln town car with driver.

And I was about to forgot: hammers are not perfect. I did not manage to
cut my chimney wood with them.

Now a little quizz, what have motorbikes, cars, hammers, saw, Erlang,
ejabberd, you name it, have in common ?
Yes, right they are "tools". Tools have in common:
- To fill a properly defined need,
- To need know-how to make them usefull.

Let's know discuss a thing that I know very well: ejabberd. I work for a
company that develops ejabberd, employs the main ejabberd developers and
have tens of customers running ejabberd.

* Joel Reymont <joelr1@REDACTED> [2006-08-29 18:02:36 +0100]:

> Even the "flagship" Erlang apps have their problems. ejabberd uses  
> tons of memory because strings are being passed around as lists  
> despite being received as binaries from the socket. This is a problem  
> on 32-bit systems as it limits the number of users you can host and  
> it's a bigger problem on 64-bit systems as words are LARGER.
> 
> The ejabberd developers came up with a fix, they are loading expat (C  
> parser) as a driver. They are still using a port per message or per  
> connection (don't remember exactly) and blow through the number of  
> ports normally configured. Yes, you can up the number of ports but a  
> better solution would be to stop using strings and create a shared  
> pool of XML parser ports.

Expat use is not at all related to a memory problem. It is related to
effiency. The XMPP protocol rely on a stream parser. Expat is probably
the most ancien, stable and efficient stream parser around. This is the
reason why it is used in ejabberd. 

> Some high-profile messaging startups are using ejabberd now, although  
> they don't advertise it. They are also considering dropping ejabberd  
> and either going with a commercial implementation or writing their  
> own stuff. I know because I keep in touch with them.

This sentence seems very definitive, but believe me, we probably are in
contact with all the big ejabberd users. We are helping them making
ejabberd working very well for them. We have tens of customers relying
on ejabberd for strategic services: this means that if ejabberd is down
they loose money. ejabberd has been deployed on sites with more than 135
000 simultaneously connected users. Some customers are switching from
commercial implementation to ejabberd. We are working on architectural
changes to support 500 000 simultaneous users.
Believe me, from experience, I know that using a commercial software
will not solve automagically the problem. ejabberd is a wonderfull tool.
At this level, commercial software are also tool and you have to put the
time and effort to make your solution work as well.

However, ejabberd and Erlang work very well when the user consider
them as a tool and not as a product. Running a service with this
amount of users if _HARD_. Read my lips. _Very hard_. You cannot take a
software out of the box deploy it and connects millions of users. Never.
This happens in big hardware vendors commercials, but not in real life.
You have to know the context of your deploiement. You have to know your
tool very well. You have to change it, improve it, optimize it. This is
what we are doing every days and our deployements are working extremely
well.
Reaching this level of knowledge and intimacy with your tool is a lot of
work, but Erlang is very rewarding in this aspect. If you take it as a
tool and are ready to doubt and question your approach and
implementation it can do wonders. 

It happens sometimes however, that some of our contacts:
- absolutly wants to deal with a product, where obviously their business
  model is based on differenciation. How could a product make them
  different ?
- does not have the knowledge to operate the tool.
- does not have the budget to have someone to help them operate the
  tool. This happens sometimes with startup. They put the money into
  hiring people, but cannot then justify to need other external
  know-how.

> Despite the Ericsson AXD 301 advocacy there are no high-profile  
> Erlang deployments that I know about. There should be and we should  
> all know! That is if we want Erlang to become mainstream. On the  
> other hand, why bother?

There are. I know many of them. I have not work of all of them but I
have also learn about them year after year in the Erlang community. But
as it is often explained in the Erlang community, companies having a
strategic advantage with Erlang does not necessarily want it to be
largely known and adopted. Erlang developers generally wants Erlang to
have the bare minimum of growth to progress, but do not care about it
being largely known or even largely used.
Erlang is a competive advantage and as such you should forget what I
have explained you previously in this mail.

Please, Joel, do me a favor. Do not thow away a tool each time someone
has trouble using it.

Cheers,

-- 
Mickaël Rémond 
 http://www.process-one.net/