[erlang-questions] Erlang http servers
Mon Oct 1 17:38:52 CEST 2012
On 10/01/2012 04:30 PM, Kenneth Lundin wrote:
> I agree with Serge that it is somewhat unfortunate that there are so
> many initiatives
> where people write their own implementation of something we already
> have included in
> the Erlang/OTP distribution and doing this without even suggesting or
> asking us if we could
> incorporate some missing feature or improve the current implementation
> in various ways.
It happens a lot. People tend to review available options, and if none
fits their needs, write something themselves without consulting any of
the authors. Sometimes they're right to do so, because the answer
they'll get is obvious, and sometimes not.
There's a few reasons this happens for this specific case:
* HTTP is simple, it's easier to go ahead and experiment on a new
project than on an existing one.
* If the experiment succeeds and the implementation has to differ
significantly from other projects, it's not worth backporting since
you'll just run into a wall of backward compatibility requirements.
* OTP has a long release process. Many companies need their fixes to be
included quickly to avoid maintaining huge branches for their changes.
They generally agree in return to keep track of changes that might
happen during the product's development.
* Inets is seriously well hidden. I only heard about it after releasing
Cowboy. I knew about Yaws/Mochiweb/Misultin but not the inets HTTP
server. Many people I talked to didn't seem to know it existed.
* Sometimes removing features is an improvement. Inets is big, it's only
natural that lightweight alternatives appear. Same happened to Apache,
and it's no surprise that Inets is quite similar to Apache from a user
point of view.
Note that while not everyone uses inets, most servers do use
erlang:decode_packet/3 which does most of the job. Cowboy doesn't
anymore, because it needed backward incompatible changes to
erlang:decode_packet/3. I didn't ask your thoughts, simply because the
experiment I did hinted that doing the parsing in full Erlang was a
better idea: this change results in improved HiPE optimizations and
better reduction management which results in better latency under load.
> In Erlang/OTP we have keep strict compatibility with many of our APIs
> which is both a strength and sometimes a burden (when we are not happy
> with the old API) but there are mostly ways around that if there is
> strong case for a change.
It's a big strength for people looking for stability and a big burden
for people looking for reactivity. Can't really blame the second group
for finding solutions to their problems.
> We still actively develop inets with its http client and server and I
> am not sure that
> all the reasons to develop alternatives still exist.
But like you said the level of support for applications included in OTP
vary widely. It is also not clearly indicated which parts are the focus
of recent works (there is a small roadmap for the most important stuff
but that's about it), nor can it be deduced from reading tickets in a
bug tracker or similar collaboration tool.
Which brings me to a very important point: please start using a public
bug tracker! It doesn't have to be 100% public (as I'm sure you have
some issues that are internal only) but it would help a lot if people
could see the existing issues instead of stumbling upon the same bugs
again and again. It probably also would help with contributions.
> I think it is of great value to have a relative complete functionality
> in Erlang/OTP so that
> the user can implement basic functionality without having to search
> for separate implementations of, with unknown quality and completeness
> at least for a beginner.
I would prefer Erlang/OTP to focus on the core features and leave the
satellite features to third parties. OTP has an HTTP server but doesn't
have proper Unicode support. OTP has OpenGL but doesn't have proper
timezones support. These 2 examples missing from OTP are some of many
things that are too complex for most people to implement, and that would
not benefit from having alternatives.
Let's take another example. While taking a quick glance at the modules
included in OTP I discovered there is a GD implementation in the percept
application. I'm pretty sure most people didn't know it was there, as
they usually end up using a NIF. It's quite unexpected that it's in OTP.
Finding things in OTP unexpectedly is a quite common occurence, though.
If the user had a package management application (perhaps working
similarly to rebar's deps, = per project/release, no site-wide install
by default) along with a package index website, it could quite easily
find what he needs. It wouldn't be of unknown quality or completeness,
simply because adding a comment and rating system to a package index
website is fairly simple to do.
Plus, he can make an opinion for himself, instead of just using whatever
someone decided for him. You might include a project in OTP that fits
your criteria for quality, but it might be crap for someone else. A more
democratic system where people choose what they use based on other
users' feedback and ratings is not only a lot more powerful but also
allows to bring some light on some more obscure projects that are
sometimes quite hard to find armed only with Google.
It makes sense to have a single choice for big corporate environments,
not so much for the rest of the world.
More information about the erlang-questions