[erlang-questions] Erlang http servers

[Serge Aleynikov]

I agree with Joe that having a design of the HTTP server architecture
included in OTP that would offer standardized internal API and allow
developers to either replace selected functionality via providing
callback modules or swap the HTTP servers easily would be a great plus.

Concerning inets, what's really strange is that whenever we see some
benchmarks posted comparing performance of different HTTP
implementations, inets is never included there, though I would think, it
would make more sense in terms of showing how a custom implementation
compares to what's included in OTP.  Does it have to do something with
what Loïc referred to as inets being well hidden in OTP?  I doubt, but
if not that, then what?  There are not even benchmarks comparing yaws to
inets, and despite the fact that both offer a rich set of functionality
make one wonder was was the idea of writing yaws in the first place
given existence of inets' httpd.

Diversity is good, but at some point it creates a burden for the user
who needs to make a selection of an HTTP server for his project.
Instead of solving the application-related problem, now the task becomes
studying several implementations to see which one offers a feature-set
best suited for the problem being solved, and the answer to that
question is not always straightforward.

Perhaps a good initiative from the OTP team would be to spec out the
open architecture of an HTTP server and welcome feedback from authors of
current http libraries, so that the successor of inets would be flexible
enough to accommodate building light-weight and heavy-weight HTTP
servers based on the supported code base included in OTP?  This would
also open a possibility of diversity for library writers
replacing/enhancing parts to suit their needs.

Serge

On 10/1/2012 3:52 PM, Loïc Hoguin wrote:
> On 10/01/2012 08:11 PM, Kenneth Lundin wrote:
>>  > * OTP has a long release process. Many companies need their fixes to
>> be included quickly to avoid maintaining huge branches for their
>> changes. They generally agree in return to keep track of changes that
>> might happen during the product's development.
>> I think we are pretty fast with service releases about every 3rd month
>> and updates on the maint branch in git between that.
> 
> But doesn't that imply upgrading the whole of OTP? If it does, that
> means more testing and more chances for new issues. If you can build a
> release upgrade with one app from one OTP version and the rest from a
> previous version, I'd like to know how.
> 
>>  > Note that while not everyone uses inets, most servers do use
>> erlang:decode_packet/3 which does most of the job. Cowboy doesn't
>> anymore, because it needed backward incompatible changes to
>> erlang:decode_packet/3. I didn't ask your thoughts, simply because the
>> experiment I did hinted that doing the parsing in full Erlang was a
>> better idea: this change results in improved HiPE optimizations and
>> better reduction management which results in better latency under load.
>>
>> Interesting, would be good if you could give us more info on this. The
>> http parsing in
>> decode_packet is there to improve performance. If it is not ( without
>> using HiPE) we should consider removing it!
> 
> Between now and then binaries appeared, got performance improvements,
> and considering Cowboy is full binary (no conversion to atom for methods
> or header names or anything) and only receives requests, not responses,
> that means it can skip many steps that decode_packet/3 can't.
> 
> Binaries also mean you can parse the request line and all the headers
> using a single match context which seems to be very fast (good job!) and
> I'm guessing is the main reason for improved performance under HiPE (I'm
> about to start reading the HiPE papers to understand that better).
> 
>>  >> In Erlang/OTP we have keep strict compatibility with many of our APIs
>>  >> which is both a strength and sometimes a burden (when we are not
>> happy
>>  >> with the old API) but there are mostly ways around that if there is
>>  >> strong case for a change.
>>  >
>>  >
>>  > It's a big strength for people looking for stability and a big burden
>> for people looking for reactivity. Can't really blame the second group
>> for finding solutions to their problems.
>>
>> I am not blaming people
> 
> I used a word too strong there. :)
> 
>> for implementing their own solution, it is good
>> to have a lot to choose from. But I still think the stability is very
>> important for the majority of users making systems with a longer life
>> cycle than a year.
>> Our experience is that we have to be very compatible in order to
>> convince our
>> users (often big projects) to step up to our latest release.
>> By this we can also stop supporting older versions.
> 
> Yes and I wouldn't want that to stop, that's a strong point. In fact
> having both ways is great, it means you can make sure the important
> parts of your system are stable while the more bleeding-edge parts are
> kept, well, bleeding-edge.
> 
>>  >> I think it is of great value to have a relative complete
>> functionality
>>  >> in Erlang/OTP so that
>>  >> the user can implement basic functionality without having to search
>>  >> for separate implementations of, with unknown quality and
>> completeness
>>  >> at least for a beginner.
>>  >
>>  >
>>  > I would prefer Erlang/OTP to focus on the core features and leave the
>> satellite features to third parties. OTP has an HTTP server but doesn't
>> have proper Unicode support. OTP has OpenGL but doesn't have proper
>> timezones support. These 2 examples missing from OTP are some of many
>> things that are too complex for most people to implement, and that would
>> not benefit from having alternatives.
>>
>> What is missing when it comes to timezones? Has it been mentioned in
>> another thread?
>> I think we rely on the OS configuration here.
> 
> Date/time conversion from one timezone to another. For example if your
> data is saved with UTC time but the user is in Paris you want to convert
> it to Europe/Paris.
> 
> Perhaps it exists, but I haven't found it.
> 
> I think there's a lot of work to be done with regards to localization
> and internationalization support, but I haven't had enough time to think
> about it in details yet. Unicode (not just Unicode though, but a native
> string type abstracting encodings, EEP coming in a few weeks),
> timezones, graphical drawing libraries are pretty much what I find lacks
> the most for the web side of things. Graphical drawing libraries are
> fine outside of OTP for a while (until there's one that can do near
> everything and do it well) though.
> 
> 
> 
> Thanks for the info on package management and bug tracker. Hope we'll be
> involved in the process of creating the former. And thanks for your nice
> reply. :)
>