[erlang-questions] Erlang http servers

[Ingela Andin]

Hi!

I feel compelled to contribute on the topic of the hidden inets
application. Once a upon a time someone in Ericsson wrote an http
server that was very much inspired by Apache, which probably seemed
sensible at the time but it does not feel like the erlang way of doing
things. It was almost not document but used extensively by
Ericsson projects, hence very hard to get rid of. Years later several
improvements have been made and  inets httpd server can be run in a
much more erlangish way without Apache config files, even though we
still have to support them. The documentation is much improved but
still quite sparse. And now I think we come to the negative circle. A
lot of pople seem to think HTTP is easy and roll their own and the
Ericsson projects are mostly content and do not have so many new
requirements. As you know HTTP is not Ericsson core business and most
open source users used something else and hence inets have not been
given the highest priority. And  there
has been hardly no user contributions. Most efforts to improve inets
has been to the HTTP client as it is more used. Lesson to be learnt it
is worth doing things properly from the start because other pople may
have to live with your code and bad naming for years and years to
come.

Regards Ingela Erlang/OTP team - Ericsson AB

2012/10/3 Michael Turner <michael.eugene.turner@REDACTED>:
> I recently received a copy of Building Web Applications with Erlang,
> which is almost entirely about yaws. The discussion of inets on this
> thread spurred me to look in the index under "i" -- except, oops, the
> book has no index. It does have an appendix about other webservers and
> frameworks in Erlang -- but inets is not listed there.
>
> "Well-hidden" indeed.
>
> -michael turner
>
>
> On Wed, Oct 3, 2012 at 12:52 AM, Serge Aleynikov <serge@REDACTED> wrote:
>> I agree with Joe that having a design of the HTTP server architecture
>> included in OTP that would offer standardized internal API and allow
>> developers to either replace selected functionality via providing
>> callback modules or swap the HTTP servers easily would be a great plus.
>>
>> Concerning inets, what's really strange is that whenever we see some
>> benchmarks posted comparing performance of different HTTP
>> implementations, inets is never included there, though I would think, it
>> would make more sense in terms of showing how a custom implementation
>> compares to what's included in OTP.  Does it have to do something with
>> what Loïc referred to as inets being well hidden in OTP?  I doubt, but
>> if not that, then what?  There are not even benchmarks comparing yaws to
>> inets, and despite the fact that both offer a rich set of functionality
>> make one wonder was was the idea of writing yaws in the first place
>> given existence of inets' httpd.
>>
>> Diversity is good, but at some point it creates a burden for the user
>> who needs to make a selection of an HTTP server for his project.
>> Instead of solving the application-related problem, now the task becomes
>> studying several implementations to see which one offers a feature-set
>> best suited for the problem being solved, and the answer to that
>> question is not always straightforward.
>>
>> Perhaps a good initiative from the OTP team would be to spec out the
>> open architecture of an HTTP server and welcome feedback from authors of
>> current http libraries, so that the successor of inets would be flexible
>> enough to accommodate building light-weight and heavy-weight HTTP
>> servers based on the supported code base included in OTP?  This would
>> also open a possibility of diversity for library writers
>> replacing/enhancing parts to suit their needs.
>>
>> Serge
>>
>> On 10/1/2012 3:52 PM, Loïc Hoguin wrote:
>>> On 10/01/2012 08:11 PM, Kenneth Lundin wrote:
>>>>  > * OTP has a long release process. Many companies need their fixes to
>>>> be included quickly to avoid maintaining huge branches for their
>>>> changes. They generally agree in return to keep track of changes that
>>>> might happen during the product's development.
>>>> I think we are pretty fast with service releases about every 3rd month
>>>> and updates on the maint branch in git between that.
>>>
>>> But doesn't that imply upgrading the whole of OTP? If it does, that
>>> means more testing and more chances for new issues. If you can build a
>>> release upgrade with one app from one OTP version and the rest from a
>>> previous version, I'd like to know how.
>>>
>>>>  > Note that while not everyone uses inets, most servers do use
>>>> erlang:decode_packet/3 which does most of the job. Cowboy doesn't
>>>> anymore, because it needed backward incompatible changes to
>>>> erlang:decode_packet/3. I didn't ask your thoughts, simply because the
>>>> experiment I did hinted that doing the parsing in full Erlang was a
>>>> better idea: this change results in improved HiPE optimizations and
>>>> better reduction management which results in better latency under load.
>>>>
>>>> Interesting, would be good if you could give us more info on this. The
>>>> http parsing in
>>>> decode_packet is there to improve performance. If it is not ( without
>>>> using HiPE) we should consider removing it!
>>>
>>> Between now and then binaries appeared, got performance improvements,
>>> and considering Cowboy is full binary (no conversion to atom for methods
>>> or header names or anything) and only receives requests, not responses,
>>> that means it can skip many steps that decode_packet/3 can't.
>>>
>>> Binaries also mean you can parse the request line and all the headers
>>> using a single match context which seems to be very fast (good job!) and
>>> I'm guessing is the main reason for improved performance under HiPE (I'm
>>> about to start reading the HiPE papers to understand that better).
>>>
>>>>  >> In Erlang/OTP we have keep strict compatibility with many of our APIs
>>>>  >> which is both a strength and sometimes a burden (when we are not
>>>> happy
>>>>  >> with the old API) but there are mostly ways around that if there is
>>>>  >> strong case for a change.
>>>>  >
>>>>  >
>>>>  > It's a big strength for people looking for stability and a big burden
>>>> for people looking for reactivity. Can't really blame the second group
>>>> for finding solutions to their problems.
>>>>
>>>> I am not blaming people
>>>
>>> I used a word too strong there. :)
>>>
>>>> for implementing their own solution, it is good
>>>> to have a lot to choose from. But I still think the stability is very
>>>> important for the majority of users making systems with a longer life
>>>> cycle than a year.
>>>> Our experience is that we have to be very compatible in order to
>>>> convince our
>>>> users (often big projects) to step up to our latest release.
>>>> By this we can also stop supporting older versions.
>>>
>>> Yes and I wouldn't want that to stop, that's a strong point. In fact
>>> having both ways is great, it means you can make sure the important
>>> parts of your system are stable while the more bleeding-edge parts are
>>> kept, well, bleeding-edge.
>>>
>>>>  >> I think it is of great value to have a relative complete
>>>> functionality
>>>>  >> in Erlang/OTP so that
>>>>  >> the user can implement basic functionality without having to search
>>>>  >> for separate implementations of, with unknown quality and
>>>> completeness
>>>>  >> at least for a beginner.
>>>>  >
>>>>  >
>>>>  > I would prefer Erlang/OTP to focus on the core features and leave the
>>>> satellite features to third parties. OTP has an HTTP server but doesn't
>>>> have proper Unicode support. OTP has OpenGL but doesn't have proper
>>>> timezones support. These 2 examples missing from OTP are some of many
>>>> things that are too complex for most people to implement, and that would
>>>> not benefit from having alternatives.
>>>>
>>>> What is missing when it comes to timezones? Has it been mentioned in
>>>> another thread?
>>>> I think we rely on the OS configuration here.
>>>
>>> Date/time conversion from one timezone to another. For example if your
>>> data is saved with UTC time but the user is in Paris you want to convert
>>> it to Europe/Paris.
>>>
>>> Perhaps it exists, but I haven't found it.
>>>
>>> I think there's a lot of work to be done with regards to localization
>>> and internationalization support, but I haven't had enough time to think
>>> about it in details yet. Unicode (not just Unicode though, but a native
>>> string type abstracting encodings, EEP coming in a few weeks),
>>> timezones, graphical drawing libraries are pretty much what I find lacks
>>> the most for the web side of things. Graphical drawing libraries are
>>> fine outside of OTP for a while (until there's one that can do near
>>> everything and do it well) though.
>>>
>>>
>>>
>>> Thanks for the info on package management and bug tracker. Hope we'll be
>>> involved in the process of creating the former. And thanks for your nice
>>> reply. :)
>>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions