[erlang-questions] Chinese whisper problem

[envelopes envelopes]

I remember there is also messagepack that supports erlang, It has good
performance.

In addition, there is Avro.
On Nov 30, 2011 3:07 PM, "Anthony Molinaro" <anthonym@REDACTED>
wrote:

> On Fri, Nov 25, 2011 at 04:18:09PM +0100, Joe Armstrong wrote:
> > I guess you all know about Chinese whispers.
> >
> > You send a whispered message round a circle and see if you get the same
> > message back.
> >
> > The computer variant is as follows:
> >
> > Send a complex data structure round a circle, and see if you get the same
> > message back.
>
> Sorry I'm late to the party with this one, but was out of town for a bit.
>
> I have used thrift, protobuffs and lwes to do binary multi-language
> message exchange, which is not quite the whispers problem but close.
> They all contain serializers and deserializers and each support different
> sorts of things.
>
> Thrift is mainly meant for RPC, so if all you want is a binary
> serialization
> format, it's a bit tricky (mostly in erlang, less so in java), in that you
> have to create a memory serializer and construct the terms then do a
> term_to_binary on them, to get a binary you could do what you want with.
> On the other hand it supports many languages (include erlang and
> javascript),
> and several protocols (sockets and http being the most common).  It can
> have
> fairly complex structures like maps (which become dict's in erlang), lists,
> different sized ints, floats, etc.
>
> Protobufs also off fairly complex structures (although not maps, you sort
> of
> have to do those yourself with repeated structures which will become lists
> of records, so not quite as useful), as well as strings, ints, floats and
> binary data.  It does not include any RPC based mechanism, and you
> basically
> get serializers/deserializers which get/give binaries, so you can use it
> with whatever you want.  Riak uses this as its faster interface via a
> socket
> and I've implemented server to server communication over HTTP which is
> easy.
>
> LWES (http://lwes.github.com/) is a system and protocol I first developed
> in 1998 and which was open sourced in 2008.  It's mostly meant for passing
> maps in a fire and forget fashion over UDP (or multicast UDP), it's a bit
> bulkier on the wire than thrift and protobuffs as the serialization format
> is self describing (key names are sent along with values).  It also only
> has support in java for floats and arrays (but those will probably make
> their way into the other languages as time permits).  It currently
> has native bindings for C, erlang, java, and .Net, as well as SWIG based
> bindings for perl, python and ruby.  It's a great system for quickly
> passing maps between different programming languages where you can afford
> to loose some data here or there.  Also, unlike thrift/protobuff it does
> not require any precompilation of modules.  While we don't have a
> javascript
> version yet, we may be developing one soon (although it most likely won't
> be able to do UDP from a javascript client which is unfortunate).  LWES
> is extremely stable and has been in production systems serving trillions
> of events for almost 13 years (every penny ever made by
> Goto/Overture/Yahoo's
> search advertising system was via lwes).
>
> I've used all three of these systems in production web systems so can
> attest
> that they all work and at large scale (ten's of thousands to hundred's of
> thousands of requests per second).
>
> I've not used Bert, BSON, Avro, MessagePack or Etch all of which do
> similiar things but offer slightly different feature sets.
>
> Hope that helps,
>
> -Anthony
>
> --
> ------------------------------------------------------------------------
> Anthony Molinaro                           <anthonym@REDACTED>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111130/0c33bfc5/attachment.htm>