[erlang-questions] Chinese whisper problem

[邢森]

protobufs, thrift and other "human-readable binary protocol" is the
best choice,small
message packets,decoding and encoding fast speed is fast.

2011/12/1 envelopes envelopes <sunwood360@REDACTED>

> I remember there is also messagepack that supports erlang, It has good
> performance.
>
> In addition, there is Avro.
> On Nov 30, 2011 3:07 PM, "Anthony Molinaro" <anthonym@REDACTED>
> wrote:
>
>> On Fri, Nov 25, 2011 at 04:18:09PM +0100, Joe Armstrong wrote:
>> > I guess you all know about Chinese whispers.
>> >
>> > You send a whispered message round a circle and see if you get the same
>> > message back.
>> >
>> > The computer variant is as follows:
>> >
>> > Send a complex data structure round a circle, and see if you get the
>> same
>> > message back.
>>
>> Sorry I'm late to the party with this one, but was out of town for a bit.
>>
>> I have used thrift, protobuffs and lwes to do binary multi-language
>> message exchange, which is not quite the whispers problem but close.
>> They all contain serializers and deserializers and each support different
>> sorts of things.
>>
>> Thrift is mainly meant for RPC, so if all you want is a binary
>> serialization
>> format, it's a bit tricky (mostly in erlang, less so in java), in that you
>> have to create a memory serializer and construct the terms then do a
>> term_to_binary on them, to get a binary you could do what you want with.
>> On the other hand it supports many languages (include erlang and
>> javascript),
>> and several protocols (sockets and http being the most common).  It can
>> have
>> fairly complex structures like maps (which become dict's in erlang),
>> lists,
>> different sized ints, floats, etc.
>>
>> Protobufs also off fairly complex structures (although not maps, you sort
>> of
>> have to do those yourself with repeated structures which will become lists
>> of records, so not quite as useful), as well as strings, ints, floats and
>> binary data.  It does not include any RPC based mechanism, and you
>> basically
>> get serializers/deserializers which get/give binaries, so you can use it
>> with whatever you want.  Riak uses this as its faster interface via a
>> socket
>> and I've implemented server to server communication over HTTP which is
>> easy.
>>
>> LWES (http://lwes.github.com/) is a system and protocol I first developed
>> in 1998 and which was open sourced in 2008.  It's mostly meant for passing
>> maps in a fire and forget fashion over UDP (or multicast UDP), it's a bit
>> bulkier on the wire than thrift and protobuffs as the serialization format
>> is self describing (key names are sent along with values).  It also only
>> has support in java for floats and arrays (but those will probably make
>> their way into the other languages as time permits).  It currently
>> has native bindings for C, erlang, java, and .Net, as well as SWIG based
>> bindings for perl, python and ruby.  It's a great system for quickly
>> passing maps between different programming languages where you can afford
>> to loose some data here or there.  Also, unlike thrift/protobuff it does
>> not require any precompilation of modules.  While we don't have a
>> javascript
>> version yet, we may be developing one soon (although it most likely won't
>> be able to do UDP from a javascript client which is unfortunate).  LWES
>> is extremely stable and has been in production systems serving trillions
>> of events for almost 13 years (every penny ever made by
>> Goto/Overture/Yahoo's
>> search advertising system was via lwes).
>>
>> I've used all three of these systems in production web systems so can
>> attest
>> that they all work and at large scale (ten's of thousands to hundred's of
>> thousands of requests per second).
>>
>> I've not used Bert, BSON, Avro, MessagePack or Etch all of which do
>> similiar things but offer slightly different feature sets.
>>
>> Hope that helps,
>>
>> -Anthony
>>
>> --
>> ------------------------------------------------------------------------
>> Anthony Molinaro                           <anthonym@REDACTED>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>


-- 
我的blog
http://blog.sina.com.cn/xingsen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111201/dfdb1a89/attachment.htm>