[erlang-questions] BERT vs protobuf in the erlang world

[Mark Geib]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

We almost always end up with a wrapper message def that includes a
"type" enum field and a "message" bytes field that holds the actual
encoded message of interest. However, in the implementation it's a bit
ugly due to the dual encodes and decodes required, and it makes the
messages a few bytes larger of course.

Mark.

On 08/23/2011 05:22 PM, Anthony Molinaro wrote:
> Protobuf is not what I would call self describing, if you have
> something like
>
> message foo { required int32 bar = 1; }
>
> message foo2 { optional int32 bar2 = 1; }
>
> and create an instance of each and then send it remotely, you can
> not distinguish between the 2 messages, unless the remote side know
> when one you mean to send.
>
> Here's an example in erlang
>
> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2]
> [async-threads:0] [hipe] [kernel-poll:false]
>
> Eshell V5.8.4 (abort with ^G) 1> rr(foo_pb). [foo,foo2] 2>
> foo_pb:encode(#foo{bar=42}). <<"\b*">> 3>
> foo_pb:encode(#foo2{bar2=42}). <<"\b*">>
>
> To me a self-describing protocol is something which also encoded
> the names of the messages and fields, so you could send it through
> to the other side and it would be unambiguous. An example of this
> is a system I wrote many years ago called lwes (www.lwes.org),
> which does not require code generation because messages are self
> describing.
>
> Now this does mean messages are longer, but sometimes that is
> okay.
>
> -Anthony
>
> On Tue, Aug 23, 2011 at 01:17:12PM -0700, Jon Watte wrote:
>> Protobuf is fully self-described, and fully forwards/backwards
>> compatible, in that it can carry through "unknown" members of a
>> structure, including the semantic of the fields. ASN1 is
>> generally not fully self-described, because it only describes
>> type, not semantic, for each field.
>>
>> Sincerely,
>>
>> jw
>>
>> -- Americans might object: there is no way we would sacrifice our
>> living standards for the benefit of people in the rest of the
>> world. Nevertheless, whether we get there willingly or not, we
>> shall soon have lower consumption rates, because our present
>> rates are unsustainable.
>>
>>
>>
>> On Tue, Aug 23, 2011 at 3:03 AM, Vincent de Phily <
>> vincent.dephily@REDACTED> wrote:
>>
>>> On Monday 22 August 2011 11:31:44 Mike Oxford wrote:
>>>> BERT is pretty much a "known good quantity" but protobuf is
>>>> more efficient on the wire.
>>>>
>>>> Anyone have thoughts on which to go with?
>>>>
>>>> Also, along the BERT side, does anyone have recommendations
>>>> on AS3 and Java implementations of the marshallers?
>>>>
>>>> The downside(s) to protobuf - hard IDLs and you have to build
>>>> the project and generators. The downside(s) to BERT - less
>>>> efficient on the wire (verbosity.)
>>>
>>> Also take a look at bjson, and one of my favourite : msgpack.
>>> BERT is rather unique in supporting atoms, which *nice* and can
>>> reduce encoded size a lot (but that's less significant if you
>>> compress). For encoded size, I have yet to see anything beat
>>> UPER (asn1).
>>>
>>> A few things to consider : * Bench size and speed with a mockup
>>> of your data : different formats are better for different types
>>> of data. Add compression to your tests. * Think about format
>>> extensibility. It's easy enough with self-described formats
>>> like bert or msgpack, but gets more complicated with protobuf
>>> or asn1. * The initial hurdle of dealing with a grammar and
>>> generators is well paid off by having data validation and nicer
>>> api (mostly via #records{}) later on. * If you need to speak
>>> with the outside world, self-describing formats will make your
>>> life easyer. -- Vincent de Phily
>>> _______________________________________________
>>> erlang-questions mailing list erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>
>> _______________________________________________ erlang-questions
>> mailing list erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>
>

- -- 
Principal Engineer
Cheyenne Software Engineering
mark.geib@REDACTED / 35-215

PGP fingerprint:6DFC 389D 9796 0188 92E5 58F5 34C5 6B47 D091 76FD
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5UOPcACgkQNMVrR9CRdv0+hACfXy1jEg0Lisbj5pPqFQt4tK8d
be8AnRCDPLKzLZE1cpfsUp8LMUfv84A+
=WqFv
-----END PGP SIGNATURE-----