[erlang-questions] BERT vs protobuf in the erlang world

[Anthony Molinaro]

Protobuf is not what I would call self describing, if you have something
like

message foo {
  required int32 bar = 1;
}

message foo2 {
  optional int32 bar2 = 1;
}

and create an instance of each and then send it remotely, you can not
distinguish between the 2 messages, unless the remote side know when
one you mean to send.

Here's an example in erlang

Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0]
[hipe] [kernel-poll:false]

Eshell V5.8.4  (abort with ^G)
1> rr(foo_pb).   
[foo,foo2]
2> foo_pb:encode(#foo{bar=42}).
<<"\b*">>
3> foo_pb:encode(#foo2{bar2=42}).
<<"\b*">>

To me a self-describing protocol is something which also encoded the names
of the messages and fields, so you could send it through to the other side
and it would be unambiguous.  An example of this is a system I wrote many
years ago called lwes (www.lwes.org), which does not require code generation
because messages are self describing.

Now this does mean messages are longer, but sometimes that is okay.

-Anthony

On Tue, Aug 23, 2011 at 01:17:12PM -0700, Jon Watte wrote:
> Protobuf is fully self-described, and fully forwards/backwards compatible,
> in that it can carry through "unknown" members of a structure, including the
> semantic of the fields. ASN1 is generally not fully self-described, because
> it only describes type, not semantic, for each field.
> 
> Sincerely,
> 
> jw
> 
> --
> Americans might object: there is no way we would sacrifice our living
> standards for the benefit of people in the rest of the world. Nevertheless,
> whether we get there willingly or not, we shall soon have lower consumption
> rates, because our present rates are unsustainable.
> 
> 
> 
> On Tue, Aug 23, 2011 at 3:03 AM, Vincent de Phily <
> vincent.dephily@REDACTED> wrote:
> 
> > On Monday 22 August 2011 11:31:44 Mike Oxford wrote:
> > > BERT is pretty much a "known good quantity" but protobuf is more
> > > efficient on the wire.
> > >
> > > Anyone have thoughts on which to go with?
> > >
> > > Also, along the BERT side, does anyone have recommendations on AS3 and
> > > Java implementations of the marshallers?
> > >
> > > The downside(s) to protobuf - hard IDLs and you have to build the
> > > project and generators.
> > > The downside(s) to BERT - less efficient on the wire (verbosity.)
> >
> > Also take a look at bjson, and one of my favourite : msgpack. BERT is
> > rather
> > unique in supporting atoms, which *nice* and can reduce encoded size a lot
> > (but that's less significant if you compress). For encoded size, I have yet
> > to
> > see anything beat UPER (asn1).
> >
> > A few things to consider :
> > * Bench size and speed with a mockup of your data : different formats are
> >  better for different types of data. Add compression to your tests.
> > * Think about format extensibility. It's easy enough with self-described
> >  formats like bert or msgpack, but gets more complicated with protobuf or
> >  asn1.
> > * The initial hurdle of dealing with a grammar and generators is well paid
> > off
> >  by having data validation and nicer api (mostly via #records{}) later on.
> > * If you need to speak with the outside world, self-describing formats will
> >  make your life easyer.
> > --
> > Vincent de Phily
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://erlang.org/mailman/listinfo/erlang-questions
> >

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions


-- 
------------------------------------------------------------------------
Anthony Molinaro                           <anthonym@REDACTED>