[erlang-questions] BERT vs protobuf in the erlang world

[Vincent de Phily]

On Tuesday 23 August 2011 16:22:46 Anthony Molinaro wrote:
> To me a self-describing protocol is something which also encoded the names
> of the messages and fields, so you could send it through to the other side
> and it would be unambiguous.  An example of this is a system I wrote many
> years ago called lwes (www.lwes.org), which does not require code generation
> because messages are self describing.

Yes, that's what I meant by self-describing. You can dump an unknown message 
and know not only that it's {42, "car"} but also that it's {id:42, 
class:"car"}.

Protobuf takes carefull steps to enable grammar extensibility, but it is still 
not as flexible as a self-described format. You cannot switch a field to be 
optional, add a required field, reorder fields, or start sending a list of 
objects when you used to send a single one for example.

One thing that irks me about protobuf is that it doesn't have proper 
"optional" fields: you'll always end up with a default value instead of "not 
present" and decoders won't tell you if they inferred the default value 
themselves. You can kludge around by using a sentinel value or a field 
repeated zero times but this is not the same thing.

If you're going the grammar-described-format route, I'd go with asn1 instead 
of protobuf unless :
* support for $OTHER_LANGUAGE is needed (despite being nearly as old as me,
  asn1 support isn't ubiquitous yet, and sometimes costs a fortune).
* codec speed is paramount (I haven't benched one against the other for CPU
  usage, but I expect protobuf to have the edge)
With asn1, you get a better grammar that supports :
* DEFAULT vs OPTIONAL
* CHOICE (like a protobuf enum, except values can be anything, not just
  integer)
* checked ranges for everything, from integers to list size to character set
  (why waste a full byte when you're only using characters [a-z_] ?)
* Neat tricks like 'NULL' and single-value ranges
Extensibility of the BER variant is just as good/quirky as protobuf's, and if 
your bytes cost money you can spend less with the UPER variant.
-- 
Vincent de Phily