You solve that in one of two ways:<div>1) Create a message that is the "PDU" that contains the union of everything that could be sent,</div><div>or</div><div>2) Create a wrapper that is just "type" followed by "binary data that is a marshaled message."<br clear="all">
<br></div><div>If you believe that the entirety of a protocol should be described by that protocol, then 1) is actually a fine solution. The preserve-and-forward semantic of unknown field ids means that forward compatibility is still possible.</div>
<div><br></div><div>message MyProtocolPDU {</div><div> optional SomeMessage some_message = 1;</div><div> optional AnotherMessage other_message = 2;</div><div> ...<br>}</div><div><br></div><div>So, encoding a MyProtocolPDU that contains a SomeMessage will end up being different than encoding a MyProtocolPDU that contains an AnotherMessage.</div>
<div>At the high level, you have to know that there is only one member in each MyProtocolPDU; I find that to be quite acceptable.</div><div><br></div><div>Sincerely,<br></div><div><br></div><div>jw</div><div><br>--<br>Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable. <br>
<br>
<br><br><div class="gmail_quote">On Tue, Aug 23, 2011 at 4:22 PM, Anthony Molinaro <span dir="ltr"><<a href="mailto:anthonym@alumni.caltech.edu">anthonym@alumni.caltech.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Protobuf is not what I would call self describing, if you have something<br>
like<br>
<br>
message foo {<br>
required int32 bar = 1;<br>
}<br>
<br>
message foo2 {<br>
optional int32 bar2 = 1;<br>
}<br>
<br>
and create an instance of each and then send it remotely, you can not<br>
distinguish between the 2 messages, unless the remote side know when<br>
one you mean to send.<br>
<br>
Here's an example in erlang<br>
<br>
Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0]<br>
[hipe] [kernel-poll:false]<br>
<br>
Eshell V5.8.4 (abort with ^G)<br>
1> rr(foo_pb).<br>
[foo,foo2]<br>
2> foo_pb:encode(#foo{bar=42}).<br>
<<"\b*">><br>
3> foo_pb:encode(#foo2{bar2=42}).<br>
<<"\b*">><br>
<br>
To me a self-describing protocol is something which also encoded the names<br>
of the messages and fields, so you could send it through to the other side<br>
and it would be unambiguous. An example of this is a system I wrote many<br>
years ago called lwes (<a href="http://www.lwes.org" target="_blank">www.lwes.org</a>), which does not require code generation<br>
because messages are self describing.<br>
<br>
Now this does mean messages are longer, but sometimes that is okay.<br>
<br>
-Anthony<br>
<div><div></div><div class="h5"><br>
On Tue, Aug 23, 2011 at 01:17:12PM -0700, Jon Watte wrote:<br>
> Protobuf is fully self-described, and fully forwards/backwards compatible,<br>
> in that it can carry through "unknown" members of a structure, including the<br>
> semantic of the fields. ASN1 is generally not fully self-described, because<br>
> it only describes type, not semantic, for each field.<br>
><br>
> Sincerely,<br>
><br>
> jw<br>
><br>
> --<br>
> Americans might object: there is no way we would sacrifice our living<br>
> standards for the benefit of people in the rest of the world. Nevertheless,<br>
> whether we get there willingly or not, we shall soon have lower consumption<br>
> rates, because our present rates are unsustainable.<br>
><br>
><br>
><br>
> On Tue, Aug 23, 2011 at 3:03 AM, Vincent de Phily <<br>
> <a href="mailto:vincent.dephily@mobile-devices.fr">vincent.dephily@mobile-devices.fr</a>> wrote:<br>
><br>
> > On Monday 22 August 2011 11:31:44 Mike Oxford wrote:<br>
> > > BERT is pretty much a "known good quantity" but protobuf is more<br>
> > > efficient on the wire.<br>
> > ><br>
> > > Anyone have thoughts on which to go with?<br>
> > ><br>
> > > Also, along the BERT side, does anyone have recommendations on AS3 and<br>
> > > Java implementations of the marshallers?<br>
> > ><br>
> > > The downside(s) to protobuf - hard IDLs and you have to build the<br>
> > > project and generators.<br>
> > > The downside(s) to BERT - less efficient on the wire (verbosity.)<br>
> ><br>
> > Also take a look at bjson, and one of my favourite : msgpack. BERT is<br>
> > rather<br>
> > unique in supporting atoms, which *nice* and can reduce encoded size a lot<br>
> > (but that's less significant if you compress). For encoded size, I have yet<br>
> > to<br>
> > see anything beat UPER (asn1).<br>
> ><br>
> > A few things to consider :<br>
> > * Bench size and speed with a mockup of your data : different formats are<br>
> > better for different types of data. Add compression to your tests.<br>
> > * Think about format extensibility. It's easy enough with self-described<br>
> > formats like bert or msgpack, but gets more complicated with protobuf or<br>
> > asn1.<br>
> > * The initial hurdle of dealing with a grammar and generators is well paid<br>
> > off<br>
> > by having data validation and nicer api (mostly via #records{}) later on.<br>
> > * If you need to speak with the outside world, self-describing formats will<br>
> > make your life easyer.<br>
> > --<br>
> > Vincent de Phily<br>
> > _______________________________________________<br>
> > erlang-questions mailing list<br>
> > <a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
> > <a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
> ><br>
<br>
> _______________________________________________<br>
> erlang-questions mailing list<br>
> <a href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a><br>
> <a href="http://erlang.org/mailman/listinfo/erlang-questions" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
<br>
</div></div><font color="#888888">--<br>
------------------------------------------------------------------------<br>
Anthony Molinaro <<a href="mailto:anthonym@alumni.caltech.edu">anthonym@alumni.caltech.edu</a>><br>
</font></blockquote></div><br></div>