[erlang-questions] BERT vs protobuf in the erlang world

Mon Aug 29 14:08:04 CEST 2011

Not sure if this is repetitive or not but I posted a similar message earlier.

http://erlang.2086793.n4.nabble.com/Erlang-with-Thrift-td3544946.html

The UBF framework has a thrift format, erlang binary format, native ubf format, and other encoder/decoder implementations.

The UBF framework supports synchronous and asynchronous models.

If you need any help getting started, let me know.

thanks,

On 2011/08/29, at 20:40, Peter Neumark wrote:

> Unfortunately, I've just started writing the portion of the code which are would be interested in.
> I will push it to github as soon as it works. In the meantime you can check out what I have here:
> https://github.com/postlock/thrift-over-socketio
> 
> Peter
> 
> On Sun, Aug 28, 2011 at 1:18 AM, Wenqiang Song <wsong.cn@REDACTED> wrote:
> You are right, the socket server is in lib/erl. The whole Erlang
> library is tightly bound to synchronous model and hard to modify. Do
> you mind sharing your code?
> 
> Thanks
> Andy
> 
> On 8/28/11, Peter Neumark <neumark.peter@REDACTED> wrote:
> > It is true that the erlang socket server which ships with thrift is
> > synchronous (it's a gen_server).
> > This code is not generated, however. It is possible to create your own async
> > server without touching the code generator.
> > I know because I am writing a server for use with socket.io-erlang (which is
> > async).
> >
> > Peter
> >
> > On Sat, Aug 27, 2011 at 3:57 PM, Wenqiang Song <wsong.cn@REDACTED> wrote:
> >
> >> I second this. But the code generator directly generates synchronous
> >> socket server which is hard to embed into an existing asynchronous
> >> server.
> >>
> >> Andy
> >>
> >> On 8/27/11, Peter Neumark <neumark.peter@REDACTED> wrote:
> >> > I'm surprised Apache Thrift wasn't mentioned yet in this thread.
> >> > Thrift is inspired by protobuf, so it sports s similar (although more
> >> > modular) design.
> >> > It supports a wide range a languages, several protocols and transports
> >> (with
> >> > the ability to add your own).
> >> > The erlang lib + code generator code is actively developed and already
> >> works
> >> > quite well.
> >> >
> >> > Peter
> >> >
> >> > On Sat, Aug 27, 2011 at 5:43 AM, Jon Watte <jwatte@REDACTED> wrote:
> >> >
> >> >> 1) works as long as you control the sender or you have a well-defined
> >> >> policy when multiple fields are present, but I'd otherwise consider it
> >> >> a
> >> >> footgun.
> >> >>
> >> >>
> >> >>
> >> >> Your policy could easily be "there is only one field present at the top
> >> >> level" -- the next field encountered would simply be considered its own
> >> >> PDU.
> >> >>
> >> >>
> >> >> I really find it odd that protobuf doesn't provide a CHOICE construct.
> >> >>
> >> >>
> >> >> The argument is that making everything optional gives you the most of
> >> >> that,
> >> >> with no additional overhead. What you want is the additional rule that
> >> >> there
> >> >> is only ONE of those optionals, something which can be added at the
> >> >> application layer if it's needed.
> >> >> I'm not saying it wouldn't be nice if "switch" and PDU
> >> >> naming/segmenting
> >> >> were supported in protobuf -- that would be sweet! But we deal pretty
> >> well
> >> >> without it, and still find it better than the alternatives, especially
> >> for
> >> >> cross-language projects.
> >> >>
> >> >> Sincerely,
> >> >>
> >> >> jw
> >> >>
> >> >> --
> >> >> Americans might object: there is no way we would sacrifice our living
> >> >> standards for the benefit of people in the rest of the world.
> >> >> Nevertheless,
> >> >> whether we get there willingly or not, we shall soon have lower
> >> >> consumption
> >> >> rates, because our present rates are unsustainable.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On Thu, Aug 25, 2011 at 3:04 AM, Vincent de Phily <
> >> >> vincent.dephily@REDACTED> wrote:
> >> >>
> >> >>
> >> >>> On Wednesday 24 August 2011 15:15:57 Jon Watte wrote:
> >> >>> > You solve that in one of two ways:
> >> >>> > 1) Create a message that is the "PDU" that contains the union of
> >> >>> everything
> >> >>> > that could be sent,
> >> >>> > or
> >> >>> > 2) Create a wrapper that is just "type" followed by "binary data
> >> >>> > that
> >> >>> > is
> >> >>> a
> >> >>> > marshaled message."
> >> >>> >
> >> >>> > If you believe that the entirety of a protocol should be described
> >> >>> > by
> >> >>> that
> >> >>> > protocol, then 1) is actually a fine solution. The
> >> preserve-and-forward
> >> >>> > semantic of unknown field ids means that forward compatibility is
> >> still
> >> >>> > possible.
> >> >>> >
> >> >>> > message MyProtocolPDU {
> >> >>> >   optional SomeMessage some_message = 1;
> >> >>> >   optional AnotherMessage other_message = 2;
> >> >>> >   ...
> >> >>> > }
> >> >>> >
> >> >>> > So, encoding a MyProtocolPDU that contains a SomeMessage will end up
> >> >>> being
> >> >>> > different than encoding a MyProtocolPDU that contains an
> >> >>> > AnotherMessage.
> >> >>> > At the high level, you have to know that there is only one member in
> >> >>> each
> >> >>> > MyProtocolPDU; I find that to be quite acceptable.
> >> >>>
> >> >>>
> >> >>> Yup, they're reasonable solutions (as long as you planed ahead). 2)
> >> >>> may
> >> >>> be
> >> >>> ugly and innefficient, but it has been used as a pragmatic solution in
> >> >>> countless formats. 1) works as long as you control the sender or you
> >> have
> >> >>> a
> >> >>> well-defined policy when multiple fields are present, but I'd
> >> >>> otherwise
> >> >>> consider it a footgun. I really find it odd that protobuf doesn't
> >> provide
> >> >>> a
> >> >>> CHOICE construct.
> >> >>>
> >> >>> --
> >> >>> Vincent de Phily
> >> >>>
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> erlang-questions mailing list
> >> >> erlang-questions@REDACTED
> >> >> http://erlang.org/mailman/listinfo/erlang-questions
> >> >>
> >> >>
> >> >
> >>
> >> --
> >> Sent from my mobile device
> >>
> >> ---------------------------------------------------------------
> >> 有志者，事竟成，破釜沉舟，百二秦关终属楚
> >> 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> >>
> >
> 
> --
> Sent from my mobile device
> 
> ---------------------------------------------------------------
> 有志者，事竟成，破釜沉舟，百二秦关终属楚
> 苦心人，天不负，卧薪尝胆，三千越甲可吞吴
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

Joseph Wayne Norton
norton@REDACTED

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110829/90c39a81/attachment.htm>