[erlang-questions] Maps & records: binding directly to bitstring comprehensions

[Felix Gallo]

Jesper, your instinct are correct as usual.  Unfortunately the protocol
designers live upstream in a different group over which I have no control,
and are incentivized to take the Facebook mantra of "move fast and break
things" to heart.

Maybe to rephrase the question: if I have a binary with, say, 850 fields
representing, in essence, a tightly packed data structure often containing
heirarchical elements, and I wish to both be able to decompose this, and
also to recompose it, and I'd like to use records or maps to enforce as
much cleanliness as possible, what's the best practice, and is there
something we can do to  while the EEP43 concrete is still wet?

F.


On Fri, Feb 7, 2014 at 4:47 AM, Jesper Louis Andersen <
jesper.louis.andersen@REDACTED> wrote:

>
> On Thu, Feb 6, 2014 at 11:35 PM, Felix Gallo <felixgallo@REDACTED> wrote:
>
>> I live in a world of pain wherein I receive 4K binary blob packets with a
>> tightly packed schema that changes every few weeks.
>
>
> For what it is worth, this is the root cause of the problem. If you have a
> packet format that changes this often, you are probably better off
> improving the packet encoding scheme than trying to parse it and adapt. You
> need a format which has more framing so it can be parsed by recursively
> stripping off more and more information. That is, you can decode the packet
> partially and skip sections your parser doesn't understand yet.
>
> Also beware coding complexity. If data is never placed at rest on disk,
> then struggling to compress data with an intricate scheme is not going to
> buy you anything since there tend to be enough bandwidth for these kind of
> things. Decoding is going to dwarf everything else[0].
>
> I would probably just wait a bit until the format settles and then go
> implement it there.
>
> [0] This describes almost everything that is wrong with JSON.
>
> --
> J.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140207/90f40a06/attachment.htm>