Binary comprehensions

[Thomas Lindgren]

--- Jay Nelson <jay@REDACTED> wrote:

> Thomas wrote:
> 
> > For some special cases, a list comprehension
> wrapped
> > in binary_to_list/list_to_binary would suffice.
> E.g.,
> > 
> > list_to_binary([ foo(X) || X <-
> binary_to_list(Bin)])
> 
> Code-wise it reads ok, but if I am a middle-man
> process
> translating from one socket to another it introduces
> a lot
> of overhead.  I would like to receive one binary,
> transform
> and send out the resulting binary in a single step.

Yes. The trade-off is between speed and specificity.
For example, one could restrict such hypothetical
notation so that it is compilable into a C loop or
less. The restrictions might then be severe, and
perhaps not very intuitive. [Should we permit GC in
the middle? If so, what about having uninitialized
preallocated memory in the heap?] Or one could go in
the other direction and be very permissive, though the
gains vs using lists might then be small. 

(I read your requirements and suggestions, but haven't
had time to digest them, so I can't be more specific.)

One option could be to revisit the "research binary
syntax", which included a lot more features than the
binary syntax we have now. 

As I recall, the "research syntax" couldn't be
compiled very effectively, because one could not
always determine bit offsets at compile time. But it
still did have a number of interesting ideas in it.
(In particular, I think one could read and/or write a
whole binary in a single expression, which is a
property shared with the comprehension approach.)

One might want other operations too. For example, a
checksum is basically a foldl over the binary. And it
might be useful to permit binaries that are not byte
aligned (for compression algorithms, say).

Finally, let me say I consider the concept of binary
comprehensions as such to be worthwhile. One just has
to overcome the devilish details.

Best,
Thomas



		
__________________________________
Do you Yahoo!?
Yahoo! Mail - 50x more storage than other providers!
http://promotions.yahoo.com/new_mail