[erlang-questions] map over bitstring

[Morten Krogh]

Hi,

I would like to understand what happens under the hood with this zip 
function, especially

<<R/binary,X,Y>>

Does this function work by copying R all the time, and then having the 
garbage collector
collect the old R immediately, or can BEAM share R in some way.

When I test it in the shell with 100MB:
C = zip(A, B).
A, B size 100 million.

I get a quite fast run time indicating that copying R all the time does 
not take place.
I get a peak memory consumption quite beyond the minimal 400 MB, however.

So what is BEAM doing here?

For the fun it, I wrote the map function in the spirit of Raimo's program,

map(F, A) ->
     map(F, A, <<>>).

map(F, <<X, A/binary>>, R) ->
     Mapped = F(X),
     map(F, A, <<R/binary, Mapped>>);
map(_F, <<>>, R) ->
     R.

To my enormous surprise, this program is about 7 times faster than the 
binary comprehension.


18>  A = binary:copy(<<"a">>, 10000000).

10 MB, that is.


19> T1 = now(), B = << <<"b">> || <<Y>> <= A>>, T2 = now(), 
timer:now_diff(T2,T1).
68860012

22> f(T1), f(T2), f(B).
ok
23> T1 = now(), B = zip2:map(fun(X) -> 98 end, A), T2 = now(), 
timer:now_diff(T2,T1).
9284114

(zip2 is my module name for this test)


What is going on?


cheers,

Morten.


On 10/22/10 9:49 AM, Raimo Niskanen wrote:
> On Fri, Oct 22, 2010 at 12:55:06AM -0400, rgowka1 wrote:
>> Thanks, that was really helpful.
>>
>> One more question, how do I zip on two bitstrings.
> For that you need to write a function of your own, matching
> out the heads of both bitstrings and building a new by appending.
>
> Uncompiled:
> zip(A, B) ->  zip(A, B,<<>>).
> zip(<<X,A/binary>>,<<Y,B/binary>>, R) ->
>      zip(A, B,<<R/binary,X,Y>>);
> zip(<<>>,<<>>, R) ->  R.
>
> The comprehensions can not do it since they do all combinations
> of the generators and do not do them in parallel.
>
>> On 10/21/10, Jesper Louis Andersen<jesper.louis.andersen@REDACTED>  wrote:
>>> On Thu, Oct 21, 2010 at 9:56 PM, rgowka1<rgowka1@REDACTED>  wrote:
>>>> Hi -
>>>>
>>>> How do I map a function over a bitstring without converting into string or
>>>> list.
>>>>
>>>> For example - how can I replace<<  "a">>  with<<  "X">>  and any other
>>>> character to<<  "Y">>??
>>>>
>>>> InBin =<<  "abcaa">>
>>>>
>>>> OutBin =<<  "XYYXX">>
>>> The trick is Binary Comprehensions or by using the (R14B) binary
>>> module. Here is a solution using the Binary Comprehensions:
>>>
>>> 3>  <<  <<case X of $a ->  $X; _ ->  $Y end>>  ||<<X>>  <=<<"abcaa">>  >>.
>>> <<"XYYXX">>
>>>
>>> --
>>> J.
>>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED