[erlang-questions] map over bitstring
Morten Krogh
mk@REDACTED
Fri Oct 22 11:25:13 CEST 2010
Hi,
I would like to understand what happens under the hood with this zip
function, especially
<<R/binary,X,Y>>
Does this function work by copying R all the time, and then having the
garbage collector
collect the old R immediately, or can BEAM share R in some way.
When I test it in the shell with 100MB:
C = zip(A, B).
A, B size 100 million.
I get a quite fast run time indicating that copying R all the time does
not take place.
I get a peak memory consumption quite beyond the minimal 400 MB, however.
So what is BEAM doing here?
For the fun it, I wrote the map function in the spirit of Raimo's program,
map(F, A) ->
map(F, A, <<>>).
map(F, <<X, A/binary>>, R) ->
Mapped = F(X),
map(F, A, <<R/binary, Mapped>>);
map(_F, <<>>, R) ->
R.
To my enormous surprise, this program is about 7 times faster than the
binary comprehension.
18> A = binary:copy(<<"a">>, 10000000).
10 MB, that is.
19> T1 = now(), B = << <<"b">> || <<Y>> <= A>>, T2 = now(),
timer:now_diff(T2,T1).
68860012
22> f(T1), f(T2), f(B).
ok
23> T1 = now(), B = zip2:map(fun(X) -> 98 end, A), T2 = now(),
timer:now_diff(T2,T1).
9284114
(zip2 is my module name for this test)
What is going on?
cheers,
Morten.
On 10/22/10 9:49 AM, Raimo Niskanen wrote:
> On Fri, Oct 22, 2010 at 12:55:06AM -0400, rgowka1 wrote:
>> Thanks, that was really helpful.
>>
>> One more question, how do I zip on two bitstrings.
> For that you need to write a function of your own, matching
> out the heads of both bitstrings and building a new by appending.
>
> Uncompiled:
> zip(A, B) -> zip(A, B,<<>>).
> zip(<<X,A/binary>>,<<Y,B/binary>>, R) ->
> zip(A, B,<<R/binary,X,Y>>);
> zip(<<>>,<<>>, R) -> R.
>
> The comprehensions can not do it since they do all combinations
> of the generators and do not do them in parallel.
>
>> On 10/21/10, Jesper Louis Andersen<jesper.louis.andersen@REDACTED> wrote:
>>> On Thu, Oct 21, 2010 at 9:56 PM, rgowka1<rgowka1@REDACTED> wrote:
>>>> Hi -
>>>>
>>>> How do I map a function over a bitstring without converting into string or
>>>> list.
>>>>
>>>> For example - how can I replace<< "a">> with<< "X">> and any other
>>>> character to<< "Y">>??
>>>>
>>>> InBin =<< "abcaa">>
>>>>
>>>> OutBin =<< "XYYXX">>
>>> The trick is Binary Comprehensions or by using the (R14B) binary
>>> module. Here is a solution using the Binary Comprehensions:
>>>
>>> 3> << <<case X of $a -> $X; _ -> $Y end>> ||<<X>> <=<<"abcaa">> >>.
>>> <<"XYYXX">>
>>>
>>> --
>>> J.
>>>
>> ________________________________________________________________
>> erlang-questions (at) erlang.org mailing list.
>> See http://www.erlang.org/faq.html
>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
More information about the erlang-questions
mailing list