[erlang-questions] Binary no-copy

[Bob Cowdery]

Humour me. I'm still learning erlang and it still takes me ages to write
a few lines at times. Comments in-line.

Bob

On 17/03/2011 20:54, Per Gustafsson wrote:
> You're right it should be as below to get the same behaviour, but I
> think it will work the same for input that makes sense.
>
Straight forward
> do_decode_frame(Frame, State) ->
>    %% Separate the command bytes from the frame data
>    <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame,
>    decode_samples(Rest, State).
>
First clause will match a full block where frames and blocks are
aligned. Never happen in practice as 63 will never be a multiple of 64
but it's right that the code should not know this.
> decode_samples(Frame, FullBlock) when bytesize(FullBlock) == ?BinSize ->
>   dsp:process_block(FullBlock),
>   decode_samples(Frame, <<>>);
Second clause will match when a FullBlock has > ?BinSize and will split
into ?BinSize and Rest
> decode_samples(Frame, <<FullBlock:?BinSize/binary, Rest/binary>>) ->
>   dsp:process_block(FullBlock),
>   decode_samples(Frame, Rest);
Third clause. Now I get lost. This will match pretty much any binary > 8
bytes and any State. So this will get called from first and second
clause. The next two clauses seem to run into each other. Don't
understand that. Is the binary comprehension hidden somewhere in there?
> decode_samples(<<I:24/big-signed, Q:24/big-signed, _M:16/big-signed,
> Rest/binary>>, State) ->
>   decode_samples(<<State/binary, (I/?S):32/float, (Q/?S):32/float>>;
> decode_samples(<<>>, State) ->
>  {ok, State}.
>
Would this work as the last two clauses:
decode_samples(Frame, State) ->
    decode_samples(<<>>, << State/binary, <<(I/?S):32/float,
(Q/?S):32/float>> || <<I:24/big-signed, Q:24/big-signed,
_M:16/big-signed>> <= Frame >>)
decode_samples(<<>>, State) ->
     {ok, State}.
>
> 2011/3/17 Bob Cowdery <bob@REDACTED
> <mailto:bob@REDACTED>>:
> > Thanks. More or less what I thought that the only copy is in the append
> > operation. The one liner is the big one in terms of time but I will
> > profile the whole thing eventually. I don't think your solution does
> > quite the same thing as the sent block must be exactly the correct size
> > not >= but I appreciate the kind of structure you are using with small
> > clauses is more elegant.
> >
> > Bob
> >
> > On 17/03/2011 17:11, Per Gustafsson wrote:
> >> It will copy binaries, but essentially the state binary will act like
> >> a growable array, pre-allocating space to grow in, so there will be
> >> some copying , but it will be linear in the size of the resulting
> >> array.
> >> It might be a little bit faster to rewrite this as shown below. The
> >> reason I think this might be faster is that you do not create a
> >> temporary samples array that just gets copied. Whether this is faster
> >> or not in reality is of course a matter of benchmarking.
> >>
> >> %% Constants
> >> %% Input scale factor
> >> -define (S, (1 bsl 23) - 1).
> >> %% Block size for DSP (interleaved I and Q)
> >> -define (BlockSz, 1024).
> >> %% Samples in a Metis frame (interleaved I and Q)
> >> -define (MetisSmpls, 63).
> >> %% Bytes in a float
> >> -define (FloatSz, 4).
> >> %% Size of binary for given block size
> >> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls).
> >>
> >> do_decode_frame(Frame, State) ->
> >>    %% Separate the command bytes from the frame data
> >>    <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame,
> >>    decode_samples(Rest, State).
> >>
> >> decode_samples(Frame, State) when size(State) >= ?BinSz ->
> >>   dsp:process_block(State),
> >>   decode_samples(Frame, <<>>);
> >> decode_samples(<<I:24/big-signed, Q:24/big-signed, _M:16/big-signed,
> >> Rest/binary>>, State) ->
> >>   decode_samples(<<State/binary, (I/?S):32/float, (Q/?S):32/float>>;
> >> decode_samples(<<>>, State) ->
> >>   {ok, State}.
> >>
> >> 2011/3/17 Bob Cowdery <bob@REDACTED
> <mailto:bob@REDACTED>>
> >>> Hi,
> >>>
> >>> Am I correct in thinking that the code fragments below will not
> copy any
> >>> binary data. I've not tested the function yet so it may have bugs but
> >>> I'm interested in whether it will be optimal in efficiency. State
> starts
> >>> off as an empty binary. Frames are decoded and placed in a block.
> There
> >>> are not an integral number of frames to a block. The function
> >>> dsp:process_block() is in another gen-server.
> >>>
> >>> Thanks
> >>> Bob
> >>>
> >>> %% Constants
> >>> %% Input scale factor
> >>> -define (S, (1 bsl 23) - 1).
> >>> %% Block size for DSP (interleaved I and Q)
> >>> -define (BlockSz, 1024).
> >>> %% Samples in a Metis frame (interleaved I and Q)
> >>> -define (MetisSmpls, 63).
> >>> %% Bytes in a float
> >>> -define (FloatSz, 4).
> >>> %% Size of binary for given block size
> >>> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls).
> >>>
> >>> do_decode_frame(Frame, State) ->
> >>>    %% Separate the command bytes from the frame data
> >>>    <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame,
> >>>    %% What's left now is <<I:24, Q:24, M:16 ..repeated 63 times.. >>
> >>>    %% This one liner converts the I and Q to scaled floats
> >>>    Samples = << <<(I/?S):32/float, (Q/?S):32/float>> ||
> >>> <<I:24/big-signed, Q:24/big-signed, _M:16/big-signed>> <= Rest >>,
> >>>    CurrentSz = size(State),
> >>>    FrameSz = size(Samples),
> >>>    if
> >>>        (CurrentSz + FrameSz) < ?BinSz ->
> >>>            NewState = <<State/binary, Samples/binary>>;
> >>>        true ->
> >>>            %% Binary part of this transfer
> >>>            CutAt = ?BinSz - CurrentSz - 1,
> >>>            Bp = binary_part(Samples, {0, CutAt}),
> >>>            %% Binary part of remainder
> >>>            Br = binary_part(Samples, {CutAt, FrameSz - CutAt}),
> >>>            %% Process this block
> >>>            dsp:process_block(<<State/binary, Bp/binary>>),
> >>>            NewState = Br
> >>>    end,
> >>>    {ok, NewState}.
> >>>
> >>> ________________________________________________________________
> >>> erlang-questions (at) erlang.org <http://erlang.org> mailing list.
> >>> See http://www.erlang.org/faq.html
> >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
> <mailto:erlang-questions-unsubscribe@REDACTED>
> >>>
> >
> >
>