[erlang-questions 34] Re: [erlang-questions] Binary no-copy
Bob Cowdery
bob@REDACTED
Sat Mar 26 20:57:41 CET 2011
Paul
Thank you for the explanations. I have data running right through the
app now using a simulator in Erlang pretending to be the hardware. Next
up is to plug in the DSP. I've done a linked-in driver before so hoping
that won't be too difficult.
Bob
On 23/03/2011 01:50, Per Gustafsson wrote:
> So essentially everytime this function is called one of the four
> clauses will match
>
> 1. Will match if the state is exactly a full block right now. It will
> send that frame to the dsp module, and then it will continue to decode
> the current frame using a new state.
>
> 2. If the current state is larger than a full block we will split out
> the full block and send that to the dsp module, we will then continue
> to decode the current frame using whatever was left in the old state
> as the new state.
>
> 3. If the frame still has samples in it and neither of the first two
> cases were true we will take out one set of samples, convert to them
> to floats and add them to the end of the state and continue to decode
> the rest of the frame.
>
> 4 If the frame is empty we will return the current state.
>
> The difference between this version and the version that uses a
> comprehension is that we are checking for every sample whether the
> state is now a full frame or not.
>
> This allows us to write the new samples directly into the state
> binary, rather than creating a temporary binary. That we then copy
> into the state binary.
>
>
> 2011/3/18 Bob Cowdery <bob@REDACTED
> <mailto:bob@REDACTED>>
>
> Humour me. I'm still learning erlang and it still takes me ages to
> write a few lines at times. Comments in-line.
>
> Bob
>
>
> On 17/03/2011 20:54, Per Gustafsson wrote:
>> You're right it should be as below to get the same behaviour, but
>> I think it will work the same for input that makes sense.
>>
> Straight forward
>
>> do_decode_frame(Frame, State) ->
>> %% Separate the command bytes from the frame data
>> <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame,
>> decode_samples(Rest, State).
>>
> First clause will match a full block where frames and blocks are
> aligned. Never happen in practice as 63 will never be a multiple
> of 64 but it's right that the code should not know this.
>
>> decode_samples(Frame, FullBlock) when bytesize(FullBlock) ==
>> ?BinSize ->
>> dsp:process_block(FullBlock),
>> decode_samples(Frame, <<>>);
> Second clause will match when a FullBlock has > ?BinSize and will
> split into ?BinSize and Rest
>> decode_samples(Frame, <<FullBlock:?BinSize/binary, Rest/binary>>) ->
>> dsp:process_block(FullBlock),
>> decode_samples(Frame, Rest);
> Third clause. Now I get lost. This will match pretty much any
> binary > 8 bytes and any State. So this will get called from first
> and second clause. The next two clauses seem to run into each
> other. Don't understand that. Is the binary comprehension hidden
> somewhere in there?
>
>
> Sorry the third clause is totally wrong. It should have been:
> decode_samples(<<I:24/big-signed, Q:24/big-signed, _M:16/big-signed,
> Rest/binary>>, State) ->
> decode_samples(Rest, <<State/binary, (I/?S):32/float,
> (Q/?S):32/float>>);
>
>> decode_samples(<<I:24/big-signed, Q:24/big-signed, _M:16/big-signed,
>> Rest/binary>>, State) ->
>> decode_samples(<<State/binary, (I/?S):32/float, (Q/?S):32/float>>;
>> decode_samples(<<>>, State) ->
>> {ok, State}.
>>
> Would this work as the last two clauses:
> decode_samples(Frame, State) ->
> decode_samples(<<>>, << State/binary, <<(I/?S):32/float,
> (Q/?S):32/float>> || <<I:24/big-signed, Q:24/big-signed,
> _M:16/big-signed>> <= Frame >>)
>
> decode_samples(<<>>, State) ->
> {ok, State}.
>>
>> 2011/3/17 Bob Cowdery <bob@REDACTED
>> <mailto:bob@REDACTED>>:
>> > Thanks. More or less what I thought that the only copy is in
>> the append
>> > operation. The one liner is the big one in terms of time but I will
>> > profile the whole thing eventually. I don't think your solution
>> does
>> > quite the same thing as the sent block must be exactly the
>> correct size
>> > not >= but I appreciate the kind of structure you are using
>> with small
>> > clauses is more elegant.
>> >
>> > Bob
>> >
>> > On 17/03/2011 17:11, Per Gustafsson wrote:
>> >> It will copy binaries, but essentially the state binary will
>> act like
>> >> a growable array, pre-allocating space to grow in, so there
>> will be
>> >> some copying , but it will be linear in the size of the resulting
>> >> array.
>> >> It might be a little bit faster to rewrite this as shown
>> below. The
>> >> reason I think this might be faster is that you do not create a
>> >> temporary samples array that just gets copied. Whether this is
>> faster
>> >> or not in reality is of course a matter of benchmarking.
>> >>
>> >> %% Constants
>> >> %% Input scale factor
>> >> -define (S, (1 bsl 23) - 1).
>> >> %% Block size for DSP (interleaved I and Q)
>> >> -define (BlockSz, 1024).
>> >> %% Samples in a Metis frame (interleaved I and Q)
>> >> -define (MetisSmpls, 63).
>> >> %% Bytes in a float
>> >> -define (FloatSz, 4).
>> >> %% Size of binary for given block size
>> >> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls).
>> >>
>> >> do_decode_frame(Frame, State) ->
>> >> %% Separate the command bytes from the frame data
>> >> <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame,
>> >> decode_samples(Rest, State).
>> >>
>> >> decode_samples(Frame, State) when size(State) >= ?BinSz ->
>> >> dsp:process_block(State),
>> >> decode_samples(Frame, <<>>);
>> >> decode_samples(<<I:24/big-signed, Q:24/big-signed,
>> _M:16/big-signed,
>> >> Rest/binary>>, State) ->
>> >> decode_samples(<<State/binary, (I/?S):32/float,
>> (Q/?S):32/float>>;
>> >> decode_samples(<<>>, State) ->
>> >> {ok, State}.
>> >>
>> >> 2011/3/17 Bob Cowdery <bob@REDACTED
>> <mailto:bob@REDACTED>>
>> >>> Hi,
>> >>>
>> >>> Am I correct in thinking that the code fragments below will
>> not copy any
>> >>> binary data. I've not tested the function yet so it may have
>> bugs but
>> >>> I'm interested in whether it will be optimal in efficiency.
>> State starts
>> >>> off as an empty binary. Frames are decoded and placed in a
>> block. There
>> >>> are not an integral number of frames to a block. The function
>> >>> dsp:process_block() is in another gen-server.
>> >>>
>> >>> Thanks
>> >>> Bob
>> >>>
>> >>> %% Constants
>> >>> %% Input scale factor
>> >>> -define (S, (1 bsl 23) - 1).
>> >>> %% Block size for DSP (interleaved I and Q)
>> >>> -define (BlockSz, 1024).
>> >>> %% Samples in a Metis frame (interleaved I and Q)
>> >>> -define (MetisSmpls, 63).
>> >>> %% Bytes in a float
>> >>> -define (FloatSz, 4).
>> >>> %% Size of binary for given block size
>> >>> -define (BinSz, ?BlockSz*2*?FloatSz*?MetisSmpls).
>> >>>
>> >>> do_decode_frame(Frame, State) ->
>> >>> %% Separate the command bytes from the frame data
>> >>> <<C1:8, C2:8, C3:8, C4:8, C5:8, Rest/binary>> = Frame,
>> >>> %% What's left now is <<I:24, Q:24, M:16 ..repeated 63
>> times.. >>
>> >>> %% This one liner converts the I and Q to scaled floats
>> >>> Samples = << <<(I/?S):32/float, (Q/?S):32/float>> ||
>> >>> <<I:24/big-signed, Q:24/big-signed, _M:16/big-signed>> <=
>> Rest >>,
>> >>> CurrentSz = size(State),
>> >>> FrameSz = size(Samples),
>> >>> if
>> >>> (CurrentSz + FrameSz) < ?BinSz ->
>> >>> NewState = <<State/binary, Samples/binary>>;
>> >>> true ->
>> >>> %% Binary part of this transfer
>> >>> CutAt = ?BinSz - CurrentSz - 1,
>> >>> Bp = binary_part(Samples, {0, CutAt}),
>> >>> %% Binary part of remainder
>> >>> Br = binary_part(Samples, {CutAt, FrameSz - CutAt}),
>> >>> %% Process this block
>> >>> dsp:process_block(<<State/binary, Bp/binary>>),
>> >>> NewState = Br
>> >>> end,
>> >>> {ok, NewState}.
>> >>>
>> >>> ________________________________________________________________
>> >>> erlang-questions (at) erlang.org <http://erlang.org> mailing
>> list.
>> >>> See http://www.erlang.org/faq.html
>> >>> To unsubscribe;
>> mailto:erlang-questions-unsubscribe@REDACTED
>> <mailto:erlang-questions-unsubscribe@REDACTED>
>> >>>
>> >
>> >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110326/f4a38a98/attachment.htm>
More information about the erlang-questions
mailing list