[erlang-questions] Passing data between Erlang and C port while preserving padding and alignment?

Frank Muller frank.muller.erl@REDACTED
Sat Dec 8 23:30:47 CET 2018


Hello Jesper,

My binaries are an infinite stream of at least ~7500B each (include lot of
floats).

Packing the associated C struct will negatively  affect the performances
I’m afraid.

This is why I would like to align every field to its natural alignement if
possible.

/Frank

There is also an attribute, packed, which can be useful here:
>
> https://gcc.gnu.org/onlinedocs/gcc-4.0.4/gcc/Type-Attributes.html
> http://www.catb.org/esr/structure-packing/
>
> If you are not too concerned about the efficiency, I would just pack the
> struct and send it over the wire. The problem with this on a modern CPU:
>
> * While it packs tighter in your L1D cache, and this gives a speedup.
> * It requires more instructions on-CPU to get alignment back in order.
> * If multiple CPU threads access the data, you will end up bouncing
> between the L1D caches of the cores, and this is usually bad. With
> alignment, this can be avoided.
>
> My attack on the problem would be to pack first, then start measuring and
> see how much more I can get out of a solution as suggested by Mikael in
> this thread.
>
> On Sat, Dec 8, 2018 at 12:11 AM Sölvi Páll Ásgeirsson <solvip@REDACTED>
> wrote:
>
>> Can you use #pragma pack on the C structs?
>> You can then easily deconstruct them in a binary match
>>
>> Regards
>> Sölvi
>>
>> Sent from my iPhone
>>
>> On 7 Dec 2018, at 21:02, Roger Lipscombe <roger@REDACTED> wrote:
>>
>> On Fri, 7 Dec 2018 at 18:02, Frank Muller <frank.muller.erl@REDACTED>
>> wrote:
>>
>>> Thanks Mikael.
>>> Any others thoughts ?
>>>
>>
>> I'm assuming that you've considered serializing and deserializing the
>> data to an unpadded (or defined padding) format, but that you've presumably
>> discounted it for performance reasons?
>>
>> Just putting it out there in case someone's facing a similar issue, but
>> has different constraints.
>>
>> We once used XDR for this (with some kind of IDL-to-C converter, if I
>> recall correctly). Some brief googling suggests that XDR is not dead yet.
>>
>> Cheers,
>> Roger.
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
> --
> J.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20181208/494d721a/attachment.htm>


More information about the erlang-questions mailing list