[erlang-questions] High Sierra and binary_to_term (compressed)

[Martin Sumner]

Jan,

Thank-you, that does explain things.

Martin

On 14 November 2017 at 07:30, Jan Chochol <jan.chochol@REDACTED> wrote:

> Hi Martin,
>
> This is bug with combination of new zlib and old Erlang -
> https://github.com/erlang/otp/commit/e27119948fc6ab28bea81019720bdd
> aac5b655a7 .
> It is fixed in OTP-18.3.4.5 and newer.
> "--enable-builtin-zlib" works, because it uses older bundled zlib.
>
> Jan
>
> On Mon, Nov 13, 2017 at 10:01 PM, Martin Sumner <martin.sumner@REDACTED>
> wrote:
>
>> Rather foolishly I accepted the upgrade to High Sierra on my Macbook last
>> night.  This has led to an interesting problem with a binary_to_term line
>> on my project.
>>
>> Following the upgrade, tests have started failing on this line:
>>
>> https://github.com/martinsumner/leveled/blob/master/src/
>> leveled_sst.erl#L875
>>
>> with error:badarg being thrown.
>>
>> However if I copy and paste the binary on which the badarg has been
>> thrown out of the crash report, and into a erlang console - binary_to_term
>> works just fine.
>>
>> See this issue for an example - https://github.com/martinsum
>> ner/leveled/issues/106
>>
>> This failure is intermittent, and indeed wrapping the binary_to_term like
>> this is enough to make the problem seemingly disappear:
>>
>> try binary_to_term(Bin) of
>>     T ->
>>         T
>> catch
>>     error:badarg ->
>>         binary_to_term(Bin)
>> end.
>>
>> Re-running the tests on another OSX device on a release previous to High
>> Sierra shows no issues with/without the wrapper.
>>
>> The problem appears to consistently exist in OTP 17.5 and 18.3.  However,
>> running the tests in 19.3 and R16B02-basho10 doesn't show the problem.
>>
>> The terms in question have all been compressed using term_to_binary's
>> zlib compression.
>>
>> If I build 18.3 on High Sierra with the  "--enable-builtin-zlib" option,
>> the problem does *not* re-occur.  This switch appears to be a reliable
>> delta between having and not having the problem, and I believe may have
>> become the default in OTP 17.
>>
>> It appears to be specific to v17/v18 Erlang with compressed terms, on
>> High Sierra where the OS zlib implementation is used, not the inbuilt OTP
>> zlib.
>>
>> There appears to be internet chatter of other issues with Mac OSX High
>> Sierra and changes to the zlib implementation -
>> https://github.com/madler/zlib/issues/305.
>>
>> So I think this doesn't look like it is an Erlang issue, but I suspect it
>> may not be resolved in High Sierra any time soon, and may impact other
>> Erlang users using zlib functionality.  Does this it seem reasonable to
>> pass this off as a High Sierra zlib bug, and use the --enable-builtin-zlib
>> workaround?  If this is the case I'm not sure why I don't get the issue on
>> 19.3, any ideas as to why?
>>
>> Martin
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20171114/c4d0c6e6/attachment.htm>