[erlang-questions] High Sierra and binary_to_term (compressed)

[Jan Chochol]

Hi Martin,

This is bug with combination of new zlib and old Erlang -
https://github.com/erlang/otp/commit/e27119948fc6ab28bea81019720bddaac5b655a7
.
It is fixed in OTP-18.3.4.5 and newer.
"--enable-builtin-zlib" works, because it uses older bundled zlib.

Jan

On Mon, Nov 13, 2017 at 10:01 PM, Martin Sumner <martin.sumner@REDACTED>
wrote:

> Rather foolishly I accepted the upgrade to High Sierra on my Macbook last
> night.  This has led to an interesting problem with a binary_to_term line
> on my project.
>
> Following the upgrade, tests have started failing on this line:
>
> https://github.com/martinsumner/leveled/blob/master/src/leveled_sst.erl#
> L875
>
> with error:badarg being thrown.
>
> However if I copy and paste the binary on which the badarg has been thrown
> out of the crash report, and into a erlang console - binary_to_term works
> just fine.
>
> See this issue for an example - https://github.com/
> martinsumner/leveled/issues/106
>
> This failure is intermittent, and indeed wrapping the binary_to_term like
> this is enough to make the problem seemingly disappear:
>
> try binary_to_term(Bin) of
>     T ->
>         T
> catch
>     error:badarg ->
>         binary_to_term(Bin)
> end.
>
> Re-running the tests on another OSX device on a release previous to High
> Sierra shows no issues with/without the wrapper.
>
> The problem appears to consistently exist in OTP 17.5 and 18.3.  However,
> running the tests in 19.3 and R16B02-basho10 doesn't show the problem.
>
> The terms in question have all been compressed using term_to_binary's zlib
> compression.
>
> If I build 18.3 on High Sierra with the  "--enable-builtin-zlib" option,
> the problem does *not* re-occur.  This switch appears to be a reliable
> delta between having and not having the problem, and I believe may have
> become the default in OTP 17.
>
> It appears to be specific to v17/v18 Erlang with compressed terms, on High
> Sierra where the OS zlib implementation is used, not the inbuilt OTP zlib.
>
> There appears to be internet chatter of other issues with Mac OSX High
> Sierra and changes to the zlib implementation - https://github.com/madler/
> zlib/issues/305.
>
> So I think this doesn't look like it is an Erlang issue, but I suspect it
> may not be resolved in High Sierra any time soon, and may impact other
> Erlang users using zlib functionality.  Does this it seem reasonable to
> pass this off as a High Sierra zlib bug, and use the --enable-builtin-zlib
> workaround?  If this is the case I'm not sure why I don't get the issue on
> 19.3, any ideas as to why?
>
> Martin
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20171114/29773ccb/attachment.htm>