[erlang-questions] High Sierra and binary_to_term (compressed)

[Martin Sumner]

Rather foolishly I accepted the upgrade to High Sierra on my Macbook last
night.  This has led to an interesting problem with a binary_to_term line
on my project.

Following the upgrade, tests have started failing on this line:

https://github.com/martinsumner/leveled/blob/master/src/leveled_sst.erl#L875

with error:badarg being thrown.

However if I copy and paste the binary on which the badarg has been thrown
out of the crash report, and into a erlang console - binary_to_term works
just fine.

See this issue for an example -
https://github.com/martinsumner/leveled/issues/106

This failure is intermittent, and indeed wrapping the binary_to_term like
this is enough to make the problem seemingly disappear:

try binary_to_term(Bin) of
    T ->
        T
catch
    error:badarg ->
        binary_to_term(Bin)
end.

Re-running the tests on another OSX device on a release previous to High
Sierra shows no issues with/without the wrapper.

The problem appears to consistently exist in OTP 17.5 and 18.3.  However,
running the tests in 19.3 and R16B02-basho10 doesn't show the problem.

The terms in question have all been compressed using term_to_binary's zlib
compression.

If I build 18.3 on High Sierra with the  "--enable-builtin-zlib" option,
the problem does *not* re-occur.  This switch appears to be a reliable
delta between having and not having the problem, and I believe may have
become the default in OTP 17.

It appears to be specific to v17/v18 Erlang with compressed terms, on High
Sierra where the OS zlib implementation is used, not the inbuilt OTP zlib.

There appears to be internet chatter of other issues with Mac OSX High
Sierra and changes to the zlib implementation -
https://github.com/madler/zlib/issues/305.

So I think this doesn't look like it is an Erlang issue, but I suspect it
may not be resolved in High Sierra any time soon, and may impact other
Erlang users using zlib functionality.  Does this it seem reasonable to
pass this off as a High Sierra zlib bug, and use the --enable-builtin-zlib
workaround?  If this is the case I'm not sure why I don't get the issue on
19.3, any ideas as to why?

Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20171113/9b82705d/attachment.htm>