[erlang-questions] High Sierra and binary_to_term (compressed)

[Sergey Yelin]

The `--enable-builtin-zlib` flag also fixes strange xref/dialyzer bugs in one of our projects built on top of Erlang 17.5 on High Sierra.

Thanks for sharing! :)
---
Best regards,
Sergey Yelin.



> On 14 Nov 2017, at 10:30, Jan Chochol <jan.chochol@REDACTED> wrote:
> 
> Hi Martin,
> 
> This is bug with combination of new zlib and old Erlang - https://github.com/erlang/otp/commit/e27119948fc6ab28bea81019720bddaac5b655a7 <https://github.com/erlang/otp/commit/e27119948fc6ab28bea81019720bddaac5b655a7> .
> It is fixed in OTP-18.3.4.5 and newer.
> "--enable-builtin-zlib" works, because it uses older bundled zlib.
> 
> Jan
> 
> On Mon, Nov 13, 2017 at 10:01 PM, Martin Sumner <martin.sumner@REDACTED <mailto:martin.sumner@REDACTED>> wrote:
> Rather foolishly I accepted the upgrade to High Sierra on my Macbook last night.  This has led to an interesting problem with a binary_to_term line on my project.
> 
> Following the upgrade, tests have started failing on this line:
> 
> https://github.com/martinsumner/leveled/blob/master/src/leveled_sst.erl#L875 <https://github.com/martinsumner/leveled/blob/master/src/leveled_sst.erl#L875>
> 
> with error:badarg being thrown.
> 
> However if I copy and paste the binary on which the badarg has been thrown out of the crash report, and into a erlang console - binary_to_term works just fine.
> 
> See this issue for an example - https://github.com/martinsumner/leveled/issues/106 <https://github.com/martinsumner/leveled/issues/106>
> 
> This failure is intermittent, and indeed wrapping the binary_to_term like this is enough to make the problem seemingly disappear:
> 
> try binary_to_term(Bin) of
>     T -> 
>         T
> catch
>     error:badarg ->
>         binary_to_term(Bin)
> end.
> 
> Re-running the tests on another OSX device on a release previous to High Sierra shows no issues with/without the wrapper.
> 
> The problem appears to consistently exist in OTP 17.5 and 18.3.  However, running the tests in 19.3 and R16B02-basho10 doesn't show the problem.
> 
> The terms in question have all been compressed using term_to_binary's zlib compression.
> 
> If I build 18.3 on High Sierra with the  "--enable-builtin-zlib" option, the problem does *not* re-occur.  This switch appears to be a reliable delta between having and not having the problem, and I believe may have become the default in OTP 17. 
> 
> It appears to be specific to v17/v18 Erlang with compressed terms, on High Sierra where the OS zlib implementation is used, not the inbuilt OTP zlib.
> 
> There appears to be internet chatter of other issues with Mac OSX High Sierra and changes to the zlib implementation - https://github.com/madler/zlib/issues/305 <https://github.com/madler/zlib/issues/305>.
> 
> So I think this doesn't look like it is an Erlang issue, but I suspect it may not be resolved in High Sierra any time soon, and may impact other Erlang users using zlib functionality.  Does this it seem reasonable to pass this off as a High Sierra zlib bug, and use the --enable-builtin-zlib workaround?  If this is the case I'm not sure why I don't get the issue on 19.3, any ideas as to why?
> 
> Martin 
> 
> 
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
> http://erlang.org/mailman/listinfo/erlang-questions <http://erlang.org/mailman/listinfo/erlang-questions>
> 
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20171114/fc524860/attachment.htm>