[erlang-questions] zlib memory leak
Tue Mar 15 13:07:19 CET 2016
To check your situation I created simple demo for your gz module (
As I can see file size about 1Gb unzipped properly. No errors or VM
crashes. Maybe I use your module incorrectly ...
Unfortunately I can't check as huge file as you have but I'll try file 10Gb
P.S. I use Debian Jessie, 64 bit, 1 Virtual CPU and Erlang installed from
Erlang Solution repo.
2016-03-14 23:25 GMT+03:00 Dmitry Kolesnikov <>:
> I’ve got an interesting issue with zlib at otp-18.2.1 I’ve not checked
> other releases yet.
> I do have a file about 30GB of compressed data, it is expanded to 300GB of
> textual UTF8 data.
> The producer of the file claims that standard gzip is used. The file
> header  is:
> 1F 8B 08 04 00 00 00 00 00 00 24 03
> My decompression program is very simple , it reads 64K binary chunks
> from file and inflates them using zlib:inflate(…).
> At some point of time, the inflate do not return and VM binary memory
> growth to infinity until it is crashed.
> The crash is reproducible all the time with my file. The file is not
> corrupted and gzip is capable to perform it check and inflate data. The
> file becomes readable by program if it is inflated - deflated again using
> command line gzip. The header of readable file is:
> 1F 8B 08 00 6E EC E6 56 00 03 D4 BD
> I am having a challenge to debug this issue further to zlib and understand
> Do you have any suggestions on it?
> Best Regards,
> P.S: the file, I am taking about, contains confidential data and cannot be
> disclosed to community.
>  http://www.zlib.org/rfc-gzip.html
>  https://github.com/fogfish/feta/blob/master/src/gz.erl
> erlang-questions mailing list
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions