[erlang-questions] zlib design flaw?

[Christopher Vance]

The zlib protocol is fully specified, and it appears the code is working
correctly, so the only issue you might have is deciding what to do when you
see input which could be seen as malicious (but which is still properly
defined). Do you want to abort handling the input stream, or do you have an
alternate suggestion?

If you don't like the way zlib does it, or prefer a compression scheme
which is more predictable, why not try a different compression algorithm,
and see if it does any better in these pathological cases. I can specify
you several 1:1 "compression" schemes where you only ever get out the same
number of bytes you put in, but you'd probably think these safe method are
too boring.

On Wed, Sep 24, 2014 at 1:15 PM, Guilherme Andrade <g@REDACTED> wrote:

>  Hi Sungjin,
>
> I've recently dealt this with very same issue, albeit only as a security
> hardening and a prevention measure; and, like you, I've looked into the
> zlib C code that's bundled with erts and arrived at that same conclusion.
>
> I was only able to limit it on a theoretical basis: the zlib guys
> themselves state[1] that a maximum compression ratio of 1032:1 is
> achievable (with big blobs of zeroes.) Therefore, if I want to limit the
> uncompressed content to less than, let's say, 5 MiB, I'll only accept
> compressed content of up to ~5 KiB. This thinking might be missing
> something, though.
>
> If there's a better/cleaner way to deal with this, I would love to know.
>
> Cheers,
>
>
> [1]: http://www.zlib.net/zlib_tech.html
>
>
>
> On 24-09-2014 03:55, Park, Sungjin wrote:
>
> Hi, I'm about to report a problem with erlang's zlib library interface
> which I think is a design flaw at this point of time.
>
>  We recently had some malicious packets which were not very big in the
> first place but inflated to really big ones - hundreds of megabytes each.
> As a result, the server crashed with out-of-memory by the processes calling
> zlib:inflate/2.  Urgency forced us to make a custom NIF library with
> inflation size limit.  We also studied erlang reference manual but couldn't
> find anything useful.  The zlib library source code shows even
> zlib:setBufSize/2 does not prevent producing very big binaries.
>
>  Not being able to know how big the data would become after inflation, it
> should be a quite common problem.  So I'm curious if I missed something
> very simple and nice.  Is there anything like that?
>
>  --
> Park, Sungjin
>
> -------------------------------------------------------------------------------------------------------------------
> Peculiar travel suggestions are dancing lessons from god.
>   -- The Books of Bokonon
>
> -------------------------------------------------------------------------------------------------------------------
>
>
> _______________________________________________
> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>
>
> --
> Guilherme
> https://www.gandrade.net/
> PGP: 0x35CB8191 / 1968 5252 3901 B40F ED8A  D67A 9330 79B1 35CB 8191
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>


-- 
Christopher Vance
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140924/fefa88f4/attachment.htm>