[erlang-questions] term_to_binary and large data structures

Fred Hebert mononcqc@REDACTED
Thu Jun 28 01:14:10 CEST 2018

On 06/27, Aaron Seigo wrote:
>We have maps with 10k keys that strain this system and easily saturate 
>our network. This is not "big" by any modern definition. As a 
>demonstration of this to ourselves, I wrote an Elixir library that 
>serializes terms to a more space efficient format. Where 
>`term_to_binary` creates 500MB monsters, this library conveniently 
>creates a 1.5MB binary out of the exact same data.

Have you tried comparing when `term_to_binary(Term, [{compressed, 9}])'?  
If you can pack 500MB of data down to 1.5 MB, chances are that 
compression could do some good things on your end.


More information about the erlang-questions mailing list