How to get HugePages to be used on Linux?

[Vans S]

 This is pretty useful thanks, we noticed often we would have 30% CPU usage via microstate accounting for the allocator.  Bumped the min_heap_size but looking for other possible optimizations.  I think then we can disable hugepages set as kernel cmdline for now and just allocate a large chunk of the system ram as a supercarrier.
    On Wednesday, May 6, 2020, 12:31:45 p.m. EDT, Mikael Pettersson <mikpelinux@REDACTED> wrote:  
 
 On Wed, May 6, 2020 at 4:51 PM Vans S <vans_163@REDACTED> wrote:
>
> Has anyone gotten Erlang to use hugepages on linux?  I allocated the default 2MB pages via kernel param, but I see they are not used when I run erlang, I also made it allocated in 2047 sized blocks as recommended by WhatsApp team (tho why 2047 and not 2048).
>
> Also what about disabling THP, anyone tested with and without?

I did some experiments a few years ago with hugepages and 2TB RAM servers.

First THP (transparent hugepages) did not work well, as they caused
major responsiveness issues whenever the kernel decided to "do stuff"
behind our back.  We always disable THP, and folks at EUC (now Code
BEAM Sto) mentioned doing the same.  Maybe the 4.x/5.x kernels have
improved, but CentOS 7 is definitely affected.  YMMV.

Second, without THP you need to tell the kernel explicitly to give you
hugepages via a flag in mmap().  Back then (R15-ish) OTP wouldn't do
that so I had to hack it in, but it didn't help much as we were
dominated by other costs.

Later on they added Super Carriers (pre-allocation), and that's a
must-have for large RAM nodes.  Mapping those with explicit hugepages
should be beneficial, but OTP would still need a patch to do so.  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20200507/fe6e8c40/attachment.htm>