[erlang-patches] Allow the pool of hipe constants to grow at runtime

[Paul Guyot]

Le 30 août 2010 à 15:41, Björn Gustavsson a écrit :

> On Sun, Aug 29, 2010 at 9:58 PM, Paul Guyot <pguyot@REDACTED> wrote:
>> Hello,
>> 
>> The hipe constants pool was previously set to a static, non-growable pool of 1.5 M words. While this works in most setups, an OTP deployment with many natively compiled modules, including modules with a lot of constants, would crash with:
>> 
>> Native code constants pool depleted!
>> 
> 
> Would it not be better to have a separate constant pool per module,
> just as BEAM handles it?

I believe that BEAM has a separate constant pool per module because constants probably go away when a module is unloaded. In HiPE, constants currently never go away. In other words, the bug is two folded:
- a memory leak with code replacement,
- a "Native code constants pool depleted!" crash when the pool is depleted.

Until the leak is fixed (with a rewrite of the loader in C ?) I suggest to simplify the code and to remove the unnecessary constant pool implementation. This would at least avoid the crash, as well as avoid unnecessary allocations. A typical erl shell configured with --enable-native-libs will have 32K words, the pool has a fixed size of 1.5 M words. We encountered this crash with nodes that allocate more than 3.5M words on boot.

A new patch is available here:

git fetch git://github.com/pguyot/otp.git pg/remove-hipe-constants-pool
http://github.com/pguyot/otp/commit/2f00cf20e222dd1ad5c0c48b3808821640ff57a6

This patch removes hipe_bifs:show_literals/0, but maintains hipe_bifs:constants_size/0. Both are debugging undocumented functions, but hipe_bifs:show_literals/0 always returned true and printed lines on the console, so I guess it is not used in any faceless program, while hipe_bifs:constants_size/0 returns a value.

Mikael, I understand that the OTP team prefers git fetch command lines, but you can see the patch in plain old diff format here:
http://github.com/pguyot/otp/commit/2f00cf20e222dd1ad5c0c48b3808821640ff57a6.diff

Regards,

Paul
-- 
Semiocast                    http://semiocast.com/
+33.175000290 - 62 bis rue Gay-Lussac, 75005 Paris