[erlang-questions] Maximising memory compactness

[Olivier BOUDEVILLE]

Hello all,

Let's suppose one would like to run a large, distributed (Erlang) 
application, with many processes, and that ends up, in some cases, being 
RAM-bound.

In order to squeeze more processes into each node, apart from using the 
best data-structures to reduce memory footprint and from having the least 
active processes hibernate, I imagine that a good option could be to rely 
on "smaller pointers" (as they must be used internally a lot by the VM, 
for most terms), i.e. either using the 64-bit half-word emulator or 
instanciating as many 32-bit VMs as needed to "pave" the actual RAM of 
each host, and have them communicate. Apparently large gains in terms of 
memory could be expected.

The half-word emulator would seem the simplest/most elegant solution to do 
so (ex: I suppose that CPU extended registers, instruction sets, etc. can 
then be used - which wouldn't be the case for VMs compiled for 32-bit 
addressing; moreover 32-bit VMs would probably have to communicate through 
the loopback or alike, involving I suppose much marshalling/demarshalling) 
but apparently:
        - we currently cannot have both the half-word emulator *and* the 
HiPe-based native compilation, so we have to choose between RAM and CPU 
(don't know if targeting Erllvm could help)
        - more importantly, with the half-word emulator, the total RAM for 
stacks + heaps for processes would still have to be under 4GB (as stated 
apparently by 
http://www.erlang-factory.com/upload/presentations/569/Halfword_Erlang_Factory_SF_2012.pdf
) 

So, my question: would relying on a set of 32-bit VMs per host, instead of 
a single 64-bit VM, look like the best option here?

Thanks in advance for any hint!

Best regards,

Olivier.
---------------------------
Olivier Boudeville

EDF R&D : 1, avenue du Général de Gaulle, 92140 Clamart, France
Département SINETICS, groupe ASICS (I2A), bureau B-226
Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 
65 27 13



Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.

Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.

Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
____________________________________________________

This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.

If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.

E-mail communication cannot be guaranteed to be timely secure, error or virus-free.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140404/a6074184/attachment.htm>