[erlang-questions] "Fat" BEAMs?

Mon Oct 14 14:03:51 CEST 2013

Hi Olivier,

Sorry, the code is not public at this time. But it's basically structured around doing many different passes over functions, modules, etc., using erlang syntax trees and profile annotations to guide what to do. Some graph algorithms, some heuristics, a lot of syntax tree passes ... 

Regarding your second observation: As I understand it, the beam compiler nowadays does the equivalent of the higher-order removal I looked at in the first paper (that is, it optimizes/specializes the most common lists functions). It might be interesting to go further in this direction (though doing so is substantially more ambitious). In particular, I'm thinking of the Haskell work on fusing higher order functions, which has yielded some very interesting results:

http://www.reddit.com/r/haskell/comments/1br0ls/haskell_beats_c_using_generalised_stream_fusion/

Best,

Thomas

On Monday, October 14, 2013 11:04 AM, Olivier BOUDEVILLE <olivier.boudeville@REDACTED> wrote:

Hello Thomas, 
>
>Thanks for these pointers. I wasn't
aware that higer-order removal could be so fruitful. Would the code of
"OM" be publicly available/usable, for inspiration? 
>
>Best regards, 
>
>Olivier.
>---------------------------
>Olivier Boudeville
>
>EDF R&D : 1, avenue du Général de Gaulle, 92140 Clamart, France
>Département SINETICS, groupe ASICS (I2A), bureau B-226
>Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47
65 27 13 
>
>
>
>thomasl_erlang@REDACTED  
>11/10/2013 08:19 
>Veuillez répondre à
>thomasl_erlang@REDACTED 
> A olivier.boudeville@REDACTED, tony@REDACTED  
>cc erlang-questions@REDACTED  
>Objet Re: [erlang-questions] "Fat"
BEAMs? 
>  
> 
>
>
>I should also add that Frej Drejhammar's
work on a tracing JIT emulator would enable cross-module inlining etc in
a somewhat different context, so it will be quite interesting to see what
comes from that. I've only seen a couple of conference presentations yet,
looking forward to more. 
>
>Best, 
>Thomas 
>
> 
>On Friday, October 11, 2013 8:09 AM, Thomas
Lindgren <thomasl_erlang@REDACTED> wrote: 
>Hi Olivier, 
>
>Regarding cross-module optimization and inlining,
have a look at these: 
>
>http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.21.4909 
>http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.2.6496 
>
>Appendix A of the second paper made me realize
how much work is going on inside mnesia. For one benchmark, that algorithm
decided to inline mnesia:dirty_write/2 something like 10 levels deep through
8 modules, resulting in a 1000-line function. The whole topic needs more
work though. 
>
>Best, 
>Thomas 
>
> 
>On Thursday, October 10, 2013 1:23 PM, Olivier
BOUDEVILLE <olivier.boudeville@REDACTED> wrote: 
>Indeed! Additionally, it is quite recent. 
>
>It could be interesting to measure the speed-up that can actually be obtained,
possibly in various settings (native or not, with the various compilers,
etc.); maybe this could also allow to have Erlang shine a bit more in some
benchmarks!
>
>Thanks for the pointer (will definitively try it),
>
>Olivier.
>---------------------------
>Olivier Boudeville
>
>EDF R&D : 1, avenue du Général de Gaulle, 92140 Clamart, France
>Département SINETICS, groupe ASICS (I2A), bureau B-226
>Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47
65 27 13
>
> 
>tony@REDACTED  
>10/10/2013 12:34  A olivier.boudeville@REDACTED  
>cc erlang-questions@REDACTED  
>Objet Re: [erlang-questions] "Fat"
BEAMs? 
>
>  
> 
>
>
>
>Maybe give https://github.com/tonyrog/inlinea try? 
>
>/Tony 
>
>On 10 okt 2013, at 11:56, Olivier BOUDEVILLE <olivier.boudeville@REDACTED>
wrote: 
>
>Hi,
>
>Let's suppose that, thanks to a parse-transform, we fetched recursively
from other modules most if not all code explicitly callable from a given
module, so that the full code (ultimately calling only basic language constructs
and BIFs) ends up in a single (presumably big, unfolded, possibly inlined)
AST. 
>
>Then, in  the prospect of native compilation (HiPe, possibly others
like ErLLVM), I imagine that not only we should get rid of the overhead
of inter-module calls, but we would also give to the compiler the best
full view on which it could operate and generate the best code it could
manage?
>
>Has this approach already been attempted? Would it have some potential
interest?
>
>Thanks in advance for any hint,
>Best regards,
>
>Olivier.
>---------------------------
>Olivier Boudeville
>
>EDF R&D : 1, avenue du Général de Gaulle, 92140 Clamart, France
>Département SINETICS, groupe ASICS (I2A), bureau B-226
>Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47
65 27 13 
>
>Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis
à l'intention exclusive des destinataires et les informations qui y figurent
sont strictement confidentielles. Toute utilisation de ce Message non conforme
à sa destination, toute diffusion ou toute publication totale ou partielle,
est interdite sauf autorisation expresse.  
>Si vous n'êtes pas le destinataire de ce
Message, il vous est interdit de le copier, de le faire suivre, de le divulguer
ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur,
merci de le supprimer de votre système, ainsi que toutes ses copies, et
de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercions également d'en avertir immédiatement l'expéditeur par retour
du message.  
>Il est impossible de garantir que les communications
par messagerie électronique arrivent en temps utile, sont sécurisées ou
dénuées de toute erreur ou virus.
>____________________________________________________  
>This message and any attachments (the 'Message')
are intended solely for the addressees. The information contained in this
Message is confidential. Any use of information contained in this Message
not in accord with its purpose, any dissemination or disclosure, either
whole or partial, is prohibited except formal approval.  
>If you are not the addressee, you may not
copy, forward, disclose or use any part of it. If you have received this
message in error, please delete it and all copies from your system and
notify the sender immediately by return message.  
>E-mail communication cannot be guaranteed
to be timely secure, error or virus-free.  
>_______________________________________________
>erlang-questions mailing list
>erlang-questions@REDACTED
>http://erlang.org/mailman/listinfo/erlang-questions
>
>"Installing applications can lead to corruption over time. Applications
gradually write over each other's libraries, partial upgrades occur, user
and system errors happen, and minute changes may be unnoticeable and difficult
to fix" 
>
>
>
> 
>
>Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis
à l'intention exclusive des destinataires et les informations qui y figurent
sont strictement confidentielles. Toute utilisation de ce Message non conforme
à sa destination, toute diffusion ou toute publication totale ou partielle,
est interdite sauf autorisation expresse. 
>Si vous n'êtes pas le destinataire de ce
Message, il vous est interdit de le copier, de le faire suivre, de le divulguer
ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur,
merci de le supprimer de votre système, ainsi que toutes ses copies, et
de n'en garder aucune trace sur quelque support que ce soit. Nous vous
remercions également d'en avertir immédiatement l'expéditeur par retour
du message. 
>Il est impossible de garantir que les communications
par messagerie électronique arrivent en temps utile, sont sécurisées ou
dénuées de toute erreur ou virus.
>____________________________________________________ 
>This message and any attachments (the 'Message')
are intended solely for the addressees. The information contained in this
Message is confidential. Any use of information contained in this Message
not in accord with its purpose, any dissemination or disclosure, either
whole or partial, is prohibited except formal approval. 
>If you are not the addressee, you may not
copy, forward, disclose or use any part of it. If you have received this
message in error, please delete it and all copies from your system and
notify the sender immediately by return message. 
>E-mail communication cannot be guaranteed
to be timely secure, error or virus-free. 
>
>_______________________________________________
>erlang-questions mailing list
>erlang-questions@REDACTED
>http://erlang.org/mailman/listinfo/erlang-questions 
>
>
> 
>
>_______________________________________________
>erlang-questions mailing list
>erlang-questions@REDACTED
>http://erlang.org/mailman/listinfo/erlang-questions 
>
> 
>
>
>Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse.
>Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message.
>Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus.
>____________________________________________________
>This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval.
>If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message.
>E-mail communication cannot be guaranteed to be timely secure, error or virus-free.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20131014/334ad045/attachment.htm>