[erlang-questions] Customizing Beam Build

Peer Stritzinger peer@REDACTED
Tue Sep 26 16:56:31 CEST 2017


Hi Joe,

On 26. Sep 2017, at 16:12, Joe Armstrong <erlang@REDACTED <mailto:erlang@REDACTED>> wrote:

> My only (I think) problem are the makefiles. I tried reading them
> and they were pretty complex - I couldn't see where to change the code to
> make a static image.

You shouldn’t need to touch the Makefiles  key is:

https://github.com/erlang/otp/tree/master/xcomp <https://github.com/erlang/otp/tree/master/xcomp>

You make a copy of one of the files in there and edit that copy.

Together with the description how to cross compile Erlang:

https://github.com/erlang/otp/blob/master/HOWTO/INSTALL-CROSS.md <https://github.com/erlang/otp/blob/master/HOWTO/INSTALL-CROSS.md>

Then for the target system set CC CFLAGS LDFLAGS etc in your copy in a way that makes your C compiler link statically.

> I think If I could make a static version of erlexec, it would become clearer
> to me.

Unfortunately the argument massaging is done in a separate executable and then the proper emulator i.e. erl_main.c is exec() called.

That makes it also harder for us since we need to duplicate what erlexec does to the environment and the args.  We just leave away erlexec and set the environment variables before calling erl_main() for arguments we keep to the subset that are understood by erl_main() directly without massaging.

> I wrote the original ring0 etc stuff but have never been deeply in the C and
> makefiles stuff.
> 
> Any advice would be welcome.
> 
> Cheers
> 
> /Joe
> 
> On Tue, Sep 26, 2017 at 3:49 PM, Peer Stritzinger <peer@REDACTED <mailto:peer@REDACTED>> wrote:
>> For GRiSP we do have a statically linked version of Erlang, what’s the problem of statically linking it?  It just needs the right build system with static libraries.
>> 
>> As for the forking OTP to get some specialized version, we have a workaround in GRiSPs tooling.   There is a rebar3 plugin that is able to build a specialized version of OTP from source.  One issue we need to solve: adding our own statically linked drivers
>> and static NIFs to this statically linked beam.  The plugin collects all C drivers etc from all the dependencies and patches them into the OTP source.  Some just get copied over
>> but e.g. one of the Makefile.in in OTP needs to be patched to add our static drivers to the driver table.
>> 
>> We even cross build with this setup and go from nothing to SD card our NAND flash filesystem contents with cross built statically linked beam plus Erlang release.
>> 
>> BTW cross building could be a way to build a special Erlang for your hostsystem ... nothing prevents from the cross target to be the same as the host architecture with some build flags tweaked.
>> 
>> One caveat remains: there are a few applications in OTP that rely on dynamic linking.  These can be either configured out (currently we still have our patched OTP but we are moving towards getting rid of this requirement) or since there is support for static NIFs in OTP now be made possible static.
>> 
>> Cheers,
>> — Peer
>> 
>>> On 25. Sep 2017, at 22:06, Joe Armstrong <erlang@REDACTED <mailto:erlang@REDACTED>> wrote:
>>> 
>>> Everything would be a lot simpler if I could build a static version of erlexec
>>> 
>>> At some level abstraction we have a list of C files that must be compiled
>>> and linked together.
>>> 
>>> If the C files and the header files were all in one directory this would be
>>> easy - as it is the makefiles are generated automatically and the C
>>> files that are needed to build the system are all over the place.
>>> 
>>> All I need is a makefile to build a static version of erlexec. It is not obvious
>>> to me how to do this (otherwise I would have done it :-)
>>> 
>>> Cheers
>>> 
>>> /Joe
>>> 
>>> 
>>> 
>>>> On Mon, Sep 25, 2017 at 9:32 PM, Tristan Sloughter <t@REDACTED <mailto:t@REDACTED>> wrote:
>>>> I like the idea of easily built static builds but for dealing with
>>>> releases being messy (if I'm understanding what you mean) is something
>>>> that may be solved simply with existing tools like makeself:
>>>> 
>>>> http://makeself.io/ <http://makeself.io/>
>>>> 
>>>> --
>>>> Tristan Sloughter
>>>> "I am not a crackpot" - Abe Simpson
>>>> t@REDACTED <mailto:t@REDACTED>
>>>> 
>>>>> On Sun, Sep 24, 2017, at 01:25 AM, Sargun Dhillon wrote:
>>>>> I've customized my OTP build to have a custom Erlang build. The reason
>>>>> I've done this to begin the process of building a fully-self contained
>>>>> Erlang program. One of the biggest benefits of the infrastructure in
>>>>> Go is that they're fully self-contained statically linked binaries. In
>>>>> Erlang, although you can statically link beam itself, unfortunately,
>>>>> the releases themselves are messy, in that you have a lot of files and
>>>>> getting the execution environment right isn't always trivial.
>>>>> 
>>>>> Eventually, the purpose is to build an init system -- something that
>>>>> will be primarily used by sysadmin-types. These need to be statically
>>>>> linked, and often times, the file system isn't fully-functioning when
>>>>> they're starting.
>>>>> 
>>>>> I've done two things:
>>>>> 
>>>>> 1) I've modified the set of preloaded files in the VM. It would be
>>>>> really nice if there was a way to do this without forking OTP. I
>>>>> didn't add any existing beam files, but I just changed around some of
>>>>> the file, and code loading code to my need.
>>>>> 
>>>>> 2) I've modified the final, linked product to include custom objects
>>>>> in the final ELF binary.
>>>>> 
>>>>> This is kind of terrible, as it requires I maintain a fork of OTP.
>>>>> Does anyone have any recommendations for solving this problem another
>>>>> way?
>>>>> 
>>>>> My proposal would be to output a beam.a file with the requisite
>>>>> objects to build a BEAM installation. It would be valuable if I could
>>>>> statically link against this binary, and own main, if it had a
>>>>> separate entrypoint that I could then call into. I've toyed around
>>>>> with this, and for the unix build, it seems possible. Is there any
>>>>> reason why Erlang isn't distributed this way already?
>>>>> 
>>>>> If there was a public API for extending the loader, or adding
>>>>> preloaded modules via sections, I imagine that solving (1) would be
>>>>> easy as well.
>>>>> 
>>>>> Is anyone open to accepting this upstream if the work is done?
>>>>> _______________________________________________
>>>>> erlang-questions mailing list
>>>>> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>>>>> http://erlang.org/mailman/listinfo/erlang-questions <http://erlang.org/mailman/listinfo/erlang-questions>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>>>> http://erlang.org/mailman/listinfo/erlang-questions <http://erlang.org/mailman/listinfo/erlang-questions>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>>> http://erlang.org/mailman/listinfo/erlang-questions <http://erlang.org/mailman/listinfo/erlang-questions>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170926/7f9a0853/attachment.htm>


More information about the erlang-questions mailing list