[erlang-questions] Reproducible and deterministic builds for GNU Guix (and Nix)

[Pjotr Prins]

On Tue, Apr 05, 2016 at 02:07:46PM +0100, Magnus Henoch wrote:
>    Debian has included a patch that lets you use the environment variable
>    SOURCE_DATE_EPOCH to fix the compile time, and thus obtain identical
>    output (given the same compiler version and other things):
>    [1]https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795834
> 
>    This was briefly discussed on this mailing list:
>    [2]http://erlang.org/pipermail/erlang-questions/2015-January/082699.html

You may have heard of GNU Guix, the modern (functional) package
manager of the GNU project. We are trying to add Erlang and Elixir to
Guix, but we are running into the problem that building the Erlang
compiler is not deterministic and therefore not reproducible, i.e. the
beam files contain time stamps. 

For normal software built by Erlang this can be overriden with
SOURCE_DATE_EPOCH (as per mentioned Debian patch), but for the
compiler itself we have not found how to do this.

Do you have a suggestion how to bootstrap the compiler with
SOURCE_DATE_EPOCH set or disable the time stamps? I am sure as a FP
compiler designer you can appreciate determinism. Because GNU Guix is
deterministic there is no need to keep track of time stamps. For hot
reloading we can assume the start of EPOCH will do the trick, right?

Pj.

On Mon, Apr 04, 2016 at 01:49:44PM -0400, Leo Famulari wrote:
> On Mon, Apr 04, 2016 at 12:50:12PM -0400, Leo Famulari wrote:
> > On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote:
> > > On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote:
> > > > Debian's package exhibits this problem. The timestamps are generated in
> > > > the following places in the source code. I don't know how to approach
> > > > this problem.
> > > > 
> > > > lib/kernel/test/global_SUITE_data/global_trace.erl:    io:format("The trace was generated at ~p~n", [EndTime]),
> > > > lib/reltool/bin/reltool.escript:    lists:flatten(io_lib:format("%% ~s generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_server.erl:    IoList = io_lib:format("%% config generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:    RelIoList = io_lib:format("%% rel generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:    ScriptIoList = io_lib:format("%% script generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > > lib/reltool/src/reltool_target.erl:            AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n",
> > > > lib/runtime_tools/src/erts_alloc_config.erl:	"generated at ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by "
> > > > lib/sasl/src/systools_make.erl:	    io:format(Fd, "%% script generated at ~w ~w\n~p.\n",
> > > > lib/wx/src/gen/gl.erl:%% The program object's information log is updated and the program is generated at the time
> > > 
> > > If there is no easy work around I suggest simply patching them. Fortunately
> > > the Erlang compiler does not change much at this level.
> > 
> > The ideal solution would be to use the value of the environment variable
> > SOURCE_DATE_EPOCH if it is set, and else to behave as it does now.
> > 
> > > We can also contact Joe Armstrong, the author of Erlang, to discuss
> > > this point. He appears to be approachable. I am sure he is open to
> > > the idea of deterministic builds in a deterministic build system ;)
> > 
> > I could go to the Erlang IRC channel or forums (whatever they use) and
> > ask for advice. Since you are actually using Erlang, I think you would
> > be the better person to contact Joe Armstrong himself, if we decide to
> > do that.
> 
> I presented the situation on IRC and it was recommended that I start the
> discussion on a mailing list.
> 
> I think that the erlang-questions list [0] could be a good place to
> start.
> 
> Pjotr, would you like to start the conversation? I can do it if you are
> too busy or something.
> 
> [0]
> http://www.erlang.org/community
> 

-- 

> 
>    Regards,
>    Magnus
> 
>    On Mon, Apr 4, 2016 at 8:59 PM, Joe Armstrong <[3]erlang@REDACTED> wrote:
> 
>      Hello,
> 
>      I think I've asked this before but cannot find the answer:
> 
>      I want the beam file produced by
> 
>        $ erl file.erl
> 
>      to always have the same sha1 checksum - there was, if I remember
>      correctly, a hidden flag that removed the time of compilation etc from
>      the beam code. Any ideas how to do this?
> 
>      /Joe
>      _______________________________________________
>      erlang-questions mailing list
>      [4]erlang-questions@REDACTED
>      [5]http://erlang.org/mailman/listinfo/erlang-questions
> 
> References
> 
>    Visible links
>    1. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795834
>    2. http://erlang.org/pipermail/erlang-questions/2015-January/082699.html
>    3. mailto:erlang@REDACTED
>    4. mailto:erlang-questions@REDACTED
>    5. http://erlang.org/mailman/listinfo/erlang-questions

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions


--