[erlang-questions] [Nix-dev] Erlang Package Manager

[stewart mackenzie]

On Wed, Dec 31, 2014 at 11:47 PM, Anthony Ramine <n.oxyde@REDACTED> wrote:
> How do you handle multiple versions of a single package?

Okay allow me to give a rather detailed answer describing in how NixOS
works. I'll also describe how I envisage Erlang adopting Nix as its
package manager.

So I like to think of Nix as the Matrix, you, Neo, can reconfigure
your world at a command. Your entire environment is made up of
thousands of symlinks that can shifted about in an instant. Dodge
speeding bullets - no problem.

Now some code.

A convenient way of creating an environment is using nix-shell. These
files sit in your project folder.

(contents of dev-shell)
#! /bin/sh
s=$(type -p nix-shell)
exec $s release.nix -A erlangEnv

(contents of release.nix)
let
  pkgs = import <nixpkgs> {};
    stdenv = pkgs.stdenv;
  in rec {
    erlangEnv = stdenv.mkDerivation rec {
      name = "erlang-env";
      version = "1.1.1.1";
      src = ./.;
      buildInputs = [ pkgs.erlang_odbc pkgs.git pkgs.openssl
pkgs.libyaml pkgs.expat pkgs.zlib pkgs.libcouchbase pkgs.automake
pkgs.autoconf pkgs.rebar ];
    };
  }

Typically the above is used for short quick projects they are project
specific environments.

See the above 'pkgs.erlang_odbc' that currently points to erlang 17
with odbc support. I can change that to 'pkgs.erlangR16' and it'll
point to Erlang R16.03.

These are the Erlang versions supported
https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/interpreters/erlang
They are exposed via the all-packages.nix file
https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/all-packages.nix#L3994

You cannot have Erlang R16 and Erlang R17 in the same environment you
will get a conflict.

One could commit dev-shell and release.nix into your repo and
thereafter run  './dev-shell' and you'll leave your current
environment and an environment as dictated by the release.nix will be
created for you.

Now if the Erlang community is considering adopting Nix as its package
manager then the below is the way to do it: Nix already supports
python, perl, php, haskell, node, lua and guile:
https://github.com/NixOS/nixpkgs/tree/master/pkgs/top-level please
notice the language specific *.nix files.

For example
When adding a node package one simply adds the package name to this
file https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/node-packages.json
and run npm2nix on the file like such: 'npm2nix node-packages.json
node-packages-generated.nix'. The program npm2nix will go to the
canonical https://www.npmjs.com/ and download all the node packages,
update each package and create nix expressions accordingly. The
packager then commits the edited *.json file and the generated file
and makes a pull request for everyone's disposal.

So referring to the previous release.nix example, notice there are no
erlang code dependencies (yes yes forget about rebar).  one would add
'erlangPackages.cowboy'. This then gets the cowboy package that is
part of the nixpkgs.

In other words the Nix community and Erlang community need to start
banging their heads together, create an erlang-packages.nix and extend
it accordingly.

So a rebar2nix program (see https://github.com/NixOS/npm2nix and
https://github.com/NixOS/cabal2nix) could simply be pointed at a git
cloned erlang project from github and it'll parse the rebar.config
file pulling down dependencies, compiling them then making them
immutable derivatives in your nix-store. Note these packages are
obviously not available to everyone in the community. Though if you
run the very same rebar2nix program on the canonical
erlang-packages.nix in
https://github.com/NixOS/nixpkgs/tree/master/pkgs/top-level and
generate a new file, then that would become available to everyone.

Both communities are filled with terribly smart people. You'll get
along just fine.

Now, Nix also supports creating long lasting custom environments
similar to nix-shell but essentially you install this environment as a
package in your /nix/store so to speak.

I have this file ~/.nixpkgs/erlang.nix
```
# $ nix-env -iA erlangEnv -f erlang.nix -p
/nix/var/nix/gcroots/profiles/per-user/erlang
# Make a convenience link.
#      $ ln -sf
/nix/var/nix/gcroots/profiles/per-user/erlang/bin/load-env-erlang .
# Activate the environment.
#      $ ./load-env-erlang
# Optionally, rollback to an earlier generation of this profile.
#      $ nix-env --list-generations -p
/nix/var/nix/gcroots/profiles/per-user/erlang

let
      pkgs = import <nixpkgs> {};
      stdenv = pkgs.stdenv;
   in rec {
       erlangEnv = pkgs.myEnvFun {
         name = "erlang";
         buildInputs = with pkgs; [ erlang rebar openssl git binutils
bash emacs which libyaml expat zlib automake autoconf mc libtool
python libevent meld tig libpcap cmake libcouchbase pidgin ];
       };
   }
```

So this allows one to have generations such that one can rollback if
something messed up. These environments can be quite complex if
needed. You are limited by your creativity and the understanding of
the nix expression language which is a lazy declarative language
designed for managing packages. Indeed even nix-shell environment can
be as complex as you want.

A neat thing about NixOS (not Nix) is that you can configure your
entire system (every daemon) in a declarative manner from one file:
/etc/nixos/configuration.nix

Nix is the package manager, it runs on Darwin, Linux, FreeBSD and IIRC
windows. NixOS is it's own linux distro with Nix the package manager
front and centre.

Last word on immutability. Nix is very strict about immutability. So
there are multiple phases when building a package. These phases allow
for mutability, thereafter that package becomes immutable. Indeed
/nix/store is read only. This is the Nix way, it's pointless fighting
it.
This is a quick grep of erlang in my /nix/store

dr-xr-xr-x    5 root    root       4096 Jan  1  1970
xi3r2y8wapfk19717k64az9973w881a3-erlang-16B03-1-odbc
-r--r--r--    1 root    root       8835 Jan  1  1970
xjllfqh87k6d0b0mag6dajgbpdkbbmxj-env-erlangEnv.drv
-r--r--r--    1 root    root       2274 Jan  1  1970
xqa5w0xldnbi6vkrqaf53y2wz55lh11x-erlang-env.drv
dr-xr-xr-x    5 root    root       4096 Jan  1  1970
xs8pn8l7zws0ck32k3jrgj53rk3ywvzx-erlang-17.3-odbc

(did I just hear wtf?)

A very happy new year to everyone

Stewart