[erlang-questions] Turning rebar into a package manager

[Joe Armstrong]

Here is a programming exercise ...

I think we can turn rebar into a package manager with 4 simple and one not so
simple tweaks.

This would turn rebar from being something that is very useful to something
that is very very useful :-)


(aside - I'm revising my Erlang book, and I'd like to describe
how package management works, but there is no package manager
which is a shame - so I got to thinking about package managers
and wondered why still don't have a decent package manager.	

The one I've seen that I like a lot is npm (node package manager)
so we could do worse than follow their design)

So this is how I think rebar could become a package manager


Tweak 1
=======

   > rebar get-deps

Fetches the dependencies in rebar.conf and stores them in
the current directory, in a directory called deps.

After a while you get dependencies scattered all over the place,
I've seen many examples of this.

This means that there is no central place to go and look if you
want to find code.

NPM sticks ALL dependencies under ${HOME}/.npm

Suggestion:

   All dependencies should be stored in ${HOME}/.erl_packages

   (Aside - if we followed npm the directory structure would be
    something like)

   ${HOME}/.erl_packages/cowboy/6.2.1/src
                                                               /ebin
                                                     /5.2.3
                        /stdlib/12.3.1
                               /14.6.8

   etc. ie a package/Vsn/src ... structure


Tweak 2
=======

   move rebar.config to ${HOME}/.erl_packages/rebar.config
   and add commands to automatically edit it

   > rebar add_package foo

   might add a line like
   {foo, ".*", "git://...."}

   to rebar.config

   The point here is to change the content of rebar.config with
   shell commands rather that editing it by hand

Tweak 3
=======
   fix rebar so it just downloads a particular version of a
   program and not all the .git stuff - most people will only
   want to use other peoples code, not hack it.

Tweak 4
=======

Fix the erlang load paths to find all the dependencies

I do this now. Put all my dependencies in
${HOME}/nobackup/erlang_imports/deps and put the following
code in my .erlang startup file

  Home = os:getenv("HOME").
  Dir = Home ++ "/nobackup/erlang_imports/deps",
  case file:list_dir(Dir) of
     {ok, L} ->
         lists:foreach(fun(I) ->
			       Path = Dir ++ "/" ++ I ++ "/ebin",
			       code:add_path(Path)
		       end, L);
    _ ->
	void
  end.

Tweak 5
=======
   This can't be done by rebar
   Make an index of all erlang apps on github that follow
   the erlang packaging structure

   Somebody has to write a program to do this.

   The package index should be at a well know URL

   > rebar get_index


   This should fetch the  index from the well-know URL and store in
   ${HOME}/.erl_packages


   > rebar search XXX

   would search the fetched index	

   > rebar add XXXX

   would take the index entry add it to the config fill fetch the code
   and compile it

Note the following:

1) The trust model.

   We trust the supplier of a program not the program.

   So for example on github we might trust programs
   published by the user joearms (me) or erlang (the OTP release)
   but not by some_guy_ive_never_hear_of

   It would be quite easy to add a trust section to rebar.config

   {trust, ["git://joearms", git://erlang", ...]}

2) There is no "publish" step

   We put our code on github (or somewhere) and hope that it gets indexed
   by the indexer.

   We might mail the indexer if it is published in an obscure place.

  Comments?

   Cheers

/Joe