[erlang-questions] Turning rebar into a package manager
Yurii Rashkovskii
yrashk@REDACTED
Sat Oct 20 21:24:01 CEST 2012
Joe,
The real question is what a "package manager" is supposed to do. What are
the problems that it should solve?
My current take is:
1. Library (package) discovery (search)
2. Library location/retrieval
3. Dependencies
4. Library publishing (enabling 1)
After my experience with Agner (http://erlagner.org) I've moved onto a much
more lightweight approach I am using in EXPM (http://expm.co)
[http://rashkovskii.com/2012/10/01/expm-or-meet-agner-2/].
It is essentially an index and, technically speaking, EXPM does not know
how to retrieve or build libraries in that index. It does know, however,
how to instruct rebar, mix or an scm of choice (git only at the moment) to
get the library. If you look at some package, for
example http://expm.co/cowboy/head, at the bottom you'll see those
automatically generated "instructions". The command line tool is capable of
exactly that as well.
I believe that EXPM currently addresses 1 & 2 to a reasonable extent and it
has an incomplete support for 3 (dependencies). Not by design, just because
it is young. It can figure out the exact versions of dependencies given
constraints, but it doesn't do dependencies' dependencies (yet). This can
be easily circumvented by writing a simple shell script that'll run expm
deps over expm deps output (recursion!)
EXPM also does a fairly good job at 4 — you can easily publish a package
with no need to go through a registration procedure (yet your package
updates are protected). It isn't without minor flaws, but it works pretty
well so far.
I personally think that either rebar or mix are good tools for retrieving
dependencies and EXPM or another similar tool to assist with finding what
libraries to use and where are they located is a good way to go.
Just my 2c
On Friday, October 19, 2012 3:36:34 AM UTC-7, Joe Armstrong wrote:
>
> Here is a programming exercise ...
>
> I think we can turn rebar into a package manager with 4 simple and one not
> so
> simple tweaks.
>
> This would turn rebar from being something that is very useful to
> something
> that is very very useful :-)
>
>
> (aside - I'm revising my Erlang book, and I'd like to describe
> how package management works, but there is no package manager
> which is a shame - so I got to thinking about package managers
> and wondered why still don't have a decent package manager.
>
> The one I've seen that I like a lot is npm (node package manager)
> so we could do worse than follow their design)
>
> So this is how I think rebar could become a package manager
>
>
> Tweak 1
> =======
>
> > rebar get-deps
>
> Fetches the dependencies in rebar.conf and stores them in
> the current directory, in a directory called deps.
>
> After a while you get dependencies scattered all over the place,
> I've seen many examples of this.
>
> This means that there is no central place to go and look if you
> want to find code.
>
> NPM sticks ALL dependencies under ${HOME}/.npm
>
> Suggestion:
>
> All dependencies should be stored in ${HOME}/.erl_packages
>
> (Aside - if we followed npm the directory structure would be
> something like)
>
> ${HOME}/.erl_packages/cowboy/6.2.1/src
> /ebin
> /5.2.3
> /stdlib/12.3.1
> /14.6.8
>
> etc. ie a package/Vsn/src ... structure
>
>
> Tweak 2
> =======
>
> move rebar.config to ${HOME}/.erl_packages/rebar.config
> and add commands to automatically edit it
>
> > rebar add_package foo
>
> might add a line like
> {foo, ".*", "git://...."}
>
> to rebar.config
>
> The point here is to change the content of rebar.config with
> shell commands rather that editing it by hand
>
> Tweak 3
> =======
> fix rebar so it just downloads a particular version of a
> program and not all the .git stuff - most people will only
> want to use other peoples code, not hack it.
>
> Tweak 4
> =======
>
> Fix the erlang load paths to find all the dependencies
>
> I do this now. Put all my dependencies in
> ${HOME}/nobackup/erlang_imports/deps and put the following
> code in my .erlang startup file
>
> Home = os:getenv("HOME").
> Dir = Home ++ "/nobackup/erlang_imports/deps",
> case file:list_dir(Dir) of
> {ok, L} ->
> lists:foreach(fun(I) ->
> Path = Dir ++ "/" ++ I ++ "/ebin",
> code:add_path(Path)
> end, L);
> _ ->
> void
> end.
>
> Tweak 5
> =======
> This can't be done by rebar
> Make an index of all erlang apps on github that follow
> the erlang packaging structure
>
> Somebody has to write a program to do this.
>
> The package index should be at a well know URL
>
> > rebar get_index
>
>
> This should fetch the index from the well-know URL and store in
> ${HOME}/.erl_packages
>
>
> > rebar search XXX
>
> would search the fetched index
>
> > rebar add XXXX
>
> would take the index entry add it to the config fill fetch the code
> and compile it
>
> Note the following:
>
> 1) The trust model.
>
> We trust the supplier of a program not the program.
>
> So for example on github we might trust programs
> published by the user joearms (me) or erlang (the OTP release)
> but not by some_guy_ive_never_hear_of
>
> It would be quite easy to add a trust section to rebar.config
>
> {trust, ["git://joearms", git://erlang", ...]}
>
> 2) There is no "publish" step
>
> We put our code on github (or somewhere) and hope that it gets indexed
> by the indexer.
>
> We might mail the indexer if it is published in an obscure place.
>
> Comments?
>
> Cheers
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-q...@REDACTED <javascript:>
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121020/f9dd39de/attachment.htm>
More information about the erlang-questions
mailing list