[erlang-questions] Turning rebar into a package manager

[Tim Watson]

I'm glad *someone* reads my (infrequently updated) blog! :)

Ironically, now that I'm looking into whether or not larger projects (like RabbitMQ) can be transitioned to rebar, I'm leaning back towards project local dependency management *precisely because* of the need to tweak version numbers and tags and the like. For example, Rabbit has many numerous components and when working on a feature you might create branches in several projects that need to be tracked in synch with one another. Whilst rebar *doesn't* do that out of the box, adding the functionality is going to be a lot simpler if the dependencies and their config files are bootstrapped locally rather than trying to use $HOME/.rebar_deps or whatever. 

On 19 Oct 2012, at 13:33, Fred Hebert wrote:

> For anyone interested, there's been an interesting blog post on the topic of package managers written at http://hyperthunk.wordpress.com/2012/05/28/does-erlangotp-need-a-new-package-management-solution/
> 
> They mention a few very interesting issues that are likely to be brought up again here, and I would recommend reading it as a nice entry point to the discussion of what problems common solutions have, and what could possibly be done to fix it.
> 
> On 12-10-19 6:36 AM, Joe Armstrong wrote:
>> Here is a programming exercise ...
>> 
>> I think we can turn rebar into a package manager with 4 simple and one not so
>> simple tweaks.
>> 
>> This would turn rebar from being something that is very useful to something
>> that is very very useful :-)
>> 
>> 
>> (aside - I'm revising my Erlang book, and I'd like to describe
>> how package management works, but there is no package manager
>> which is a shame - so I got to thinking about package managers
>> and wondered why still don't have a decent package manager.	
>> 
>> The one I've seen that I like a lot is npm (node package manager)
>> so we could do worse than follow their design)
>> 
>> So this is how I think rebar could become a package manager
>> 
>> 
>> Tweak 1
>> =======
>> 
>>    > rebar get-deps
>> 
>> Fetches the dependencies in rebar.conf and stores them in
>> the current directory, in a directory called deps.
>> 
>> After a while you get dependencies scattered all over the place,
>> I've seen many examples of this.
>> 
>> This means that there is no central place to go and look if you
>> want to find code.
>> 
>> NPM sticks ALL dependencies under ${HOME}/.npm
>> 
>> Suggestion:
>> 
>>    All dependencies should be stored in ${HOME}/.erl_packages
>> 
>>    (Aside - if we followed npm the directory structure would be
>>     something like)
>> 
>>    ${HOME}/.erl_packages/cowboy/6.2.1/src
>>                                                                /ebin
>>                                                      /5.2.3
>>                         /stdlib/12.3.1
>>                                /14.6.8
>> 
>>    etc. ie a package/Vsn/src ... structure
>> 
>> 
>> Tweak 2
>> =======
>> 
>>    move rebar.config to ${HOME}/.erl_packages/rebar.config
>>    and add commands to automatically edit it
>> 
>>    > rebar add_package foo
>> 
>>    might add a line like
>>    {foo, ".*", "git://...."}
>> 
>>    to rebar.config
>> 
>>    The point here is to change the content of rebar.config with
>>    shell commands rather that editing it by hand
>> 
>> Tweak 3
>> =======
>>    fix rebar so it just downloads a particular version of a
>>    program and not all the .git stuff - most people will only
>>    want to use other peoples code, not hack it.
>> 
>> Tweak 4
>> =======
>> 
>> Fix the erlang load paths to find all the dependencies
>> 
>> I do this now. Put all my dependencies in
>> ${HOME}/nobackup/erlang_imports/deps and put the following
>> code in my .erlang startup file
>> 
>>   Home = os:getenv("HOME").
>>   Dir = Home ++ "/nobackup/erlang_imports/deps",
>>   case file:list_dir(Dir) of
>>      {ok, L} ->
>>          lists:foreach(fun(I) ->
>> 			       Path = Dir ++ "/" ++ I ++ "/ebin",
>> 			       code:add_path(Path)
>> 		       end, L);
>>     _ ->
>> 	void
>>   end.
>> 
>> Tweak 5
>> =======
>>    This can't be done by rebar
>>    Make an index of all erlang apps on github that follow
>>    the erlang packaging structure
>> 
>>    Somebody has to write a program to do this.
>> 
>>    The package index should be at a well know URL
>> 
>>    > rebar get_index
>> 
>> 
>>    This should fetch the  index from the well-know URL and store in
>>    ${HOME}/.erl_packages
>> 
>> 
>>    > rebar search XXX
>> 
>>    would search the fetched index	
>> 
>>    > rebar add XXXX
>> 
>>    would take the index entry add it to the config fill fetch the code
>>    and compile it
>> 
>> Note the following:
>> 
>> 1) The trust model.
>> 
>>    We trust the supplier of a program not the program.
>> 
>>    So for example on github we might trust programs
>>    published by the user joearms (me) or erlang (the OTP release)
>>    but not by some_guy_ive_never_hear_of
>> 
>>    It would be quite easy to add a trust section to rebar.config
>> 
>>    {trust, ["git://joearms", git://erlang", ...]}
>> 
>> 2) There is no "publish" step
>> 
>>    We put our code on github (or somewhere) and hope that it gets indexed
>>    by the indexer.
>> 
>>    We might mail the indexer if it is published in an obscure place.
>> 
>>   Comments?
>> 
>>    Cheers
>> 
>> /Joe
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121019/5b98cf55/attachment.bin>