[erlang-questions] Code Archives (Turning rebar into a package manager)

Tim Watson watson.timothy@REDACTED
Wed Oct 31 15:05:01 CET 2012 

Hi

On 31 Oct 2012, at 13:57, Ola Andersson A wrote:

> Yes, I suppose that would allow for a few interesting possibilities. ;-)
> 
> An improvement would be to make it configurable in OTP apps where to find the included drivers. 
> application:set_env could be used for that purpose.
> Of course it's probably not realistic to implement this change in every OTP app with C drivers.
> 
> Any other ideas?

There's nothing to stop applications from doing the same workaround you're already using. If a common library added support for this, then everyone could share the same code to achieve it as well. One thing we discussed on the erlware mailing list when this idea was being knocked around was using .ez as one of the distribution formats. Certainly https://github.com/hyperthunk/rebar_dist_plugin supports this packaging method, although that plugin may not work with the latest version of rebar as I've not had time to bring it up to date. Now that rebar is moving properly over to semver, it'll be easier to manage that sort of thing.

> /OLA. 
> 
>> From: Tim Watson
>> 
>> I think you'll find that constraint is enforced by the 
>> operating system, rather than the erlang runtime. If 
>> operating systems allowed you to dynamically load a library 
>> from a process address space, that would open up a massive 
>> security hole.
>> 
>> On 31 Oct 2012, at 12:59, Ola Andersson A wrote:
>> 
>>> Hi,
>>> 
>>> I've been experimenting a little with escripts including 
>> code archives for test purposes.
>>> It's a really useful feature that seems to work nicely at 
>> least this far.
>>> 
>>> The only stumbling block I have seen is when an application 
>> included in the archive uses C drivers.
>>> When the path (code:priv_dir(App)) provided to e.g. 
>> erl_ddll points into a code archive it will result in an error.
>>> 
>>> I fixed it with a hack that extracted the driver to a temp 
>> directory and patched the app to read the driver from the 
>> temp directory and then remove it.
>>> Not something I would use in production code. 
>>> 
>>> Are there any plans to add the possibility to load a driver 
>> directly from an archive?
>>> 
>>> BR
>>> /OLA.
>>> 
>>>> -----Original Message-----
>>>> From: erlang-questions-bounces@REDACTED
>>>> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Joe 
>>>> Armstrong
>>>> Sent: den 30 oktober 2012 18:34
>>>> To: Andrew Berman
>>>> Cc: Erlang
>>>> Subject: Re: [erlang-questions] Turning rebar into a 
>> package manager
>>>> 
>>>> On Tue, Oct 30, 2012 at 6:31 PM, Joe Armstrong <erlang@REDACTED> 
>>>> wrote:
>>>>> On Tue, Oct 30, 2012 at 6:17 PM, Andrew Berman
>>>> <rexxe98@REDACTED> wrote:
>>>>>> I come from the Java world and I'm sorta blown away that Erlang 
>>>>>> doesn't have something comparable to JAR files, which work
>>>> just fine.
>>>>> 
>>>>> Well it does actually - but this is not stunning well documented.
>>>>> 
>>>>> In the manual page for code there is a section entited
>>>>> 
>>>>> LOADING OF CODE FROM ARCHIVE FILES
>>>> 
>>>> My mail got sent before I'd completed it ...
>>>> 
>>>> programs like rebar use these features
>>>> 
>>>> /Joe
>>>> 
>>>> 
>>>>> 
>>>>> 
>>>>>> Everyone needs to
>>>>>> agree on a standard for which to package the applications. 
>>>> JARs are
>>>>>> just zips of class files with config files, so why not 
>> start there 
>>>>>> and just have a zip file with the BEAMs and priv directory with 
>>>>>> configs.  Then we need erl to be able to read these zip
>>>> files and use
>>>>>> them as libraries without having to unzip them.  I think
>>>> we need to
>>>>>> just start simple and then evolve over time if we need to.
>>>>>> 
>>>>>> --Andrew
>>>>>> 
>>>>>> On Tue, Oct 30, 2012 at 4:08 AM, Tim Watson 
>>>>>> <watson.timothy@REDACTED>
>>>>>> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> On 30 Oct 2012, at 09:41, Ignas Vyšniauskas wrote:
>>>>>>> 
>>>>>>>> On 10/23/2012 09:46 AM, Joe Armstrong wrote:
>>>>>>>>> The above link has an example command
>>>>>>>>> 
>>>>>>>>>>> git fetch-pack --include-tag -v 
>>>>>>>>>>> git@REDACTED:hyperthunk/gitfoo.git
>>>>>>>> refs/tags/lager-0.9.4
>>>>>>>> 
>>>>>>>> Wouldn't --depth=1 make it even better? Do we need all of the 
>>>>>>>> history here?
>>>>>>>> 
>>>>>>> 
>>>>>>> Indeed we do not, though I've not tried that. Someone
>>>> also mentioned
>>>>>>> to me offline (a while ago) that they were concerned
>>>> about using git
>>>>>>> as a backend rather than adopting something that is not only 
>>>>>>> distributed but actually has mirrors in place, e.g., apt/yum or 
>>>>>>> whatever.  Stashing things using git simply seemed like a
>>>> *free* way
>>>>>>> to get started when I was first looking at it.
>>>>>>> 
>>>>>>> So, I'm fairly sure we're not going to figure out a way
>>>> to do this
>>>>>>> that pleases everybody. And *even* if the package 
>> management and 
>>>>>>> distribution problem was neatly solved, there would be
>>>> cases where
>>>>>>> you might actually
>>>>>>> *want* source code dependencies instead. Another reason
>>>> we looked at
>>>>>>> git was because once you've built everything, if you're 
>> resolving 
>>>>>>> dependencies from a central location (like $HOME/repo or
>>>> whatever)
>>>>>>> then you've got to deploy them as a snapshot each time
>>>> they're built
>>>>>>> and so on. This stuff actually gets quite involved - using say 
>>>>>>> maven, which is actually very good at managing this, it 
>> can still 
>>>>>>> make the development process over complicated at times.
>>>>>>> 
>>>>>>> Take an example project: RabbitMQ. You're building
>>>> multiple projects
>>>>>>> that depend on each other, and the bug you're fixing
>>>> involves making
>>>>>>> changes to three different components, each of which
>>>> lives in its own repository.
>>>>>>> How're you going to figure out which projects need to 
>> be built if 
>>>>>>> they're all completely discrete by the time they go into the 
>>>>>>> repository? Of course its easy to solve dependencies with
>>>> make and
>>>>>>> rebar will compile *including* dependencies by default,
>>>> which goes a
>>>>>>> long way to making this situation workable in practise.
>>>>>>> 
>>>>>>> Of course what Joe is really talking about is the ability
>>>> to install
>>>>>>> *other people's work* for which you have absolutely no 
>> (or maybe 
>>>>>>> just very
>>>>>>> little) interest in the source code. I typically don't 
>> care how a 
>>>>>>> particular bit of cowboy or webmachine or whatever work,
>>>> as long as
>>>>>>> it does I just ignore it.
>>>>>>> 
>>>>>>> And this problem cannot be *that hard* to solve, 
>> because you know 
>>>>>>> what - stuff like rubygems/bundler is really damn useful, as it 
>>>>>>> PyPI. I'm sure the node thing is great, though I've 
>> never looked.
>>>>>>> Even OCaml Godi comes in useful and despite the vitriol I
>>>> hear about it, cabal+hackage is invaluable.
>>>>>>> Point is, these things make it easy to get stuff done,
>>>> thought they
>>>>>>> have gaps, flaws and whatnot - they still add a lot of value.
>>>>>>> 
>>>>>>> Oh and BTW if someone does come up with a system to do
>>>> this, then it
>>>>>>> *should* just be packaged as a rebar plugin, as there's
>>>> no need to
>>>>>>> change rebar at all in order to override things like these.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Tim
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> erlang-questions mailing list
>>>>>>> erlang-questions@REDACTED
>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> erlang-questions mailing list
>>>>>> erlang-questions@REDACTED
>>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>>> 
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>> 
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-questions
>> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20121031/8ff3c139/attachment.bin>

More information about the erlang-questions mailing list