[erlang-questions] Building, Packaging and Installing

Wed May 9 10:20:17 CEST 2012

Of course I know that Maven will never get a wide acceptance in the Erlang community and furthermore I think Erlang deserves its own tool chain (build, dependency/package management tooling) written in Erlang. With my previous post I just wanted to point out what the current tooling can offer *today* (although I could have left out the build-tool-only aspects) and of course future tooling will have to compete against that to convince the community.

We didn't rewrite the maven-erlang-plugin because we don't like Rebar. Rebar is a great build tool that supports everything you can compile in the Erlang world. We just found (as the previous discussion also revealed) that its dependency mechanism based on links to SCM repos and pulling the sources wasn't sufficient for us.

I think that build/dependency tooling can't really be kept separate from package management. The build and dependency tooling must make use of packages installed in local or remote repositories and also has to make available built artefacts to these repositories. So there must be some kind of support at least in the build tool. I think a symbiosis or combination of Rebar and Sinan (if Sinan is the tool that will evolve in the direction Tim and Eric worked out) could really become the standard Erlang tool chain in this area and I would be the last one refusing to use it.

Now some words to Tim's question regarding native code support:
The maven-erlang-plugin doesn't deal with native code itself. As Tim pointed out there already are Maven plugins handling native code (afaik pretty complicated ). The user is free to use them. Either in the same project putting the native output manually in a directory of the application or in a separate project specifying a dependency to it in the Erlang project. In the latter case the maven-erlang-plugin would put all non-Erlang dependencies into the target application's priv directory automatically. Alternatively the OS mechanism could be used to install native dependencies on a (target) system, an Erlang application would just presume that the needed artefacts are available. Another solution would be a well-defined, unified production and build environment. In this case native code could be built everywhere and native artefacts could be put under version control.

Regards
Tobias

________________________________________
Von: Eric Merritt [ericbmerritt@REDACTED]
Gesendet: Dienstag, 8. Mai 2012 18:57
An: Tim Watson
Cc: Tobias Schlager; erlang-questions@REDACTED
Betreff: Re: [erlang-questions] Building, Packaging and Installing

Sorry guys , I am late to the party.

On Tue, May 8, 2012 at 4:49 AM, Tim Watson <watson.timothy@REDACTED> wrote:
> On 8 May 2012 08:43, Tobias Schlager <Tobias.Schlager@REDACTED> wrote:
>> I know that this is not a very popular opinion but to me it sounds like what you're looking for is pretty close to what maven delivers:
>
> We used maven and the maven-erlang-plugin for a long time in my
> previous job - it worked very well.

The problem is always adding the (rather large) non-erlang dependency.
Even if the barrier is small in reality, the mental/social barrier is
huge.

>
>> * binary-based packaging
>
> I think for a package or dependency manager, this is a given -
> otherwise you're reinventing too many other things at once. Unlike
> maven, I think what we want here is either a package manager or
> (simpler) a dependency manager - not another build tool.
>
>> * well-defined dependency management (handling version conflicts)
>> * (open source) repository managers (e.g. sonatype nexus)
>
> Eric Merrit and I have tabled a design that uses git, so you can host
> your personal/organisational repository on github and consumers simply
> add your repository to their local index.

The goal is to make it trivial for anyone to host their own repo, and
let tools that are good at that handle it. We may reinvent some of the
wheel but we can make that is minimal as possible.

>
>> * group id concept (could be used to avoid clashes when forking github projects)
>
> Yes we very much need this to support multiple forks of the same
> application name.

an unfortunate reality in the modern open source world.

>> * support for continuous integration (e.g. jenkins)
>
> To my mind, this is a corollary theme and not central to this
> discussion, but yes, clearly people want this.

if you get building correct, you should have CI correct as well.

>
>> * a manifest file (pom.xml)
>
> Sure, and you and I both know that almost nobody on this list is going
> to settle for xml. There is a maven extension that supports defining
> the pom using scala/grooxy instead, but again, I doubt it'll get much
> love here.
>
>> * built-in support for documentation and reporting
>
> No - this is the build tool's responsibility and AFAICT that is *not*
> what we're talking about here.

agreed.

>
>> * support for code/package signing
>>
>
> Yes very much so, especially to get around a problem that maven
> doesn't actually solve particularly well. When Project-X is available
> in source only form and not built of packaged in a way that the tool
> understands, what do we do???
>
> In the maven world, you build it locally and do `mvn deploy:file ...`
> to put it into your local repository and perhaps push it into a
> corporate/group nexus instance. But there is *no indication of the
> original of this file* when you do that, so trust is difficult to
> establish.
>
> What I'd like to see is support for a kind of 'third party signing'
> where you build + package the missing app yourself, sign it with your
> own private key and publish it into your own repository, which gets
> synchronised back onto github. Eventually someone else *might* decide
> they want to use it, and if they cannot persuade the original author
> to publish his own binary artefact repository (for whatever reason)
> then they may choose to either package and sign it themselves, OR to
> utilize the version that you have packaged and signed, because the
> know and trust you.
>
> I would, for example, consider using a published artefact that has
> been signed by basho or esl, because I trust them not to publish
> garbage or malicious code, given that doing so would implicitly damage
> their corporate reputation.
>
>> Of course there are certain drawbacks like:
>> * not written in erlang (its written in java)
>
> I suspect this is why it'll not see much traction in the community at large.
>
>> * requires a certain amount of (static) xml configuration
>
> As I mentioned earlier, there are alternatives.
>
>> * poor support for building native code
>
> Actually the C/C++ build plugins for maven are pretty good, but I'm
> not sure how capable the maven-erlang-plugin is at doing this. Perhaps
> the author could chime in and let us know?
>
>> * slower than other tools
>>
>
> It's about the same speed as most of the make based projects I've
> seen. Rebar is MUCH faster though!
>
>> Currently, everyone is hot to re-implement all this in erlang and that's perfectly ok, I just wanted to throw in another point of view.
>>
>
> Another thing I'd like to point at at this juncture is that there is a
> difference between a package manager and a dependency manager. A very
> important point here is that a package manager will allow you to
> install AND uninstall software, whereas a dependency manager will
> fetch artefacts and solve dependency graphs in order for you to
> utilise them, but nothing more.
>
> The other thing I want to point out is that with binary artefacts, you
> need to be able to install them once and use them across multiple
> (compatible) erts versions, which is something Eric and I have
> discussed at some length on the erlware-questions mailing list.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions