[erlang-questions] how do you build your releases with/without rebar?
Michael Truog
mjtruog@REDACTED
Sat Apr 12 11:35:29 CEST 2014
On 04/12/2014 01:41 AM, Benoit Chesneau wrote:
> I tend to think like Fred. Between `apps/*` and having 1 repo per app, I think it all depends the way your are coding how how independent the apps are. Also depends how big you are.
>
> I tend to use the apps/ structure when I need to have different supervision strategy / apps but apps are not useful by themselves. Or at least at this stage not enough isolated to be used by themselves. Another reason for that is that it make sometimes really difficult to test a *product* update that goes over different apps, if they are isolated:
>
> - you have to make sure to use the correct branch / apps
> - you need to dl/link each apps
> - you need to communicate to other what you're doing, freezing some branches, etc..
>
> All of this requires a mechanic, that - if manageable - takes times. (Changing he makefile to point to the right branch, communicate overseas, sync the source repo, updating the release to tell we are testing such feature...) and need to be done very carefully once you have more than one developer working on your product. I'm thinking that most of the time when your product needs to use self-dependent apps it's more solid to take a cathedral logic: keeping these apps in one place and update them. (the apps/* folder similar to the /usr and / folder on bsd systems). I am only speaking about core applications - the one you invent/write -. Imo you only need to split them as independent applications, when you think that their code could be used independently (and want to support it as a standalone app) or habe become optional in your core application.
>
> SInce your applications may use applications coming from external, you still need a way to handle external depencies though. And then we are back to my question ;)
>
> I originally tested the cloudi approach, but I wonder how do you support bugs and features request? What happen if someone post you a patch on the Cloudi repository instead of the standalone apps? Can really cloudi works only with its core? I mean is the core like riak_core be usable by itself ? (don't see any offense in these questions, I just don't know).
With the (main) CloudI repository at https://github.com/CloudI/CloudI all the dependencies are contained within the repository (with a README that describes where the code has come from with details, commit, authors, etc.) for the sake of having a stable platform. Also, since CloudI wants to support private cloud installations, it wants to have only a single installable source which is transparent. The other CloudI repository https://github.com/CloudI/cloudi_core and the associated https://github.com/CloudI/cloudi_service_* repositories are relying on rebar deps for their dependencies. Both approaches work, but the rebar deps method of doing things can be prone to problems if tags are not being used or if tags are changed for a dependency, or even if a dependency history is rewritten.
Having all the dependencies in the main repo requires tracking the dependencies for changes manually, which is more time consuming but helps protect the stability of the main project, despite the service isolation (present in CloudI) that almost all the dependencies utilize. So, this choice really depends on your stability requirements and your testing for verification of new dependencies, it all takes time, and the goal is to avoid failures in the source code.
If you do go the rebar deps route, it generally is best if you don't checkin a rebar binary that lacks transparency (into your source code repo, this has been a problem in the past) and if you use tags for your external dependencies to lock down the potential problems partially.
>
> I also tested erlang.mk <http://erlang.mk>, + erl.mk <http://erl.mk> and was aware about the deps passed as variable in the makefile, but I didn't test it yet how it works with C binding. For example I wonder how Garrett does with erlang-czmq to embed it in another application with only the makefile. With rebar it's quite easy, but with erlang.mk <http://erlang.mk> how does it works? (in the case you have have 1 app depending from erlang-czmq and building from it).
>
>
> - benoit
>
>
>
>
> On Thu, Apr 10, 2014 at 8:22 PM, Fred Hebert <mononcqc@REDACTED <mailto:mononcqc@REDACTED>> wrote:
>
> That's all fine by me too. As I said, the apps/* layout is a layout I'd
> love to see get better support overall (I am mostly using rebar, which
> does a lot of things halfway for that pattern, and relx, which does it
> right).
>
> I'm in no position to tell tool builders what to do or how to write
> their code -- I'm free to go around and fix it myself. If erlang.mk <http://erlang.mk>
> allows to override behavior (you did mention using a top-level makefile
> to make things work), that's usually good enough and will let people fix
> things the way they need it.
>
> Rebar makes it more complex because of how it builds its list of path
> for transient deps (it needs to go down the directories recursively, as
> with a 'compile' command or by specifying -r) -- which often conflicts
> with running commands within a single directory (apps/ but not deps/).
> This is what requires, for CT, calling 'rebar ct -r skip_deps=true',
> which is far from obvious.
>
> But I felt like I should defend the directory structure I feel makes the
> more sense for release-building, and therefore that I would like to see
> love given for from tool builders :)
>
> Regards,
> Fred.
>
> On 04/10, Loïc Hoguin wrote:
> > Another clarification.
> >
> > On 04/10/2014 08:10 PM, Loïc Hoguin wrote:
> > >Just want to make sure one thing is clear.
> > >
> > >On 04/10/2014 07:52 PM, Fred Hebert wrote:
> > >>If you end up having to pay that price purely because the build tool you
> > >>settled for was opinionated about whether the apps in the apps/ or lib/
> > >>directory need to be checked out from other repositories or if they can
> > >>be in there already, fix that build tool of yours, because it's
> > >>clearly making decisions it should not be making.
> > >>
> > >>What's more important is that a tool that supports both the apps/* and
> > >>the OTP app model can be able to also support *both* of these workflows
> > >>(using deps or not to build the release), depending on which is
> > >>appropriate for your project, team, and/or community.
> > >>
> > >>Denying people a way to organize their own workflow because of your
> > >>personal opinion when the end result for the files on disk is the exact
> > >>same (someone just disagrees about how they ended up there), to me,
> > >>tells me that maybe your tool is a bit too opinionated and may need to
> > >>take a step back.
> > >
> > >With regards to erlang.mk <http://erlang.mk>, there's nothing to fix. There's nothing it
> > >does that prevent you from using the apps/* layout. There's nothing in
> > >it that makes it all happen for you either, because that's simply not
> > >the goal of the tool. But it can still fit very easily in an apps/*
> > >layout, as demonstrated by the gist previously posted. IRCCloud is one
> > >such big project that uses both erlang.mk <http://erlang.mk> and an apps/* layout, all
> > >without any problems AFAIK.
> > >
> > >erlang.mk <http://erlang.mk> will always focus on one OTP application because that's what
> > >it's designed to do well. Making it work on two different scopes just
> > >makes everything harder, and then you end up with nonsense like "rebar
> > >ct compile -r skip_deps=true" (quoting you) to make it do what you want.
> > >I do not want that. On the other hand I want to give greater power to
> >
> > "I do not want that" refers to supporting the two scopes in erlang.mk <http://erlang.mk>
> > itself. It is of course very easy to have two different scopes that do
> > exactly what you want by simply having one Makefile at the top-level and one
> > Makefile for each application. There's nothing preventing you to achieve
> > that in a clean manner.
> >
> > I will not provide a .mk file for the apps/* layout because I do not use it
> > and have no idea what it should do, but I probably would accept a community
> > supported "apps/* layout" file in the repository. As long as it's in a
> > contrib/ directory or something of course.
> >
> > >the user to override or complement the default behavior, and that
> > >alongside Windows support is most of the work that remains to be done
> > >for erlang.mk <http://erlang.mk> 1.0. (Not that you can't change its behavior already -it's
> > >a text file after all!- but I want to make it even simpler.)
> > >
> > >tl;dr Do one thing and do it well; do it right by default but allow
> > >overriding its behavior easily.
> >
> > --
> > Loïc Hoguin
> > http://ninenines.eu
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140412/f7625267/attachment.htm>
More information about the erlang-questions
mailing list