[erlang-questions] how do you build your releases with/without rebar?
Loïc Hoguin
essen@REDACTED
Sat Apr 12 14:23:22 CEST 2014
When I read this and Fred's posts there's something that strikes me
though. It sounds to me that the only way this can work in a team is if
you misuse git. Git is made for non-linear development: that is, someone
forks your repository, does changes, then sends a patch and you merge
it, with one repository = one user. If you do non-linear development
with the apps/* layout where you have to always make sure the whole code
is synced, this makes it pretty hard to merge things, especially bigger
changes. Basically the guy merging things would be doing nothing else
than make sure it works.
So I'm guessing you guys must be having more than one committer per
repository, which comes with its own set of problems, but sounds more
workable than my initial assumptions. But it also makes it more
difficult to push things, because the number of things you must take
care of increases, and everytime something new gets added before you
could push, you have to test again or risk breaking things.
To take a well known example, look at what OTP does. Basically they pull
patches, put them in a nightly test build and then depending on the
results can decide whether to merge things. Now imagine how faster it
could be if when you send a patch to the ssl app you don't have to worry
about anything else? It could be tested faster, you would get feedback
faster and fix things faster and get it merged faster. Smaller
iterations = faster development speed. Now OTP would still need to do
nightly test of everything, but they would only need to do so when they
bump the version of an included application.
Of course in the case of your smaller "big" repositories where you only
need to support one target OS and Erlang version you don't run into
these issues. I suppose you are in a sweet spot where doing things the
messy way doesn't stabs you in the back too much and you can live with
it. But add proper support for more combinations of OS and Erlang
versions and you'll start suffering. (Or I suppose you could do it wrong
again, use travis-ci and not care if your repository gets broken and
just fix it afterwards, which also means you won't be able to use git
bisect etc.)
tl;dr A rant that probably makes little sense to you.
But you know, whatever works.
On 04/12/2014 10:41 AM, Benoit Chesneau wrote:
> I tend to think like Fred. Between `apps/*` and having 1 repo per app, I
> think it all depends the way your are coding how how independent the
> apps are. Also depends how big you are.
>
> I tend to use the apps/ structure when I need to have different
> supervision strategy / apps but apps are not useful by themselves. Or at
> least at this stage not enough isolated to be used by themselves.
> Another reason for that is that it make sometimes really difficult to
> test a *product* update that goes over different apps, if they are isolated:
>
> - you have to make sure to use the correct branch / apps
> - you need to dl/link each apps
> - you need to communicate to other what you're doing, freezing some
> branches, etc..
>
> All of this requires a mechanic, that - if manageable - takes times.
> (Changing he makefile to point to the right branch, communicate
> overseas, sync the source repo, updating the release to tell we are
> testing such feature...) and need to be done very carefully once you
> have more than one developer working on your product. I'm thinking that
> most of the time when your product needs to use self-dependent apps it's
> more solid to take a cathedral logic: keeping these apps in one place
> and update them. (the apps/* folder similar to the /usr and / folder on
> bsd systems). I am only speaking about core applications - the one you
> invent/write -. Imo you only need to split them as independent
> applications, when you think that their code could be used independently
> (and want to support it as a standalone app) or habe become optional in
> your core application.
>
> SInce your applications may use applications coming from external, you
> still need a way to handle external depencies though. And then we are
> back to my question ;)
>
> I originally tested the cloudi approach, but I wonder how do you support
> bugs and features request? What happen if someone post you a patch on
> the Cloudi repository instead of the standalone apps? Can really cloudi
> works only with its core? I mean is the core like riak_core be usable by
> itself ? (don't see any offense in these questions, I just don't know).
>
> I also tested erlang.mk <http://erlang.mk>, + erl.mk <http://erl.mk> and
> was aware about the deps passed as variable in the makefile, but I
> didn't test it yet how it works with C binding. For example I wonder how
> Garrett does with erlang-czmq to embed it in another application with
> only the makefile. With rebar it's quite easy, but with erlang.mk
> <http://erlang.mk> how does it works? (in the case you have have 1 app
> depending from erlang-czmq and building from it).
>
>
> - benoit
>
>
>
>
> On Thu, Apr 10, 2014 at 8:22 PM, Fred Hebert <mononcqc@REDACTED
> <mailto:mononcqc@REDACTED>> wrote:
>
> That's all fine by me too. As I said, the apps/* layout is a layout I'd
> love to see get better support overall (I am mostly using rebar, which
> does a lot of things halfway for that pattern, and relx, which does it
> right).
>
> I'm in no position to tell tool builders what to do or how to write
> their code -- I'm free to go around and fix it myself. If erlang.mk
> <http://erlang.mk>
> allows to override behavior (you did mention using a top-level makefile
> to make things work), that's usually good enough and will let people fix
> things the way they need it.
>
> Rebar makes it more complex because of how it builds its list of path
> for transient deps (it needs to go down the directories recursively, as
> with a 'compile' command or by specifying -r) -- which often conflicts
> with running commands within a single directory (apps/ but not deps/).
> This is what requires, for CT, calling 'rebar ct -r skip_deps=true',
> which is far from obvious.
>
> But I felt like I should defend the directory structure I feel makes the
> more sense for release-building, and therefore that I would like to see
> love given for from tool builders :)
>
> Regards,
> Fred.
>
> On 04/10, Loïc Hoguin wrote:
> > Another clarification.
> >
> > On 04/10/2014 08:10 PM, Loïc Hoguin wrote:
> > >Just want to make sure one thing is clear.
> > >
> > >On 04/10/2014 07:52 PM, Fred Hebert wrote:
> > >>If you end up having to pay that price purely because the build
> tool you
> > >>settled for was opinionated about whether the apps in the apps/
> or lib/
> > >>directory need to be checked out from other repositories or if
> they can
> > >>be in there already, fix that build tool of yours, because it's
> > >>clearly making decisions it should not be making.
> > >>
> > >>What's more important is that a tool that supports both the
> apps/* and
> > >>the OTP app model can be able to also support *both* of these
> workflows
> > >>(using deps or not to build the release), depending on which is
> > >>appropriate for your project, team, and/or community.
> > >>
> > >>Denying people a way to organize their own workflow because of your
> > >>personal opinion when the end result for the files on disk is
> the exact
> > >>same (someone just disagrees about how they ended up there), to me,
> > >>tells me that maybe your tool is a bit too opinionated and may
> need to
> > >>take a step back.
> > >
> > >With regards to erlang.mk <http://erlang.mk>, there's nothing to
> fix. There's nothing it
> > >does that prevent you from using the apps/* layout. There's
> nothing in
> > >it that makes it all happen for you either, because that's
> simply not
> > >the goal of the tool. But it can still fit very easily in an apps/*
> > >layout, as demonstrated by the gist previously posted. IRCCloud
> is one
> > >such big project that uses both erlang.mk <http://erlang.mk> and
> an apps/* layout, all
> > >without any problems AFAIK.
> > >
> > >erlang.mk <http://erlang.mk> will always focus on one OTP
> application because that's what
> > >it's designed to do well. Making it work on two different scopes
> just
> > >makes everything harder, and then you end up with nonsense like
> "rebar
> > >ct compile -r skip_deps=true" (quoting you) to make it do what
> you want.
> > >I do not want that. On the other hand I want to give greater
> power to
> >
> > "I do not want that" refers to supporting the two scopes in
> erlang.mk <http://erlang.mk>
> > itself. It is of course very easy to have two different scopes
> that do
> > exactly what you want by simply having one Makefile at the
> top-level and one
> > Makefile for each application. There's nothing preventing you to
> achieve
> > that in a clean manner.
> >
> > I will not provide a .mk file for the apps/* layout because I do
> not use it
> > and have no idea what it should do, but I probably would accept a
> community
> > supported "apps/* layout" file in the repository. As long as it's
> in a
> > contrib/ directory or something of course.
> >
> > >the user to override or complement the default behavior, and that
> > >alongside Windows support is most of the work that remains to be
> done
> > >for erlang.mk <http://erlang.mk> 1.0. (Not that you can't change
> its behavior already -it's
> > >a text file after all!- but I want to make it even simpler.)
> > >
> > >tl;dr Do one thing and do it well; do it right by default but allow
> > >overriding its behavior easily.
> >
> > --
> > Loïc Hoguin
> > http://ninenines.eu
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
--
Loïc Hoguin
http://ninenines.eu
More information about the erlang-questions
mailing list