Longstanding issues: structs & standalone Erlang

[Romain Lenglet]

> > Here is the same misunderstanding again...
> > The proposal was *not* to use ./configure ; make ; make
> > install for deployment / installation, but for *building
> > packages*, which in turn can be deployed and installed.
>
> i am sorry, but i have trouble understanding your terminology.
> what is the difference between *building packages* and
> ''deployed and installed''?

On Debian, building a package means:
1- taking the sources provided by a developer,
2- apply some patches, e.g. to have files installed in the right 
location according to the Debian policy, to write manpages to 
programs that don't have one, to add shell scripts to start 
applications (e.g. Java applications often don't come with 
appropriate shell scripts to start them), etc.
3- define packaging-specific files, e.g. on Debian:
- the control file specify what "binary" packages will be 
generated from that single "source" package, and what are the 
dependencies of the source package (for building it), and of 
every binary package (for installing and using it).
- the packaging changelog (which has a strict format, because it 
is interpreted by tools to get the package version and packager 
name)
- the copyright file (which follows some conventions)
- the rules file, which is in fact a Makefile which rules must 
have standard names (build, install, etc.): usually, these rules 
simply call make for the developer-provided Makefile's rules
- etc.
4- run the package's building Makefile rules: this generates the 
installable .deb package files for the current architecture 
(e.g. i386) and a source package tarball, and digitally signs 
the files,
5- upload the .deb and source package files into the official 
repository (or a private repository) using Debian's tools (dput, 
etc.)
6- if uploaded into the official Debian repository:
- the architecture-specific binary packages are automatically 
built (compiled, etc. cf. step 4) for the 10+ hardware 
architectures supported by Debian, from the source package 
tarball, and any build bug is reported to the packager
- any bug that is declared to be closed in the source package's 
changelog are automatically closed in Debian's Bug Tracking 
System
- etc.

Being a Debian packager also means tracking bug reports 
(sometimes related only to packaging), tracking changes in 
packages we depend on, changes in the Debian policy, etc. And be 
an interface between Debian users and upstream developers (to 
submit patches, etc.).

Then, people can use the Debian commands (apt-get...) to 
automatically download the .deb files from the repositories and 
install, or upgrade, or uninstall them.
Installation typically takes one command, e.g.:
$ apt-get install gtknode

> > To make this possible, a common form for source packages
> > provided by developers must be defined, e.g. by including a
> > configure script and a Makefile in every source tarball.
> > (Fredrik, you are right when writing that the configure
> > script must not necessarily be generated by GNU Autoconf...)
>
> i belive that when you say that ''configure must not
> necessarily be generated by GNU Autoconf'' you still expect
> ''configure'' to behave exactly as if it had been generated by
> gnu autotools. correct?

Yes and no.
In fact the basic "interface" provided by a configure script is 
described in the GNU Coding Standards:
http://www.gnu.org/prep/standards/html_node/Configuration.html#Configuration
Autotoconf-generated configures provide more options.

We could specify our own subset or superset of those options. For 
instance, the --prefix option provided by Autoconf-generated 
configures is very useful, as Fredrik pointed out.
Because normally a source package should copy files 
into /usr/local on Unix, but Debian (and any other Linux 
distribution, I guess) requires packages to install into /usr. 
Then, it is very usual to specify --prefix=/usr to the configure 
scripts.

> > Nothing should prevent a packager to take a source Erlang
> > application, to build it by running the provided configure
> > script and Makefile rules, and to create a user-installable
> > package using any "pure-Erlang" packaging system you want:
> > in this scenario, end users of the package would not have to
> > install anything else than Erlang.
>
> from what you have sofar written i think that you are creating
> a unneccessary distinction between ''packager'' and ''end
> users''. i belive that for some (pure) applications their
> needs are similar enough to treat them as one and the same.

This distinction exists, believe it or not. Perhaps you don't see 
it on the systems that you use, but it exists.

I am a Debian user, and as an end user the only thing that I 
accept to do to install an application is executing:
$ apt-get install application
(idem for FreeBSD users, etc. of course)
If packagers do not exist, then who creates the .deb files that I 
install?

Just take a look at a well-known application: Unison.
Unison is a "pure" OCaml application.
http://www.cis.upenn.edu/~bcpierce/unison/download.html
It is distributed only in source form (as a tarball) by its 
developer.
All packages, for many packaging systems (Fink, FreeBSD, Debian, 
etc.) are made by different packagers, who are specialists of 
their packaging systems. As you can see, there is a 
language-specific packaging system for OCaml applications, 
called GODI. But the Unison GODI package is made by just another 
packager, and Unison is not provided by the Unison developer 
itself as a GODI package but as a source tarball because it 
would make packaging difficult for all the other packagers.
And "everybody is happy": the developer does not have to matter 
for packaging issues, packagers get sources from the developer 
in a packaging-friendly form (tarball with Makefile), and 
end-users have packages readilly installable on their system (be 
it MacOS X, FreeBSD, Debian, etc.).

In the case of Unison, there is no configure script in the source 
tarball: configuration is mixed up with the Makefile, which I 
think is a bad idea. But this is still friendly to packagers.

> i also think you want to force application developers to use a
> gnu development system, to make things simpler for yourself.

Of course, I try to make things simpler for myself. ;-)

> even if other users of the ''application'' would have a harder
> time because of this requierment. i could be wrong again.

I am not particularly attached to GNU tools.

I want to have a common, general, simple interface between Erlang 
application developers and packagers. In the Unix world, this 
interface is ./configure ; make ; make install. This is a 
starting point for discussion.

The good point about ./configure ; make ; make install is that it 
makes no assumption about packaging: it deals with configuration 
and building *only*. And any developer-packager interface should 
limit to that also.

> > Yes, just like there is a point in having a pure Ruby system
> > that can deploy "pure" Ruby packages, and a pure Java system
> > that can deploy "pure" Java packages, and a pure Perl system
> > that can deploy "pure" Perl packages, and a pure Python
> > system that can deploy "pure" Python packages...
> > (Sorry for the ironic tone... ;-))
>
> no need to excuse yourself, i see no irony. i have repeatedly
> come to the understanding that cpan is a reason for the
> success of perl.

If tomorrow Debian stopped packaging Perl modules, and I were 
forced to use CPAN to install Perl modules, I would simply stop 
using Perl application. CPAN is not end-user friendly, because 
it interferes with the OS's packaging system (when there is one, 
of course).

> > I am just wondering how you can manage / upgrade
> > applications on a system with such a proliferation of
> > incompatible packaging systems?
>
> how about if they are not packaging systems? what if they are
> deployment systems?

I do not distinguish between packaging and deployment systems: I 
think that they are synonymous.

> and they knows how to tailor the 
> ''application'' for a particular hardware, os and installation
> directory?
> would it be such a nightmare for a ''professional packager''
> to write the interface from his favored tool to these pure
> tools? it would only need to be done once for each tool.

It depends...
It can really be a nightmare if those tools are not designed as a 
developer-packager interface.
First, it would be a nightmare if those pure tools do anything 
else than configuring and building (e.g. if they try to download 
code at build time, or try to install files anywhere...).

> > Really, I am not against this idea. I just say that a pure
> > Erlang packaging system should not be imposed, and
> > applications should be delivered by developers in source
> > form with a configure script and a Makefile, and not in
> > compiled form as a pure Erlang installable package.
>
> i agree about not imposing, but i do not want to have a gnu
> system imposed.

OK. :-)
./configure --prefix=... ;  make ; make install DESTDIR=... is a 
starting point for specifying a configuration and building 
interface. And this does not have to be GNU-specific.
I am not even against the idea of having the equivalent of the 
configure scripts and Makefiles implemented in Erlang.

> i think that a good erlang deployment system 
> (for pure erlang applicaitons, etc) could handle uncompiled
> erlang.

Sure. Only, Debian packagers and buildit packagers, etc. should 
not have to deal with it. Developers should not provide source 
code in a form that depends on that deployment system (or any 
other packaging or deployment system, for that matters).

> > And even packages containing binary native code could be
> > deployed using a packaging system in pure Erlang, there is
> > no problem with that.
> > One issue would be to be able to have several versions of
> > every such package, one for every architecture / system, and
> > the system should be able to choose and install the right
> > version on a system. But this can be solved.
>
> i do not understand why it is neccessary to have several
> packages if the deploymenmt system can handle hardware, os and
> installation directory.

In the scenario I described just above, there would have been one 
package containing binary code for i386/linux, one for 
i386/win32, one for ppc/darwin, one for alpha/linux, etc.
"if the deploymenmt system can handle hardware, os", it only 
means that it can select the right package to install (a package 
being a unit of installation).

-- 
Romain LENGLET