application meta-information

[Vlad Dumitrescu]

Hi all,

As part of my work with erlide, I need to work with
projects/applications that don't follow the standard OTP structure.
More precisely, the largest user base doesn't :-)

The tooling needs to know about the project structure and currently
the needed meta-information is stored in a highly Eclipse-dependent
way. What I want to do is to change that and have it in a format where
it can be used by other tools too. For this to be useful, we should
have an agreement about what information is needed and where it is to
be stored. Here I will propose a way to do that and look forward
suggestions and criticism.

-------------

Tools that work on source code need to be able to understand the
codebase's structure and dependencies. For example a builder needs to
locate the source code and all the needed include files; a launcher
needs to find all the required beam files. In the standard libraries
there are two files that describe this information: the .app file and
the Emakefile. Other build tools use other formats or infer the
information at runtime. However, all these make an important
assumption: the code must be structured as an OTP application. Also,
there are important bits of data that aren't present in these files,
like the location of the application's dependencies, because at
runtime the applications can be located anyway.

What I'd like to suggest is to extend the .app format to include
information about
- the layout of the source code
- the location of dependencies
- compiler options

This information should be present in a file named "meta" in the root
directory of the application. The contents is an #appmeta term like
below, but with the options expanded as a property list (i.e. just
like in regular .app files):

-type options() :: [{atom(), any()}].
-type meta_path() :: string() |
                     {atom(), string()}.
-record(layout, {
                 src = ["src"] :: [meta_path()],
                 include = ["include"] :: [meta_path()],
                 ebin = "ebin" :: meta_path(),
                 doc = "doc" :: [meta_path()]
                }).
-type meta_module() :: module() |
                       {module(), options()}.
-type meta_app() :: atom() |
                    {atom(), meta_path()} |
                    {atom, #layout{}}.

-record(meta, {
               description = "" :: string(),
               id = "" :: string(),
               vsn = "" :: string(),
               modules = source :: [meta_module()] | 'source',
               maxT  = infinity :: integer() | 'infinity',
               registered = [] :: [atom()],
               included_applications = [] :: [meta_app()],
               applications = [] :: [meta_app()],
               env = [] :: [{any(), any()}],
               mod = undefined :: {module(), any()} | 'undefined',
               start_phases = undefined :: [{atom(), any()}] | 'undefined',
               otp_version = undefined :: string() | 'undefined',
               layout = #layout{},
               compiler_options = [] :: options()
              }).

-record(appmeta, {name, meta=#meta{}}).

The additions and changes are as follows:

* otp_version specifies the minimum required version of the runtime in
a format like "R13B"
* layout describes where source, include, ebin and doc directories are
placed, if the setup is not standard
* compiler_options specifies the global options to use when compiling

* 'source' instead of a module list in modules means "all modules in
the source directories"
* a module can be specified by name with or without a list of extra
compiler options that supersede the global ones

* applications and include_applications can be specified by name, by
name and a path (these assume the app is a standard OTP apps or a
"meta" file can be found), or by name and a precise layout of the
source code. This is so that the source code for libraries and such
(if available) can be located.

* paths can be specified as regular strings, or by a pair {variable,
path} where the variable is to be interpreted (tool-specific) to a
path that is to be prepended to the one specified. For example,
{'PRJ_1', "my/path"} where PRJ1 evaluates to "/home/me/prj_1" gives a
result of "home/me/prj_1/my/path". Variables can be retrieved from the
environment or from the tool that processes the file (Eclipse, for
example). With variables it is possible to use the same settings for
multiple users/environments and just set the variables to the right
values, or to quickly switch to a different version of a library.

* unknown options are ignored, other tools could add specific ones.

* if an .app file exists, values should be merged with these (for
backwards compatibility). Have to specify what happens if there are
mismatches and who (which tool) is responsible to check the
consistency.

* even the data in the 'info' file used by the OTP applications can be
added here

Where/how can this be used:
- as a makefile, since it contains all details needed
- to generate the .app file (so that data isn't duplicated)
- to be able to locate the right dependencies to be able to provide
code completion, show documentation and other goodies
- to automatically locate the binaries for all dependencies when
launching an application

The point is that it's better to have a single standard place to find
all this information than having it spread in different places and
tool-dependent formats. I am planning to use this to configure erlide
projects, which is better than today's use of Java properties files,
but it would feel a lot better if something like this would become
universal.

Of course, everything would be much simpler if we could assume that
all applications are standard OTP applications, but the reality is
that we can't - as a tool provider we have to be able to support even
these settings.

--------------------------------------

This is just a first draft, I'm sure that I missed some details and
others are suboptimal. Please let me know what you think.

best regards,
Vlad