[erlang-bugs] bug in code:lib_dir

[Fred Hebert]

It sounds like you could solve the problem quite easily by adding an app 
file to your directory, transforming it into a library application, 
assuming the following project structure:

lib_dir/ebin/my_app.app
lib_dir/include/my_app.hrl

where my_app.app looks as follows:

{application, my_app,
  [{description, "useful header files for Joe"},
   {vsn, "1.0.0"},
   {modules, []}]}.

This should give you an entirely empty library application -- with no 
beam code, but a .app file is present. This makes it visible to 
ERL_LIBS, usable as a dependency with rebar, visible and manageable in 
releases, etc.


On 12-09-18 2:54 PM, Joe Armstrong wrote:
> On Tue, Sep 18, 2012 at 1:57 PM, Serge Aleynikov <serge@REDACTED> wrote:
>> Joe,
>>
>> When I wrote the code supporting ERL_LIBS that got included in the code
>> server (*), it was explicitly designed to iteratively check for paths
>> only containing "ebin" child directories (see the
>> code_server:get_user_lib_dirs_1/0) before adding them to the loader
>> path.  Therefore I wouldn't consider the behavior you mentioned to be a
>> bug but rather a feature.
>>
>> Initially I wrote it in a way that checked for directories containing
>> "*.app" files, and included those.  However, I found cases when certain
>> libraries didn't come with "*.app" files, but merely contained
>> "ebin/*.beam" bytecode files, so the ebin directory checking was more
>> reliable and conventional.
>>
>> In your case, since 'b' application doesn't seem to have neither b.app
>> nor b/ebin.  Should it be considered a valid code path (there's nothing
>> to load for the code server)?  If so, what is the right selection
>> criteria that would avoid bloating the loader's path?
> Well the problem is that code:lib_dir(a) is called when
> the compiler tries to compile a file containing an include_lib
> directive like
>
>      -include_lib("a/include/foo.hrl").
>
> If there really is a file called "a/include/foo.hrl" then it seems
> rather harsh to fail the compilation.
>
> rebar in particular only creates an ebin direction as a result of a
> compile - I googled a bit and found many cases where a
> missing include file message was worrying people - and I think this
> was the reason. In the project I was looking at
> this bug occurred and was fixed with a work-around.
>
> I'm not sure about "code loader path bloat" if you trust
> the LIB_DIRS env variable is correctly set. I'm not sure if
> you recursively scan all directories below  the LIB_DIRS
> but I guess just one level would be ok.
>
> The problem also happens (I think) if git doesn't remake an
> empty ebin directory.
>
> The problem is in find_lib_dir (line 637) of epp.erl
>
> Actually the compile error message could be more helpful
>
>      can't find include lib "aaa/include/d.yrl"
>
> It's true - but confused me for a hour - why could't it find
> this file - ERL_LIBS had aaa in it's path - if the message had been
>
>        can't find include lib "aaa/include/d.yrl"
>        (possible reason aaa/ebin is missing)
>
> I would have guessed what was going wrong.
>
> The root of the problem is that you can't identify app directories -
> sooner or later this must be fixed - I think
> there should be a proper manifest.json (or something) in
> top level of the app - this and only this would identify this as an app.
>
> Until then we need various work-arounds. Probably easiest
> to change epp so is calls something other than code:lib_dir
> when resolving the filename
>
> Cheers
>
> /Joe
>
>
>
>
>
>> Serge
>>
>> (*) http://www-rohan.sdsu.edu/doc/R12B/lib/kernel-2.12.3/doc/html/notes.html
>>
>> On 9/18/2012 4:14 AM, Joe Armstrong wrote:
>>>    > code:lib_dir(b)
>>>     {error,bad_name}
>>>
>>> Fails if the b sub-directory does not contain an ebin directory.
>>> This means things like include_lib("b/include/foo.hrl") will fail
>>> even though the file b/include/foo.hrl exists.
>>>
>>>
>>> Suppose I have a directory structure like this:
>>>
>>>       /home/joe/deps/a/src
>>>                       /include
>>>                       /ebin
>>>                     /b/src
>>>                       /include/foo.hrl
>>>
>>>
>>> a and b are Erlang applications - note that b has no ebin directory
>>>
>>> Then I say
>>>
>>>      > export ERL_LIBS=/home/joe/deps
>>>      > erl
>>>
>>> Then
>>>
>>>     > code:lib_dir(a).
>>>     "/home/joe/deps/a/"
>>>
>>> But
>>>
>>>     > code:lib_dir(b)
>>>     {error,bad_name}
>>>
>>> Now suppose a.erl contains the line
>>>
>>>    -include_lib("b/include/foo.hrl")
>>>
>>>    I'll now get an error. The code server calls lib_dib(b) to resolve
>>> the file name
>>> reference. This fails and the system concludes (incorrectly) that
>>> b/include/foo.hrl does not
>>> exist.
>>>
>>>    If I add an empty ebin directory to b then the error goes away.
>>>
>>>    If you create src, ebin, include directories by hand you will never
>>> see this error.
>>>
>>>    But if you use rebar then the ebin directory is not created when you do
>>>    > rebar create-app appid=<name>
>>>
>>>    The ebin directory is created when you first run rebar compile.
>>>
>>>    So recursive includes (a includes b/include and b includes
>>> a/include) can possibly
>>>    fail if no ebin directory is created.
>>>
>>> Cheers
>>>
>>> /Joe
>>> _______________________________________________
>>> erlang-bugs mailing list
>>> erlang-bugs@REDACTED
>>> http://erlang.org/mailman/listinfo/erlang-bugs
>>>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs