[erlang-questions] file operations on zipped files (Re: how to add a module to preLoaded?)

Ulf Wiger <>
Thu Mar 31 20:04:39 CEST 2011


So after finding and trying 'make install', I finally noticed that my new module was in fact loaded, even though the {preLoaded, …} entry in the boot script didn't say so. As a result, I still don't know which step actually made the difference, nor how to make the boot script reflect the change.

Anyway, this now works:

This patch makes most file operations transparent to whether the file is in a normal file system or in a zip archive.

git fetch :uwiger/otp.git uw-prim_file-reads-zip

$ $ERL_TOP/bin/erl -pa run_proper.ez/proper/ebinErlang R14B03 (erts-5.8.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.4  (abort with ^G)
1> compile:file("test/run_proper_test.erl",[debug_info]).
{ok,run_proper_test}
2> beam_lib:chunks("run_proper_test.beam",[abstract_code]).
{ok,{run_proper_test,
        [{abstract_code,
             {raw_abstract_v1,
                 [{attribute,1,file,{"test/run_proper_test.erl",1}},
                  {attribute,1,module,run_proper_test},
                  {attribute,0,export,[]},
                  {attribute,2,compile,export_all},
                  {attribute,1,file,
                      {"run_proper.ez/proper/include/proper.hrl",1}},   <====== HERE 
                  {attribute,1,file,
                      {"run_proper.ez/proper/include/proper_common.hrl",1}}, <=== HERE
                  {attribute,24,file,
                      {"run_proper.ez/proper/include/proper.hrl",24}},  <=== HERE

Note that the compiler fetched proper.hrl et al from the run_proper.ez zip file, which contains PropEr as a single zip file. The idea (see http://github.com/esl/run_eqc) was that it should be possible to deliver an application as a single archive, supporting command-line operations, interactive code loading, and compilation including hrl files, parse_transforms etc., without having to unpack the archive.

All (?) sensible file operations on a read-only filesystem now work on files within a zip archive, as far as I can tell.

The change in prim_file gives no overhead in the normal case, except for an extra pattern-match on the error value, and - if {error, enotdir}, a scan of the file name to see if there is a .ez file in the path. If so, the corresponding action is mapped on the archive. If the file is to be opened, as with epp, a "read_file()" is done, and the contents moved into a ram_file, which allows the same operations as prim_file.

For this last case, I needed prim_ram_file in preloaded…

A minor quirk is that code:lib_dir/1 has been defined to return a path *outside* the zip archive, presumably because you cannot load DLLs from an archive. Epp uses code:lib_dir/1 too, so even with the above changes, epp would fail to find the .hrl files. I solved this by having epp call code:where_is_file(Lib ++ ".app") and then finding the include from there. If that fails, it calls code:lib_dir/1 as before.

I've not written a proper commit message (or actually sent this to erlang-patches), nor have I updated documentation or written test suites. If this approach is rejected on principle, I'd rather not waste the time, so, feedback first, please.

BR,
Ulf W

On 31 Mar 2011, at 18:20, Ulf Wiger wrote:

> 
> I'm trying to add a preloaded module for a patch I'm working on, but this turns out to be exceptionally difficult.
> 
> I found the ./otp_build update_preloaded, which works great for existing preloaded files.
> 
> Now trying to add (in my case) prim_ram_file, to the list of preloaded modules, I've so far found the list hard-coded in the following places:
> 
> - erts/preloaded/src/Makefile
> - $ERL_TOP/Makefile.in
> - lib/sasl/src/systools_make.erl
> 
> I feel that I have updated all the hard-coded representations, and tried various combinations of 
> - ./otp_build update_preloaded
> - make
> - make release
> - ./otp_build setup
> - ./otp_build boot
> - ./otp_build update_primary
> 
> None of them seem to actually change the start scripts - not that I'm even sure that this would make a difference, since the whole preloading is a deep dark mystery.
> 
> What is the magic incantation that I'm missing?
> 
> BR,
> Ulf W
> 
> Ulf Wiger, CTO, Erlang Solutions, Ltd.
> http://erlang-solutions.com
> 
> 
> 
> _______________________________________________
> erlang-questions mailing list
> 
> http://erlang.org/mailman/listinfo/erlang-questions

Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com






More information about the erlang-questions mailing list