[erlang-questions] Re: file operations on zipped files (Re: how to add a module to preLoaded?)

Phil Stanhope stanhope@REDACTED
Thu Mar 31 23:40:33 CEST 2011


Very cool. I can see how this will make deployments very easy.

On Mar 31, 2011 2:05 PM, "Ulf Wiger" <ulf.wiger@REDACTED> wrote:

So after finding and trying 'make install', I finally noticed that my new
module was in fact loaded, even though the {preLoaded, …} entry in the boot
script didn't say so. As a result, I still don't know which step actually
made the difference, nor how to make the boot script reflect the change.

Anyway, this now works:

This patch makes most file operations transparent to whether the file is in
a normal file system or in a zip archive.

git fetch git@REDACTED:uwiger/otp.git uw-prim_file-reads-zip

$ $ERL_TOP/bin/erl -pa run_proper.ez/proper/ebinErlang R14B03 (erts-5.8.4)
[source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.4  (abort with ^G)
1> compile:file("test/run_proper_test.erl",[debug_info]).
{ok,run_proper_test}
2> beam_lib:chunks("run_proper_test.beam",[abstract_code]).
{ok,{run_proper_test,
       [{abstract_code,
            {raw_abstract_v1,
                [{attribute,1,file,{"test/run_proper_test.erl",1}},
                 {attribute,1,module,run_proper_test},
                 {attribute,0,export,[]},
                 {attribute,2,compile,export_all},
                 {attribute,1,file,
                     {"run_proper.ez/proper/include/proper.hrl",1}},
<====== HERE
                 {attribute,1,file,
                     {"run_proper.ez/proper/include/proper_common.hrl",1}},
<=== HERE
                 {attribute,24,file,
                     {"run_proper.ez/proper/include/proper.hrl",24}},  <===
HERE

Note that the compiler fetched proper.hrl et al from the run_proper.ez zip
file, which contains PropEr as a single zip file. The idea (see
http://github.com/esl/run_eqc) was that it should be possible to deliver an
application as a single archive, supporting command-line operations,
interactive code loading, and compilation including hrl files,
parse_transforms etc., without having to unpack the archive.

All (?) sensible file operations on a read-only filesystem now work on files
within a zip archive, as far as I can tell.

The change in prim_file gives no overhead in the normal case, except for an
extra pattern-match on the error value, and - if {error, enotdir}, a scan of
the file name to see if there is a .ez file in the path. If so, the
corresponding action is mapped on the archive. If the file is to be opened,
as with epp, a "read_file()" is done, and the contents moved into a
ram_file, which allows the same operations as prim_file.

For this last case, I needed prim_ram_file in preloaded…

A minor quirk is that code:lib_dir/1 has been defined to return a path
*outside* the zip archive, presumably because you cannot load DLLs from an
archive. Epp uses code:lib_dir/1 too, so even with the above changes, epp
would fail to find the .hrl files. I solved this by having epp call
code:where_is_file(Lib ++ ".app") and then finding the include from there.
If that fails, it calls code:lib_dir/1 as before.

I've not written a proper commit message (or actually sent this to
erlang-patches), nor have I updated documentation or written test suites. If
this approach is rejected on principle, I'd rather not waste the time, so,
feedback first, please.

BR,
Ulf W

On 31 Mar 2011, at 18:20, Ulf Wiger wrote:

>
> I'm trying to add a preloaded module for a patch I'm working on, but this
turns out to be exceptionally difficult.
>
> I found the ./otp_build update_preloaded, which works great for existing
preloaded files.
>
> Now trying to add (in my case) prim_ram_file, to the list of preloaded
modules, I've so far found the list hard-coded in the following places:
>
> - erts/preloaded/src/Makefile
> - $ERL_TOP/Makefile.in
> - lib/sasl/src/systools_make.erl
>
> I feel that I have updated all the hard-coded representations, and tried
various combinations of
> - ./otp_build update_preloaded
> - make
> - make release
> - ./otp_build setup
> - ./otp_build boot
> - ./otp_build update_primary
>
> None of them seem to actually change the start scripts - not that I'm even
sure that this would make a difference, since the whole preloading is a deep
dark mystery.
>
> What is the magic incantation that I'm missing?
>
> BR,
> Ulf W
>
> Ulf Wiger, CTO, Erlang Solutions, Ltd.
> http://erlang-solutions.com
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com



_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://erlang.org/mailman/listinfo/erlang-questions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110331/b7bb3a67/attachment.htm>


More information about the erlang-questions mailing list