[erlang-questions] Code loaders other than efile and inet
Ryan Volpe
ryan.volpe@REDACTED
Mon Jul 11 08:31:38 CEST 2011
Greetings all;
Sorry for the lengthy post -- TL;DR, see bottom. I asked a rather condensed
version of this on the #erlang IRC channel last week; the consensus was that
it should be asked here, so I put together some further notes. I previously
attempted (last week) to send this message to this list to no avail, so
forgive me if it somehow does wind up as a double post.
*Overview*
I'm not sure if it's something I'm missing, but the docs seem to suggest
that custom code loaders are supported by the standard ERTS (from
erl_prim_loader(3) <http://www.erlang.org/doc/man/erl_prim_loader.html>):
The -loader Loader command line flag can be used to choose the method used
> by the erl_prim_loader. Two Loader methods are supported by the Erlang
> runtime system: efile and inet. If another loader is required, then it has
> to be implemented by the user. The Loader provided by the user must
> fulfill the protocol defined below, and it is started with the
> erl_prim_loader by evaluating open_port({spawn,Loader},[binary]).
…
The following protocol must be followed if a user provided loader port
> program is used. The Loader port program is started with the command
> open_port({spawn,Loader},[binary]). The protocol is as follows:
Function Send Receive
-------------------------------------------------------------
get_file [102 | FileName] [121 | BinaryFile] (on success)
[122] (failure)
stop eof terminate
*Trial Run*
So far, so good, right? Looks like there's a fairly well-defined means of
implementing a code loader port. So let's try it…
*annabel:loaders ryan$* *cat test_loader.c*
> #include <stdlib.h>
> #include <stdio.h>
> #include <unistd.h>
> #include <string.h>
#define BUFSIZE 10 * 1024
> extern int errno;
int main(int argc, char * const argv[])
> {
> char req;
> char readbuf[BUFSIZE];
> char fname[BUFSIZE + 1];
> ssize_t nread;
>
> fprintf(stderr, "**%s:: received %d arguments\n", argv[0], argc-1);
> for (int i=1; i<argc; ++i)
> {
> fprintf(stderr, "**%s:: argument %d: %s\n", argv[0], i-1, argv[i]);
> }
>
> while (1)
> {
> /* use unbuffered I/O [read/write(2) v. fread/fwrite(3)] */
> nread = read(0, (void *)&req, sizeof(char));
> if (nread == 0)
> {
> fprintf(stderr, "**%s:: stdin at eof, terminating...\n",
> argv[0]);
> exit(0);
> } else if (nread < 0) {
> perror("*** error reading port request");
> exit(-errno);
> } else if (req != 'f') {
> fprintf(stderr, "**%s:: unknown mode ('%c') requested\n",
> argv[0], req);
> write(1, (void *)"z", sizeof(char));
> } else {
> nread = read(0, (void *)&readbuf, sizeof(char) * BUFSIZE);
> strncpy(fname, readbuf, nread + 1);
> fprintf(stderr, "**%s:get_file(\"%s\")\n", argv[0], fname);
> write(1, (void *)"z", sizeof(char));
> }
> }
>
> return 0;
> }
> *annabel:loaders ryan$* *cat test_loader.erl*
> -module (test_loader).
> -export ([get_file/1]).
> -export ([start/0, start/1, interact/1]).
-spec get_file(string() | atom()) -> binary() | failure.
> -spec start() -> atom().
> -spec interact(binary()) -> {char(), binary()}.
get_file(Filename) when is_atom(Filename) ->
> get_file(atom_to_list(Filename));
> get_file(Filename) ->
> Req = list_to_binary([$f|Filename]),
> {Flag, BinaryFile} = interact(Req),
> case Flag of
> $z -> failure;
> $y -> BinaryFile
> end.
start() -> sta rt(test_loader).
> start(Loader) ->
> Port = open_port({spawn, Loader}, [binary]),
> register(test_loader, Port).
interact(Req) ->
> case whereis(test_loader) of
> undefined ->
> start(),
> interact(Req);
> Port ->
> Port ! {self(), {command, Req}},
> receive
> {Port, {data, <<Flag, Binary/binary>>}} -> {Flag, Binary}
> end
> end.
> *annabel:loaders ryan$* *CFLAGS='-std=c99 -Wall -pedantic' make
> test_loader*
> cc -std=c99 -Wall -pedantic test_loader.c -o test_loader
> *annabel:loaders ryan$* *PATH=.:$PATH erl*
> Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2]
> [async-threads:0] [hipe] [kernel-poll:false]
> Eshell V5.8.4 (abort with ^G)
> 1> c(test_loader).
> {ok,test_loader}
> 2> test_loader:start().
> true
> 3> **test_loader:: received 0 arguments
3> test_loader:get_file("testing").
> **test_loader:get_file("testing")
> failure
> 4>
> User switch command
> --> q
> **test_loader:: stdin at eof, terminating...
*annabel:loaders ryan$* *PATH=.:$PATH erl -loader test_loader*
{"cannot start
> loader",{function_clause,[{erl_prim_loader,start_it,["test_loader",none,<0.2.0>,[]]}]}}
{"init terminating in
> do_boot",{function_clause,[{erl_prim_loader,start_it,["test_loader",none,<0.2.0>,[]]}]}}
(no error logger present) error: "Error in process <0.3.0> with exit value:
> {function_clause,[{erl_prim_loader,start_it,[\"test_loader\",none,<0.2.0>,[]]}]}\n"
> Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
*Error Location*
Referring to erl_prim_loader ($OTP_SRC/erts/preloaded/erl_prim_loader.erl),
the error reported soon reveals itself in start_it/4 (presumably where the
open_port/2 call referenced in erl_prim_loader(3) would be made):
start_it("ose_inet"=Cmd, Id, Pid, Hosts) ->
> …
> start_it("inet", Id, Pid, Hosts);
> start_it("inet", Id, Pid, Hosts) ->
> …
> loop(State, Pid, []);
> start_it("efile", Id, Pid, _Hosts) ->
> …
> loop(State, Pid, []).
*Issue 1: *No clauses are defined here for custom code loaders, only
"ose_inet", "inet" and "efile".
*Further Issues*
Continuing down the file, further issues crop up. erl_prim_loader:loop/3 relies
on a corpus of helper functions (all defined in the same module) to do its
dirty work; several of these have issues.
*Issue 2:* Several functions only accept efile or inet for #state.loader --
any other value will cause a function_clause error:
- handle_get_file/3
- handle_list_dir/2
- handle_get_cwd/2
- handle_stop/2
- handle_exit/3
- handle_timeout/2
*Issue 3:* Two other functions only accept efile for #state.loader -- same
error as above:
- handle_set_primary_archive/4
- handle_release_archives/1
*Issue 4:* handle_get_files/4 will fail for any loader besides efile that
sets #state.multi_get to true.
*TL;DR Summary*
*
*
I can't see how this shipped for so many versions as "broken" as it seems --
I can't have been the first one to attempt to implement a custom code
loader. Have I just entirely missed the boat on how this is done, or are
these actually bugs? Most importantly, is there any interest in a patch that
fixes these apparent issues?
*
*
Regards,
Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110711/4bc0ee48/attachment.htm>
More information about the erlang-questions
mailing list