[erlang-questions] What is the point of Spawn(Node, Fun) if Node has to have the same module loadable as a client node?

[Matthias Lang]

Hi,

On 02. September 2016, Joe Armstrong wrote:

> I posted some information about why things are as they are:

> http://erlang.org/pipermail/erlang-questions/2016-July/089809.html

Interesting thread, shame I missed it during summer.

I thought a bit about how this could be hacked together and came up
with two ideas. One is to hack the code server. Tony showed how to
do that later in the thread you linked above.

The other is to use the 'inet' loader to do demand loading. I half
expected that to require some hacking, but it actually works straight
off. Here's an example of receiving a fun from a remote node and
running it without having a local copy of the .beam:

  >erl -sname a -loader inet -hosts 192.168.1.4
  Erlang/OTP 18 [erts-7.3] [source] ...
  Eshell V7.3  (abort with ^G)
  (a@REDACTED)1> global:register_name(mml, self()).
  yes
  (a@REDACTED)2> Y = receive X -> X end.
  #Fun<cc.0.109044139>
  (a@REDACTED)3> code:is_loaded(cc).
  false
  (a@REDACTED)4> erlang:fun_info(Y, module).
  {module,cc}
  (a@REDACTED)5> erlang:fun_info(Y, pid).
  {pid,<7389.70.0>}
  (a@REDACTED)6> Y().
  "this is a fun"

The fun I sent was defined, in a .beam, like this:

  d() -> fun() -> "this is a fun" end.

and I sent it from the remote node like this:

  global:whereis_name(mml) ! cc:d().

the remote node was running a boot server, i.e.

  erl_boot_server:start(["192.168.1.4"]).

---

One limitation is that the code server has to contain all the code
that might be run in the network of nodes. You can specify multiple
code servers (the -hosts argument), but they don't get used
concurrently, it's just fail-over. So you can't have some modules
on one node and some on another and have it magically work.

Another limitation is that you can't run a code server on a diskless
node. 'erl_boot_server' uses erl_prim_loader:prim_get_file/2 which
only reads from the filesystem, unlike erl_prim_loader:get_file/2.
I'm not sure why, maybe just "keep it simple" and avoid problems with
cyclic code server graphs.

I briefly wondered why there's no 'get_code/1' BIF which gives you
loaded code even if no .beam file is present. I then remembered that
the beam loader rewrites things, so you'd have to store a copy of
the .beam anyway to provide that.

A comment at the top of 'erl_boot_server.erl' mentions CP and DP, so
I'm guessing the feature is courtesty of AXD-301 (CP = Central
Processor, DP = Device Processor) and also I'd guess that DPs were
diskless.

> The reason is historical - In the first distributed erlang all nodes
> accessed a common file store - this is no longer true when the nodes
> have separated stores.

It looks like AXD-301 moved forwards on this, though in a pragmatic
"change as little as possible" way.

(I'm not advocating sending code around a distributed erlang system, I
can't think of a practical use where "just start off with all code on
all nodes" isn't the sane, robust approach. I just got curious.)

Matt