<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 02/12/2016 12:40 AM, José Valim
wrote:<br>
</div>
<blockquote
cite="mid:CAGnRm4J_GfNTUjrDNrvyDdYAegVS0+PTC5TRULW5PYukKe9VWA@mail.gmail.com"
type="cite">
<div dir="ltr">Thanks Michael and Benoit. There are some
interesting ideas in gen_rpc.
<div><br>
</div>
<div>In particular, they have one gen_server per node. This is
quite smart. Instead of naming the process "rex", they name
the process node(). This way, if I have nodes named A, B and
C, each node will have processes A, B and C in them. RPC
between nodes goes directly to those processes. This could
potentially solve the bottlenecks mentioned above without
requiring developers to start their own RPC.</div>
</div>
</blockquote>
<br>
I think using node names to name the process, is a problem due to it
being dynamic based on the erl command line arguments. If the name
is dynamic, it can't be provided to the registered section of the
.app file and any potential conflicts wouldn't be easy to
understand. The argument for it, is that there is only one process
like this, who cares? However, with a complex system, it is good to
nail down dependencies to what can be static and easily understood
in isolation.<br>
<br>
<blockquote
cite="mid:CAGnRm4J_GfNTUjrDNrvyDdYAegVS0+PTC5TRULW5PYukKe9VWA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Btw, this leads to how we could introduce consistent
hashing if desired. On the first request to a node, it won't
have an entry in the local ETS table, we can request the node
and ask how many local rpc processes it has and their PIDs.
These are relatively simple solutions which could also be
backwards compatible.</div>
<div><br>
</div>
<div>At this point, we have a couple different solutions, each
with different complexity. I personally prefer the consistent
hashing one. I am definitely willing to work on any of the
solutions above and I could draft a more detailed proposal. I
just need some guidance from the OTP team about if and which
solution they are more inclined to accept.</div>
</div>
</blockquote>
<br>
If consistent hashing uses the via syntax for process naming, it can
be decoupled from the rpc source code itself, so then it should be
more flexible for reuse. The via syntax can be used to allocate
more than 1 pid per name, and there is an example of doing so with
cpg at
<a class="moz-txt-link-freetext" href="https://github.com/okeuday/cpg/blob/master/test/cpg_test.erl#L82-L105">https://github.com/okeuday/cpg/blob/master/test/cpg_test.erl#L82-L105</a>
.<br>
<br>
<blockquote
cite="mid:CAGnRm4J_GfNTUjrDNrvyDdYAegVS0+PTC5TRULW5PYukKe9VWA@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra"><br clear="all">
<div>
<div>
<div dir="ltr">
<div>
<div><br>
</div>
<div><span style="font-size:13px">
<div><span
style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse"><b>José
Valim</b></span></div>
<div><span
style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse">
<div><span
style="font-family:verdana,sans-serif;font-size:x-small"><a
moz-do-not-send="true"
href="http://www.plataformatec.com.br/"
style="color:rgb(42,93,176)"
target="_blank">www.plataformatec.com.br</a></span></div>
<div><span
style="font-family:verdana,sans-serif;font-size:x-small">Skype:
jv.ptec</span></div>
<div><span
style="font-family:verdana,sans-serif;font-size:x-small">Founder
and Director of R&D</span></div>
</span></div>
</span></div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Fri, Feb 12, 2016 at 7:26 AM,
Benoit Chesneau <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:bchesneau@gmail.com"
target="_blank">bchesneau@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="white-space:pre-wrap">Hrm wouldn'mtwould'nt
the method in gen_rpc <br>
<a moz-do-not-send="true"
href="https://github.com/priestjim/gen_rpc"
target="_blank">https://github.com/priestjim/gen_rpc</a><br>
<br>
partially fit the bill?<span><font color="#888888"><br>
<br>
- benoit</font></span></div>
<div>
<div><br>
<div class="gmail_quote">
<div dir="ltr">On Fri, 12 Feb 2016 at 02:15, José
Valim <<a moz-do-not-send="true"
href="mailto:jose.valim@plataformatec.com.br"
target="_blank">jose.valim@plataformatec.com.br</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Thanks everyone for the feedback so
far.
<div><br>
</div>
<div>Douglas, I thought about such solution. The
problem is that the caller does not know how
many rex instances there are per node.
Therefore you would need to first message a
single process that does the consistent
hashing on the node, which is an additional
step (extra copying) and may still be a
bottleneck. How would you solve this
particular problem?</div>
<div><br>
</div>
<div>Chandru, thanks for validating the concerns
raised. I agree your proposal has many
improvements. IIRC the release project from
where the paper came from explored similar
ideas as well. I am afraid however such
solution can't be a replacement. A lot of the
usage of rex in OTP and other applications is
to request information from a particular node
and therefore the load balancing and node
grouping wouldn't be desired.</div>
<div><br>
</div>
<div>Sean, the idea is to leverage all of the
rpc functionality without reimplementing it.
Like the group leader handling, the
call/async_call API and so forth. So while we
could tell folks to use a gen_server (or to
use spawn), we would be neglecting what rpc
provides. In other words, I would use it for
the same reason I use rpc, I'd just have it in
my own app's tree.</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div>
<div dir="ltr">
<div>
<div><br>
</div>
<div><br>
</div>
<div><span style="font-size:13px">
<div><span
style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse"><b>José
Valim</b></span></div>
<div><span
style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse">
<div><span
style="font-family:verdana,sans-serif;font-size:x-small"><a
moz-do-not-send="true"
href="http://www.plataformatec.com.br/"
style="color:rgb(42,93,176)" target="_blank">www.plataformatec.com.br</a></span></div>
<div><span
style="font-family:verdana,sans-serif;font-size:x-small">Skype:
jv.ptec</span></div>
<div><span
style="font-family:verdana,sans-serif;font-size:x-small">Founder
and Director of R&D</span></div>
</span></div>
</span></div>
</div>
</div>
</div>
</div>
<br>
</div>
<div class="gmail_extra">
<div class="gmail_quote">On Thu, Feb 11, 2016 at
10:58 PM, Sean Cribbs <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:seancribbs@gmail.com"
target="_blank">seancribbs@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div dir="ltr">
<div>José,</div>
<div><br>
</div>
It's interesting to me that your example
of making registered RPC receivers isn't
that much different from having registered
processes with specific messages
(call/cast/etc) they handle. What do you
see as the use-case of allowing generic
RPC in that scenario?
<div><br>
</div>
<div>Cheers,</div>
<div><br>
</div>
<div>Sean</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">
<div>
<div>On Thu, Feb 11, 2016 at 3:04 PM,
José Valim <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:jose.valim@plataformatec.com.br"
target="_blank">jose.valim@plataformatec.com.br</a>></span>
wrote:<br>
</div>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>
<div>
<div dir="ltr">Hello everyone,
<div><br>
</div>
<div>I was reading the
publication "<a
moz-do-not-send="true"
href="http://www.dcs.gla.ac.uk/%7Eamirg/publications/DE-Bench.pdf"
target="_blank">Investigating
the Scalability Limits of
Distributed Erlang</a>" and
one of the conclusions is:</div>
<div><br>
</div>
<div><i>> We observed that
distributed Erlang scales
linearly up to 150 nodes
when no global command is
made. Our results reveal
that the latency of rpc
calls rises as cluster size
grows. This shows that spawn
scales much better than rpc
and using spawn instead of
rpc in the sake of
scalability is advised. </i><br
clear="all">
<div>
<div>
<div dir="ltr">
<div>
<div><br>
</div>
<div>The reason why is
highlighted in a
previous section:</div>
<div><br>
</div>
<div><i>> To find
out why rpc’s
latency increases
as the cluster
size grows, we
need to know more
about rpc. (...)
There is a generic
server process
(gen server) on
each Erlang node
which is named
rex. This process
is responsible for
receiving and
handling all rpc
requests that come
to an Erlang node.
After handling the
request, generated
results will be
returned to the
source node. In
addition to user
applications, rpc
is also used by
many built-in OTP
modules, and so it
can be overloaded
as a shared
service.</i><br>
</div>
<div><br>
</div>
<div>In other words,
the more
applications we have
relying on rpc, the
more likely rpc will
become a bottleneck
and increase
latency. I believe
we have three
options here:</div>
<div><br>
</div>
<div>1. Promote spawn
over rpc, as the
paper conclusion did
(i.e. mention spawn
in the rpc docs and
so on)</div>
<div>2. Leave things
as is</div>
<div>3. Allow "more
scalable" usage of
rpc by supporting
application specific
rpc instances</div>
<div><br>
</div>
<div>In particular, my
proposal for 3 is to
allow developers to
spawn their own rpc
processes. In other
words, we can
expose:</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
<blockquote style="margin:0 0 0
40px;border:none;padding:0px">
<div>
<div>
<div>
<div dir="ltr">
<div>
<div>rpc:start_link(my_app_rpc)
%% start your own
rpc</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<blockquote style="margin:0 0 0
40px;border:none;padding:0px">
<div>
<div>
<div>
<div dir="ltr">
<div>rpc:call({my_app_rpc,
nodename}, foo, bar,
[1, 2, 3]) %% invoke
your own rpc at the
given node</div>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<div>
<div>
<div dir="ltr">
<div>
<div><br>
</div>
<div>This is a very
simple solution that
moves the bottleneck
away from rpc's rex
process since
developers can place
their own rpc
processes in their
application's tree.
The code changes
required to support
this feature are
also minimal and are
almost all at the
API level, i.e.
support a tuple were
today a node is
expected or allow
the name as
argument, mimicking
the same API
provided by
gen_server that rpc
relies on. We won't
change
implementation
details. Finally, I
believe it will
provide a more
predictable usage of
rpc.</div>
<div><br>
</div>
<div>Feedback is
appreciated!</div>
<span><font
color="#888888">
<div><br>
</div>
<div><span
style="font-size:13px">
<div><span
style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse"><b>José
Valim</b></span></div>
<div><span
style="font-family:arial,sans-serif;font-size:13px;border-collapse:collapse">
<div><span
style="font-family:verdana,sans-serif;font-size:x-small"><a
moz-do-not-send="true" href="http://www.plataformatec.com.br/"
style="color:rgb(42,93,176)"
target="_blank">www.plataformatec.com.br</a></span></div>
<div><span
style="font-family:verdana,sans-serif;font-size:x-small">Skype:
jv.ptec</span></div>
<div><span
style="font-family:verdana,sans-serif;font-size:x-small">Founder
and Director
of R&D</span></div>
</span></div>
</span></div>
</font></span></div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<span>_______________________________________________<br>
erlang-questions mailing list<br>
<a moz-do-not-send="true"
href="mailto:erlang-questions@erlang.org"
target="_blank">erlang-questions@erlang.org</a><br>
<a moz-do-not-send="true"
href="http://erlang.org/mailman/listinfo/erlang-questions"
rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
<br>
</span></blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
_______________________________________________<br>
erlang-questions mailing list<br>
<a moz-do-not-send="true"
href="mailto:erlang-questions@erlang.org"
target="_blank">erlang-questions@erlang.org</a><br>
<a moz-do-not-send="true"
href="http://erlang.org/mailman/listinfo/erlang-questions"
rel="noreferrer" target="_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
erlang-questions mailing list
<a class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
</blockquote>
<br>
</body>
</html>