[erlang-questions] 0MQ libraries

[Joe Armstrong]

On Sat, Feb 8, 2014 at 2:39 PM, Ciprian Dorin Craciun <
ciprian.craciun@REDACTED> wrote:

>     (Sorry for "reviving" this thread, but only now I've managed to
> drain the mail queue.)  :)
>
>
> On Fri, Jan 31, 2014 at 9:12 AM, Joe Armstrong <erlang@REDACTED> wrote:
> > Why don't we just use gen_tcp and interact with 0MQ peers using the
> > wire-line protocol. Erlang is good at getting data in and out of sockets,
> > and good at parsing binary packets.
> >
> > I'm not suggesting implementing all the protocols and transports to start
> > with just the REQ - REP protocols of TCP
> >
> > [...]
>
>
>     Implementing a pure Erlang ZeroMQ library that understands only
> TCP and some of the basic patterns (such as (X)REQ/REP, PULL/PUSH), as
> Joe has suggested, could easily be a weekend project (within some
> acceptable performance limits), as the wire protocol is almost
> trivial, and the routing logic for the mentioned patterns is simple.
>
>
>     However ZeroMQ offers two orthogonal set of "abstractions":
>     * the patterns, which can be natively implemented in Erlang;
>     * the transports, some of which can't be implemented in pure
> Erlang (e.g. UNIX socket domains);
>     * (let alone the other features such as HVM, and the new additions in
> 4.x);
>
>     And the true power of ZeroMQ lies in the various ways we can
> combine the above features, all interoperable from one language to the
> other.  Thus having a myriad of partial implementations would just
> break the ecosystem into isolated islands  (like I guess happened with
> AMQP).
>
>
>     However...
>
> > ( actually all I want to do is get Erlang talking to Julia - Julia speaks
> > 0MQ and I found that I
> > couldn't build the Erlang 0MQ bindings with the latest 0MQ - so I'm again
> > back to
> > "fixing stuff thats broken" in order to do what I want.
>
>     For such a use-case, and other situations where a full ZeroMQ
> library is not needed, one could take the following shortcut:
>     * he limits the usage to only (X)REQ/REP or PULL/PUSH, with only
> one frame per interaction (plus the "destination" one in case of XREP,
> and a "correlation" in case of parallel request / reply sessions);
>     * he builds a simple C application which reads / writes on stdin /
> stdout messages in the well-supported format <payload-size payload>
> (maybe one per "destination", "correlation" and "payload") and
> translates that to a ZeroMQ `send` / `recv`;
>     * he then just uses this executable with `open_port` with `packet`
> option;  (which in fact it could be reused in other non-Erlang
> environments;)
>
>     Of course if more about the application is known, one could make
> other "simplifications", like use only JSON without embedded newlines,
> and transform the protocol into one line per message  (such an
> application could be used even from `sh`).  Moreover such an
> application could be used at both ends, thus obtaining a poor-man's
> messaging solution.  :)
>
>
>     As such I would definitely see the need of both kind of ZeroMQ
> interfaces for Erlang (or other languages): a binding based one, and a
> "process" based one.
>

Absolutely - I want to send message to 0MQ the "erlang way"

To send a message to 0MQ I should say

     Zmq ! IoList

and *not* call an erlang routine that is the equivalent of some C routine
in the ZMQ libraries

The Erlang Zmq ! Msg should pack data in the appropriate wire-line format
and blast it onto the
wire. Incoming messages should be parsed and sent to the appropriate
message.

The trick is to make an interface that makes ZMQ endpoints look and feel
like erlang processes.
We should not "know" that they are ZMQ endpoints. So socket closes should
be converted to
exit signals and so on.

When we talk to Julia we have to perform a load of data conversions - an
erlang list is
[1,2,3] and a julia list is [1 2 3] somebody somewhere must do some
conversion. I was thinking that
Erlang could send julia data structures down the wire, and Julia could send
Erlang structures
down the wire since it's far easier to generate Julia in Erlang than parse
it, and in Julia it will be
easier to generate Erlang than parse it. JSON would be equally bad for both
sides.


/Joe

>
>     Ciprian.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140208/b0f507d4/attachment.htm>