[erlang-questions] Distributed publish/subscribe system
Wed Jan 26 14:00:51 CET 2011
On Tue, Jan 25, 2011 at 4:00 PM, Kaiduan Xie <> wrote:
> Thanks Jack, Tony, Toby, Joe and Thomas for the helpful comments and
> resources. Joe pointed out a lot of excellent points. The requirements
> are listed below,
> 1) The system should be able to support tens to hundreds of millions
> users spreading around the world, so the system will consist hundred
> of nodes located in different physical locations.
Who own the nodes? are they all owned by the same organisation?
Who is legally responsible for the system?
> 2) Any user can subscribe to the interested topic, but only topic
> owner can publish message to the group. We have no limit on the number
> of subscribers in each group. It can be huge, for example, the fans of
> Rolling Stone around the world.
How do your stop spam - bogus groups etc?
> 3) The number of topics/groups in the system is unlimited.
How do you stop name squatting - ie flooding the system with stupid channel
> 4) User is not always connected to the system, not always connected to
> the same node in the system, and the message delivery should be
Uuug - impossible - it's the Byzantine generals problem. How do you know
they got a
message? - send an ack - how do you know if they got the ack?- ack the ack -
how do you know
if they acked the ack, ack the acked ack ...
> 5) As to the latency, it should be in the range of 1 minute if
> subscriber is connected.
> I am doing research on this topic now, the other requirements are not
> sorted out yet.
> On Tue, Jan 25, 2011 at 3:56 AM, Joe Armstrong <> wrote:
> > Standard question:
> > How big? - how scalable - what latency - fault tolerance requirements do
> > have.
> > Distributed can mean 2 nodes - or 10^9
> > There might be ten channels or 10^12
> > How fault tolerant? - 2 replicas, 100 replicas.
> > What the latency? - do you want the message to be delivered with 10 ms or
> > 100 years
> > What's the security model - who owns the data - is there centralized
> > security or weak
> > per-node security policies.
> > The point is your requirements need to say things like this otherwise
> > are not much use.
> > The architectures for these different cases will be *completely
> > /Joe
> > On Tue, Jan 25, 2011 at 12:54 AM, Kaiduan Xie <>
> >> Hi,
> >> Any one has experience to build distributed publish/subscribe system?
> >> The system has the following requirements,
> >> 1. Subscriber subscribes to the interested topic
> >> 2. Publisher publishes event related to the topic
> >> 3. System delivers the event to all the users subscribed to this
> >> topic. If subscriber is connected, the event is delivered immediately.
> >> If the subscriber is not connected, system should persist the
> >> event. When the user next time becomes online, system delivers the
> >> event to the user.
> >> Can you share the experience?
> >> Best regards,
> >> /Kaiduan
> >> ________________________________________________________________
> >> erlang-questions (at) erlang.org mailing list.
> >> See http://www.erlang.org/faq.html
> >> To unsubscribe; mailto:
More information about the erlang-questions