[erlang-questions] Twoorl: an open source Twitter clone
Wed Jun 4 00:16:15 CEST 2008
On Wed, 2008-06-04 at 00:06 +0200, Dominic Williams wrote:
> Hi Joe,
> > It would be a zillion times easier if the *client* did
> > all this messing around. If the client knew about N
> > server address it could automatically redirect to a
> > different server if the primary server failed. if the
> > client knew about N machines and performed the md5(Key)
> > calculation to find the correct machine then the problem
> > would be very easy to solve with no messing around in the
> > server.
> Indeed, and I've used this strategy in the past on systems
> in which we were responsible for both client and server.
> It has one problem, though, which is if the network is not
> completely reliable and can become partitioned - different
> clients can end up using different servers, resulting in an
> inconsistent state. The way around this - leader election -
> works nicely in small systems but surely it doesn't scale to
> millions of clients?
This is why systems that use hashing to distribute load like Amazon's
dynamo also employ a "gossip protocol" to maintain group consistency and
perform failure detection. Even though it is not stated more
specifically in the paper, it is a reasonable guess, base on the
background of their CTO (Werner Vogels), that this employs some form of
The node receiving the call from a client (which may be behind in group
membership and partition knowledge) will set it straight.
More information about the erlang-questions