[erlang-questions] Large scale deployments testing

[Joel Meyer]

Hi Sergey,

We (OpenX) have a riak_core based application that's running on a 125 node
cluster (there are also other smaller clusters). We never really tested to
see where it would fall over (and the cluster was much smaller when it
started), but I see no indicators that it will fall over when we add the
126th node. FWIW, it's running riak_core 0.13.0, and I assume the newer
versions of riak_core have only gotten better. Answers to some of your
other questions (based solely on my experience) in-line below.

On Fri, Sep 21, 2012 at 6:29 AM, Zhemzhitsky Sergey <
Sergey_Zhemzhitsky@REDACTED> wrote:

>  Hello gurus,****
>
> ** **
>
> We’re developing a riak-core application, that does not include any
> persistence and works in-memory, and are wondering what are the best use
> cases to test riak-core and erlang itself in large-scale deployments (>100
> physical nodes).****
>
> ** **
>
> For example some of the map-reduce frameworks (like hadoop) have
> performance tests like terasort, etc., which can show to what extent the
> whole framework can be scaled. ****
>
> ** **
>
> So could you share some ideas what are the best practices to test
> large-scale deployments of riak-core and erlang applications? What
> synthetic tests and benchmarks can be executed to answer the following
> questions: ****
>
> ** **
>
> **1.       **Does the system scale well?
>
Yes, so far it has scaled well.

> ** **
>
> **2.       **Can the system be considered as linearly scalable?
>
Yes, the riak_core portion can be considered linearly scalable. The overall
behavior is largely dependent on what you're doing in your vnodes and how
well you hash the things you want distributed. In theory, if you hash
poorly you can get hot-spots that will prevent linear scalability, but I
haven't seen that happen with our workload.

> ****
>
> **3.       **Is the system truly fault-tolerant?
>
For the most part, 'yes', but that again depends on how you implement your
vnode. The problems that I've encountered were due to my own inexperience
with erlang when implementing my vnode.

In general I've been very happy with riak_core and we're definitely looking
at using it more for places where it's the right solution.

Cheers,
Joel

> ****
>
> ** **
>
> Best Regards,****
>
> Sergey  ****
>
> ** **
>
> _______________________________________________________
>
>
>
> The information contained in this message may be privileged and conf
> idential and protected from disclosure. If you are not the original
> intended recipient, you are hereby notified that any review,
> retransmission, dissemination, or other use of, or taking of any action in
> reliance upon, this information is prohibited. If you have received this
> communication in error, please notify the sender immediately by replying to
> this message and delete it from your computer. Thank you for your
> cooperation. Troika Dialog, Russia.
>
> If you need assistance please contact our Contact Center (+7495) 258 0500or go to
> www.troika.ru/eng/Contacts/system.wbp
>
>
>
> _______________________________________________
> riak-users mailing list
> riak-users@REDACTED
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120921/d1e8e3ad/attachment.htm>