Feature: Mnesia hooks

k32 k32@REDACTED
Fri Jan 28 17:26:50 CET 2022


We've been experimenting with the ways to make large Mnesia clusters
more viable. Our basic idea is to move away from full-mesh topology to a
"mesh+star" topology: a small part of the cluster behaves like a regular
mnesia cluster (we call it core cluster) connected in a full mesh, but
the rest of the nodes are read-only; they passively and asynchronously
replicate transactions from the core cluster over custom protocol, and
delegate write operations to the core nodes via RPC, hence "star".

So far this approach looks quite promising, we've been able to achieve
decent throughput in 20+ node clusters, so this may be a possible
solution to the mnesia scalability challenge. But we had to patch mnesia
a bit: https://github.com/emqx/otp/pull/16/files (the initial idea to
rely on the events alone proved to introduce too much overhead).

Ideally, we would like to contribute this patch upstream, so we're
seeking the opinion from the OTP team. Hook API may be of use for other
things too. Not sure if this is the right channel, though.


More information about the erlang-questions mailing list