[erlang-questions] Using CouchDB to hold state for FSM - pros and cons?
Ulf Wiger
ulf.wiger@REDACTED
Mon May 2 10:37:37 CEST 2011
Another piece of information that would be good to know is: what are the persistency requirements?
Is there a need for state to survive system crashes/restarts? Is there a need for redundancy at all?
If there is no need for persistency, Erlang has a wonderful model for holding FSM state - a process per FSM. Keeping tens of thousand processes in one erlang node is no problem whatsoever.
Keeping track of tens of thousand live processes used to be a problem, but that was why gproc was created, and it works very well as a process "index".
Various requirements on fault tolerance may make this model impractical. Speed or data volume requirements* may also drive towards a different solution, quite often in conflicting ways, so as Joe says, it's important to be specific - also about which tradeoffs are acceptable.
BR,
Ulf W
On 2 May 2011, at 09:52, Joe Armstrong wrote:
> You haven't sent enough information to make a reasonable judgment.
>
> what is "lots" in the phrase "lots of state transactions per transaction" - 10? 1000? 10^10
> what is "high" in "extremely high"
>
> You need to quantity words like "lots" and "high" with numbers or ranges of numbers
>
> How many transactions/second?
> How many simultaneous sessions?
> How large is the state
> Do you want the fault-tolerance?
> What are the latency requirements
> Up-time requirements
>
> and so on.
>
> Sounds to me like you want a fault-tolerant low-latency key-value store.
>
> CouchDb will store all the data forever, which is probably not what you want. Scalaris or Riak
> sounds like a better fit.
>
> /Joe
>
>
> On Mon, May 2, 2011 at 8:56 AM, David Mitchell <monch1962@REDACTED> wrote:
> Hi all,
>
> I've got to build an enormous finite state machine (FSM) to hold state for tens of thousands of in-flight transactions. The various state transitions aren't well known yet; all I know is there's going to be lots of state transactions per transaction, transaction volume will be extremely high, and I'll probably have a bare minimum of time to throw together a working solution ;->
>
> In the past I've used mnesia to hold transaction state for similar projects, but I've read a few articles about problems with it scaling and have a bit of a concern on that count for this particular project. I've been using CouchDB for a few unrelated projects recently, and have been impressed with it on every score - on the surface it seems to be a good fit for my specific problem. However I haven't stumbled on anybody using CouchDB as a high throughput, transitional data store - everything I've read about it has involved CouchDB working more or less as a NoSQL version of a "traditional" write-once, read-often data store.
>
> My application is far more "write-once, update-many-times-very-quickly, delete" from a data perspective.
>
> Has anyone used CouchDB as a FSM data store in a similar project in the past? Pros and cons? Any tips or suggestions?
>
> Thanks in advance
>
> David Mitchell
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110502/66f97b74/attachment.htm>
More information about the erlang-questions
mailing list