[erlang-questions] Using CouchDB to hold state for FSM - pros and cons?

[Ulf Wiger]

Another piece of information that would be good to know is: what are the persistency requirements?

Is there a need for state to survive system crashes/restarts? Is there a need for redundancy at all?

If there is no need for persistency, Erlang has a wonderful model for holding FSM state - a process per FSM. Keeping tens of thousand processes in one erlang node is no problem whatsoever.

Keeping track of tens of thousand live processes used to be a problem, but that was why gproc was created, and it works very well as a process "index".

Various requirements on fault tolerance may make this model impractical. Speed or data volume requirements* may also drive towards a different solution, quite often in conflicting ways, so as Joe says, it's important to be specific - also about which tradeoffs are acceptable.

BR,
Ulf W


On 2 May 2011, at 09:52, Joe Armstrong wrote:

> You haven't sent enough information to make a reasonable judgment.
> 
> what is "lots" in the phrase "lots of state transactions per transaction" - 10? 1000? 10^10
> what is "high" in "extremely high"
> 
> You need to quantity words like "lots" and "high" with numbers or ranges of numbers 
> 
> How many transactions/second?
> How many simultaneous sessions?
> How large is the state
> Do you want the fault-tolerance?
> What are the latency requirements
> Up-time requirements
> 
> and so on.
> 
> Sounds to me like you want a fault-tolerant low-latency key-value store.
> 
> CouchDb will store all the data forever, which is probably not what you want. Scalaris or Riak 
> sounds like a better fit.
> 
> /Joe
> 
> 
> On Mon, May 2, 2011 at 8:56 AM, David Mitchell <monch1962@REDACTED> wrote:
> Hi all,
> 
> I've got to build an enormous finite state machine (FSM) to hold state for tens of thousands of in-flight transactions.  The various state transitions aren't well known yet; all I know is there's going to be lots of state transactions per transaction, transaction volume will be extremely high, and I'll probably have a bare minimum of time to throw together a working solution ;->
> 
> In the past I've used mnesia to hold transaction state for similar projects, but I've read a few articles about problems with it scaling and have a bit of a concern on that count for this particular project.  I've been using CouchDB for a few unrelated projects recently, and have been impressed with it on every score - on the surface it seems to be a good fit for my specific problem.  However I haven't stumbled on anybody using CouchDB as a high throughput, transitional data store - everything I've read about it has involved CouchDB working more or less as a NoSQL version of a "traditional" write-once, read-often data store.
> 
> My application is far more "write-once, update-many-times-very-quickly, delete" from a data perspective.
> 
> Has anyone used CouchDB as a FSM data store in a similar project in the past?  Pros and cons?  Any tips or suggestions?
> 
> Thanks in advance
> 
> David Mitchell
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
> 
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110502/66f97b74/attachment.htm>