[erlang-questions] "actor database" - architectural strategy question

Miles Fidelman mfidelman@REDACTED
Tue Feb 18 14:42:28 CET 2014

Michael Truog wrote:
> On 02/17/2014 03:22 PM, Miles Fidelman wrote:
>> Any thoughts on how to do this - perhaps in combination with 
>> extending the hibernate BIF? 
> Most people use gen_server with hibernate in the return value from the 
> callback (e.g., 
> http://www.erlang.org/doc/man/gen_server.html#Module:handle_call-3). 
> If you need something that is more complex, CloudI has a hibernate 
> service configuration option that allows you to hibernate based on the 
> incoming service request rate, so when the service request rate drops 
> too low, the process hibernates whenever possible 
> (http://cloudi.org/api.html#2_services_add_config_opts_hibernate).

Ok... see that.  Thanks!

And thanks to Motiejus Jakštys for the existence proof, sizing info, 
(and the checkpointing scheme - not quoted):
> One portability request is one process. At any time we could have had 
> up to 1M processes (practically it was much lower, but we used this 
> number when designing the system). A "portability process" is a finite 
> state machine with communication in SOAP between two or three parties 
> with many internal state changes.
> A single process could last from a few hours up to few months (median 
> ~3-4 days), each state up to 10-100KB of uncompressed text (mean ~15KB 
> uncompressed).

The question about extending the hibernate BIF, and maybe playing with 
the scheduler, focuses on scaling further - to Joe Armstrong's comments 
> Some kind of "getting things out of memory and onto disk when not 
> needed" layer is needed for your problem,
> Yes - I guess the real solution would be to change the scheduler to 
> swap processes to disk after they had waited for more than (say) 10 
> minutes for a message, and resurrect them when they are sent a message.

It strikes me that hibernate addresses a big piece of the problem - 
reducing footprint of inactive processes, while keeping them "alive" 
(i.e., able to receive messages, wake up, and all that) - and when 
called via proc_lib:hibernate, keeps exception handling and logging working.

For scaling, the next step would be to swap out to disk - if that 
happens as a hibernate to disk, all the other functionality would stick 
around (as opposed to an application level write-state-to-disk).  Hence 
the question re. internals of the hibernate BIF and the scheduler, and 
how one might think about wiring in a hibernate-to-disk function.

Anyway... the overall approach is starting to look feasible!



In theory, there is no difference between theory and practice.
In practice, there is.   .... Yogi Berra

More information about the erlang-questions mailing list