[erlang-questions] Caching server

[Matt Stancliff]

On Feb 22, 2011, at 7:59 AM, Nicholas Wieland wrote:

> My question wasn't really about the example, but about the  
> architecture itself, I wanted to know if that was a "possible"  
> production architecture, and I think you all clarified my doubts.

(jumping in late)

   As nox graciously pointed out, my pcache project (a knockoff of  
Jay's Concurrent Caching paper [1]) does what you describe.  I use  
pcache for storing compiled website templates, rendered content (wiki  
markup -> HTML), generated rss feeds, appcast feeds, and half a dozen  
other things.

   A benefit of putting each cache item in its own process is you can  
use erlang's receive/after statement to auto-garbage-collect cache  
items if they go unused for a period of time (pattern: exploit  
erlang's built in event loop to simplify your code).  You can think of  
the cache data as entirely stored in the spawned function's argument  
[2].  When the timeout is reached, the spawned processes dies, and the  
cache data get garbage collected.

   Recently I converted the pcache API to store cache items in ets  
directly.  With ets as a cache backend, you can get compression with  
just an option to ets (since R14B01).  Individual processes are still  
spawned to terminate cache items when the TTL is over.  Each process  
goes from storing the data (as in pcache) to just being a timeout to  
delete the cache item when the TTL expires [3].

[1]: http://duomark.com/erlang/publications/acm2006.pdf
[2]: https://github.com/mattsta/pcache/blob/885cd037517011465cb00adf305fee4a3a94bc2e/src/pcache_server.erl 
#L312
[3]: https://github.com/mattsta/ecache/blob/f56a38af12b5f57968761bf09fffbed588e4b0ef/src/ecache_server.erl 
#L219


-Matt
-- 
Matt Stancliff                    San Jose, CA
@mattsta                  iPhone: 678-591-9337
"The best way to predict the future is to invent it." --Alan Kay