[erlang-questions] tcerl memory usage, plans for hash-type db

Viktor Sovietov victor.sovetov@REDACTED
Fri Aug 15 02:42:49 CEST 2008


If you want to estimate tc memory requirements without Erlang driver
attached, you can speak to tokyocabinet Tyrant
(http://tokyocabinet.sourceforge.net/tyrantdoc/) with using memcached
protocol. At least it will geve an imagination about real cache sizes on
your system.



jason pellerin wrote:
> I'm interested in using tcerl for a few projects, but I'm running into
> issues that I don't know how to resolve. The big one is memory use.
> While I can tune it a bit by decreasing the leaf and non-leaf node
> caches, tcerl still seems to grab tons of RAM, much more than I was
> (naively?) expecting. With the two cache params set to 64 each, a db
> of 7000 40k records takes up almost 400mb of ram. With the default
> cache params of 1024 and 512, it consumes over 1G. Syncs of a db of
> this size also take several seconds, which effectively means that I'd
> never be able to sync, except when shutting down a node.
> Tcerl folks -- am I doing something wrong? I need to support dbs of
> about 100,000 records of this size (with a *lot* of reads and writes)
> with reasonable enough RAM usage that each node can fit onto an EC2
> instance with a bunch of other stuff -- realistically, 256M at most.
> I'd guess that the tokyocabinet hash db would be better suited to our
> needs, since we don't need ordered_set. But would the RAM use be any
> different, and do you have near-term plans to add support for the hash
> db to tcerl?
> Does anyone have any other suggestions for storage backends to try?
> Dets unfortunately is much too slow, and mnesia with disc tables too
> RAM-hungry.
> Thanks (and thanks for tcerl!)
> JP
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

View this message in context: http://www.nabble.com/tcerl-memory-usage%2C-plans-for-hash-type-db-tp18984873p18989955.html
Sent from the Erlang Questions mailing list archive at Nabble.com.

More information about the erlang-questions mailing list