Billion-triple store

Leif Johansson <>
Wed May 3 19:29:28 CEST 2006

Hash: SHA1

Leif Johansson wrote:
>>> Well as the lead maintainer of kowari, I would be very happy to discuss
>>> any requirements you might have, and see if we can't help you.
> My application is a sink datastore for an enterprise message-bus. My
> message-bus is distributed and collects event-based data from a variety
> of sources including syslog (from 1000s of clients), hr-systems and
> student administration systems (for identity-management), LDAP
> directories, computer-telephony integration servers and network-
> management servers. I want to collect all messages in a datastore
> normalized as RDF. The datastore can quite possibly be distributed
> and must be able to support a sustained rate of 1000s of insertions
> of small RDFs per second.

I have done a couple of experiments with a totally brain-dead (probably)
schema on a single node with {n_fragments, 1000}. I can insert ~250k
statements at a sustained rate of ~2300 statements per second using
disk-copies of everything and a transaction per statement. The resulting
mnesia disk db is ~55M from a 25M n3 file turned into erlang statements
using perl :-)

Stay tuned...

	Cheers Leif
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla -


More information about the erlang-questions mailing list