How to get configuration data to a large number of threads?

[Heinrich Venter]

A bit more detail on the problem is in order I see :)

Each incoming transaction spwawns a process that handles it up to a
point where output is generated.  We need to be able to handle large
volumes of transactions in a short time in bursts (SMS system using
oserl http://oserl.sourceforge.net/)
Each process needs to interpret the transaction content based on a set
of configuration data.  Unfortunately the entrie config set is needed
for this and it could be as large as 5k (worst case, 1-2k is probably
more realistic.)
Some quick profiling shows that we can expect in the order of thousands
of processes active at the same time, making the memory overhead a
problem.
In order to keep the processing speed as fast as possible I want as many
prallel processes as I can manage.  Obviosly if I can get clever with
the config data, this would mean more processes.  Failing that, I will
have to place a lower limit on the number of processes so that they will
fit into memory or to prevent memory churn.

>From the coments up to now it seems that I am most likely going to have
to be satisfied with passing the config data as a parameter to the
function that executes in each thread and rely on the GC to keep things
sane.

-]-[einrich


"Vodacom Look4me - Click here for more info"
http://asp1.rocketseed.com/RocketSeed/mail/433a32353a3733373030313a33303639373a2d323a313533

Bought to you by Launchpad.
http://asp1.rocketseed.com/RocketSeed/mail/433a32353a3733373030313a33303639373a2d323a313438
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20041026/a65b8964/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MTk2LjMwLjc5LjE1NQ==
Type: image/gif
Size: 16471 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20041026/a65b8964/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MTk2LjMwLjc5LjE1NQ==
Type: image/gif
Size: 557 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20041026/a65b8964/attachment-0001.gif>