How to get configuration data to a large number of threads?

[Mark Scandariato]

(Sorry if this is a duplicate - the first attempt seems to have been eaten by a 
grue. I also changed it a little.)

If the config data is static per process (that is, changes won't affect active
transactions) you could keep it in a fun that you spawn for each transaction. 
When the config changes, create a new fun.

You can keep the fun in a server that accepts (relatively infrequent) config 
change messages as well as call request messages.

The basic notion would look like:

start(Config) -> loop(txn(Config)).

txn(Config) ->
     fun(Req) -> do_txn(Config, Req) end.

do_txn(Config, Req) ->
	% slice off an appropriately sized chunk of Config
         spawn(fun() -> txn_handler(Subconfig, Req) end).

loop(Txn) ->
     receive
         {config, Config} ->
             loop(txn(Config));
         {txn, Req} ->
             Txn(Req),
             loop(Txn)
     end.

(This looks suspiciously like Joe's generic server. Hmm.)

Mark.


Vlad Dumitrescu wrote:
> Untitled----- Original Message ----- From: Heinrich Venter
> 
>> Each incoming transaction spwawns a process that handles it up to a
>> point where output is generated. We need to be able to handle large
>> volumes of transactions in a short time in bursts (SMS system using
>> oserl http://oserl.sourceforge.net/)
>> Each process needs to interpret the transaction content based on a set
>> of configuration data. Unfortunately the entrie config set is needed
>> for this and it could be as large as 5k (worst case, 1-2k is probably
>> more realistic.)
> 
> 
> Hi,
> 
> I got an interesting (hopefully) idea that might solve your problem and 
> also other similar ones.
> 
> What if it was possible to clone a process, with all it's data? If there 
> are no external references (large binaries that need to be reference 
> counted, file handles, links to other processes and so on) then this 
> could be faster and cheaper than just starting a process and copying the 
> data as Erlang terms.
> 
> Of course, the problem is how to ensure there are no such external 
> references, but maybe someone else has a solution to that.
> 
> regards,
> Vlad