How to get configuration data to a large number of threads?

[Raimo Niskanen]

I guess the fun's environment gets copied to all processes that gets
the fun. Hence no improvement regarding memory usage. No process
can reference data on some other process's heap.

But if you had Shared Heap or Hybrid Heap things would be different...



mscandar@REDACTED (Mark Scandariato) writes:

> (Sorry if this is a duplicate - the first attempt seems to have been
> eaten by a grue. I also changed it a little.)
> 
> If the config data is static per process (that is, changes won't affect active
> transactions) you could keep it in a fun that you spawn for each
> transaction. When the config changes, create a new fun.
> 
> You can keep the fun in a server that accepts (relatively infrequent)
> config change messages as well as call request messages.
> 
> The basic notion would look like:
> 
> start(Config) -> loop(txn(Config)).
> 
> txn(Config) ->
>      fun(Req) -> do_txn(Config, Req) end.
> 
> do_txn(Config, Req) ->
> 	% slice off an appropriately sized chunk of Config
>          spawn(fun() -> txn_handler(Subconfig, Req) end).
> 
> loop(Txn) ->
>      receive
>          {config, Config} ->
>              loop(txn(Config));
>          {txn, Req} ->
>              Txn(Req),
>              loop(Txn)
>      end.
> 
> (This looks suspiciously like Joe's generic server. Hmm.)
> 
> Mark.
> 
> 
> Vlad Dumitrescu wrote:
> > Untitled----- Original Message ----- From: Heinrich Venter
> >
> >> Each incoming transaction spwawns a process that handles it up to a
> >> point where output is generated. We need to be able to handle large
> >> volumes of transactions in a short time in bursts (SMS system using
> >> oserl http://oserl.sourceforge.net/)
> >> Each process needs to interpret the transaction content based on a set
> >> of configuration data. Unfortunately the entrie config set is needed
> >> for this and it could be as large as 5k (worst case, 1-2k is probably
> >> more realistic.)
> > Hi,
> > I got an interesting (hopefully) idea that might solve your problem
> > and also other similar ones.
> > What if it was possible to clone a process, with all it's data? If
> > there are no external references (large binaries that need to be
> > reference counted, file handles, links to other processes and so on)
> > then this could be faster and cheaper than just starting a process
> > and copying the data as Erlang terms.
> > Of course, the problem is how to ensure there are no such external
> > references, but maybe someone else has a solution to that.
> > regards,
> > Vlad

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB