How to get configuration data to a large number of threads?

[Joe Armstrong]

   How about some lateral thinking here:

> I have a transaction based system that spawns a
> thread to handle every incoming transaction.  Unfortunately there is
> quite a large chunk of relatively static configuration information that
> is needed by every thread....
> The question is, how do I get this information to every thread without
> significantly slowing things down or using up all the available memory?


   You don't - you move the functions to the data - not the data to the
functions :-)

   Assume the  configuration data is HUGE  (GBytes) - if  this case I'd
send the computations to  the configuration data. Exactly the opposite
of what you did :-)

   So you make a global server like this

   -module(config).

   start() ->
 	AHugeDataStructure = read_initiaal_config_data(),
 	loop(AHugeDataStructure).


   loop(AHugeDataStructure) ->
 	receive
 	   {upgrade, AnotherHugeDataStructure} ->
 		loop(AnotherHugeDataStruce);
 	   {compute, From, Fun} ->
 		Val = (catch Fun(AHugeDataStructure)),
 	        From ! {self(), Val},
 	        loop(AHugeDataStructure)
 	end.

... now about the clients ...

    Suppose the client wants a username and password

    Make a query

 	Query = fun(Config) ->
 		   {get_from_config(username, Config),
 		    get_from_config(password, Config)
 		end,

   Send it to the config process with a

 	Config ! {compute, self(), Query}

 	and wait for the reply


   Note that sending Funs in messages (locally) is a very lightweight
operation (as I have explained before :-)

   Now you possibly have a bottleneck in the server - how to solve this?

   Make several servers with identical copies of the configuration data.

   Remember - it might be cheaper to move the functions to the data
than moving the data to the functions.

   RPC style programming in "Other Languages" (TM) makes you believe
that there is only only way of doing RPCs (ie moving the data to the functions)

   Erlang offers freedom of choice.

   Cheers

/Joe