How to get configuration data to a large number of threads?

[Joe Armstrong]

   The naive solution might be  to keep *all* the configuration data in
a single server.

   When one of  threads wants configuration data it  just does a normal
server RPC and the configuration data is returned to the client.

   If you are smart then only  the server would have a complete copy of
the configuration  data, the  individual processes would  only request
the  sub-set  of the  configuration  data  that  they actually  need
(assuming here that they did not need all the configuration data).

   Once the  client processes are done  with their data  the space needed
for the configuration data will be garbed away.

   It's probably  not a  good idea letting  the clients carry  around a
complete  copy of the  configuration data,  just in  case they  need a
small sub-set of this data for a small time.

   Try writing the code in a  lazy manner, so that the clients carry no
configuration data and  only request exactly what they  need when they
need it, and try  to reduce the amount of time that  they have to hold
copies of the sub-sets of the configuration data.

   If the configuration  data needs any complex processing  that can be
pre-computed then you can cache the necessary values in the server.

   This might help.  Without being  more explicit about your problem it
is difficult to give a better answer. Often the "pure Erlang" solution
(just use processes, no gets/puts ets/dets/mnesia) is the fastest way,
but in particular  cases ets/dets etc will be faster  - it all depends
upon your problem.

   Cheers

   /Joe

   BTW -  Erlang doesn't have threads  - it has  processes.  What's the
difference?  Formally, threads share  stuff - processes don't.

   Process provide much more protection than threads since they provide
you  with  error encapsulation  domains  -  Erlang  has "light  weight
processes" meaning that  creating an Erlang process is  a much quicker
operation that creating a thread in "Most other" (TM) languages.







On Tue, 26 Oct 2004, Heinrich Venter wrote:

> Hello all
>
> I need a bit of advice.  I have a transaction based system that spawns a
> thread to handle every incoming transaction.  Unfortunately there is
> quite a large chunk of relatively static configuration information that
> is needed by every thread.
> With relatively static I mean that it is changeable through user
> intervention at any time.
>
> The question is, how do I get this information to every thread without
> signifficantly slowing things down or using up all the available memory?
>
> I am not sure, but I suspect passing it as a parameter will cause
> thousands of copies to be made.  Is this correct?
> I was thinking of making a globally registered ets table, but this will
> cause copies to be made again once it is read from the table.
> The last option might be to declare all the information in a hrl file,
> but then changing the configuration would require recompilation.
> As I understand, using a binary formated chunk of data will prevent
> copying, but when converting back to a more usable format the data will
> be copied again.
>
> Any other ways of getting large chunks of configuration data quickly to
> a large number of transient threads?
>
> Help and advice appreciated.
>
> -]-[einrich Venter
>
>
>
> Look 4 Help - Click here for more info
> http://asp1.rocketseed.com/RocketSeed/mail/433a32353a3733353731353a33303639373a2d323a313933
>
>
> http://asp1.rocketseed.com/RocketSeed/mail/433a32353a3733353731353a33303639373a2d323a313836
>