How to get configuration data to a large number of threads?
Tue Oct 26 16:04:04 CEST 2004
The naive solution might be to keep *all* the configuration data in
a single server.
When one of threads wants configuration data it just does a normal
server RPC and the configuration data is returned to the client.
If you are smart then only the server would have a complete copy of
the configuration data, the individual processes would only request
the sub-set of the configuration data that they actually need
(assuming here that they did not need all the configuration data).
Once the client processes are done with their data the space needed
for the configuration data will be garbed away.
It's probably not a good idea letting the clients carry around a
complete copy of the configuration data, just in case they need a
small sub-set of this data for a small time.
Try writing the code in a lazy manner, so that the clients carry no
configuration data and only request exactly what they need when they
need it, and try to reduce the amount of time that they have to hold
copies of the sub-sets of the configuration data.
If the configuration data needs any complex processing that can be
pre-computed then you can cache the necessary values in the server.
This might help. Without being more explicit about your problem it
is difficult to give a better answer. Often the "pure Erlang" solution
(just use processes, no gets/puts ets/dets/mnesia) is the fastest way,
but in particular cases ets/dets etc will be faster - it all depends
upon your problem.
BTW - Erlang doesn't have threads - it has processes. What's the
difference? Formally, threads share stuff - processes don't.
Process provide much more protection than threads since they provide
you with error encapsulation domains - Erlang has "light weight
processes" meaning that creating an Erlang process is a much quicker
operation that creating a thread in "Most other" (TM) languages.
On Tue, 26 Oct 2004, Heinrich Venter wrote:
> Hello all
> I need a bit of advice. I have a transaction based system that spawns a
> thread to handle every incoming transaction. Unfortunately there is
> quite a large chunk of relatively static configuration information that
> is needed by every thread.
> With relatively static I mean that it is changeable through user
> intervention at any time.
> The question is, how do I get this information to every thread without
> signifficantly slowing things down or using up all the available memory?
> I am not sure, but I suspect passing it as a parameter will cause
> thousands of copies to be made. Is this correct?
> I was thinking of making a globally registered ets table, but this will
> cause copies to be made again once it is read from the table.
> The last option might be to declare all the information in a hrl file,
> but then changing the configuration would require recompilation.
> As I understand, using a binary formated chunk of data will prevent
> copying, but when converting back to a more usable format the data will
> be copied again.
> Any other ways of getting large chunks of configuration data quickly to
> a large number of transient threads?
> Help and advice appreciated.
> -]-[einrich Venter
> Look 4 Help - Click here for more info
More information about the erlang-questions