large amount of processes - how to organize workers/manager processes ?

[Gaspar Chilingarov]

Hi all!

Another Question: how do supervisor handles large number of processes ?

I have about 8000-20000 processes, which sometimes - on timeouts or on 
data arrival do processing and time to time save their state to dets.
Also I have manager which maps FooID* to worker PID, but does not 
restart workers - it's ok for worker to die or lose some little amount 
of data, if no alive worker for some FooID found, worker is started by 
manager, it reads state from dets and continues to process arriving data.

Usually I have at least one message per minute for each worker process.

In case if manager process dies (for some reason) -- which is the better 
  way ? kill all worker processes and let them spawn again? or each 
worker process should somehow understand, that manager died, find 
restarted manager and register as worker for FooID -- because worker is 
totally independent from manager, when it processes data.


FooID -- some key to identify to whom data block should be sent.

-- 
Gaspar Chilingarov
System Administrator

t +37491 419763 (mob)
t +37410 240399 (office)
w www.web.am
i 63174784
e nm@REDACTED