[erlang-questions] How to split a single huge server?

[semmit mondo]

Hi, I have a single-process solution for a server that, I feel, could be implementetin a more concurrent fashion.  Its internal state is a vector that consists of n numbers.It receives casts without parameters.  Every cast is basically a matrix multiplication.The matrices are known in advance.  So, if the internal state is v, and the serverprocess receives a cast, and it has the matrix M associated with that cast, then thenew state will be M*v (M is a matrix, * is matrix multiplication, and v is a column vectorrepresenting the previous internal state of the server process). My problem is that the size of v (and the matrices) can be very large, and myserver runs in a single process.  It would be great to split it apart and use severalsmaller processes to calculate the new state.  I would like to use separateprocesses for each number in v.  But because of the nature of matrix multiplication,that's not so easy to achieve, because in order to be able to calculate a singlenumber in th
 e new state, I need to know all the numbers in the previous state.The prev state could be shared between processes in advance, but that wouldrequire large messages containing all the old values to all the processes around.I believe that is a wrong idea and there must be a better one. My question is how would you crach this problem if efficiency matters?  I have one possible solution in mind, and would like to know your opinion:  therecould be a main server process and size(v) calculator processes, one for everynumber in v.  The server process handles the casts, and it has the whole v vectorin it.  The calculator processes have only one number from v, and they have theproper lines from M.  When a cast arrives, the server process builds a fun that hasthe whole v encoded in it as a clojure, and this anon function gets sent to thecalculator processes.  Then the calculator processes apply the fun to theappropriate line of M, and they have the new number that they have to sendback to the 
 server.  I'm not sure if sending a huge !
 function with a large body ischeaper than sending a large list of numbers, but I hope there's some optimisationgoing on in BEAM with funs...  Am I right with all this? THX  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140805/4f1fd3e0/attachment.htm>