[erlang-questions] Efficient sum matrix by column

Vance Shipley vances@REDACTED
Mon Apr 13 08:57:21 CEST 2015


On Mon, Apr 13, 2015 at 1:33 AM, Alexander Petrovsky <askjuise@REDACTED> wrote:
> Every element in proplist it's number of requests from client for 1 second

So conceptually a table where each row is indexed by client with
columns having per second totals for various data points.

> Every second I want to recalculate summary rate per client per cluster

Every second, at each node, for each row, hash the client to select a
node (e.g. ClientNum rem NumNodes = NodeNum).
Send each node just it's client data.
At each node sum the data received from all nodes and send the result
to the collecting node.
At the collecting node receive the summary data from all nodes and
reassemble into the summary table.

On Sun, Apr 12, 2015 at 10:53 PM, Alexander Petrovsky
<askjuise@REDACTED> wrote:
> I have about 100 nodes in cluster, each node in cluster contains about
> 100000 elements in proplist. All proplists have equal length, the same keys,
> but different values. I need to get the sum values for every key between all
> proplists. By example, it can be represented as matrix NxM (N~=100,
> M~=100000), and I need to get the sum by column.

-- 
     -Vance



More information about the erlang-questions mailing list