[erlang-questions] Efficient sum matrix by column
Sun Apr 12 21:49:28 CEST 2015
The proplists already sorted. Thanks, I will look at N-ary tree. Looks like
the NIF it's not a good idea due the sum of two lists with 100000 each will
be more then 1ms (btw I know about dirty schedulers).
2015-04-12 22:42 GMT+04:00 Hynek Vychodil <vychodil.hynek@REDACTED>:
> Sort by proper key and then merge join. Form N-nary tree between your
> nodes for merging and experiment with N. If not fast enough, convert to
> binaries and write NIFs, compress (Snappy, LZ4, ...) and so on. Unless you
> write more how keys in your proplists look like, there is not so much to
> On Sun, Apr 12, 2015 at 7:23 PM, Alexander Petrovsky <askjuise@REDACTED>
>> I have about 100 nodes in cluster, each node in cluster contains about
>> 100000 elements in proplist. All proplists have equal length, the same
>> keys, but different values. I need to get the sum values for every key
>> between all proplists. By example, it can be represented as matrix NxM
>> (N~=100, M~=100000), and I need to get the sum by column.
>> The problem is that it's too slow, even when I use the hackish way with
>> ets:update_counter and N=2, it's take about 1.5 secs, If I make in
>> parallel, it's take about 2-3 seconds.
>> How can I make it fast and efficient?
>> Петровский Александр / Alexander Petrovsky,
>> Skype: askjuise
>> Phone: +7 914 8 820 815
>> erlang-questions mailing list
Петровский Александр / Alexander Petrovsky,
Phone: +7 914 8 820 815
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the erlang-questions