[erlang-questions] Strange optimization result

[Thomas Lindgren]

--- Anders Nygren <anders.nygren@REDACTED> wrote:

> I did the change to ets:update_counter last night,
> on my dual
> core laptop that made an improvement from my
> previously
> reported
> real    0m6.305s
> user    0m10.149s
> sys     0m0.380s
> 
> to
> real    0m5.094s
> user    0m8.781s
> sys     0m0.352s
> 
> To avoid the update_counter race condition I have
> only one process
> that all workers report their matches to.
> 
> I had some problems with native compilation but
> finally made it work
> and
> real    0m2.192s
> user    0m3.260s
> sys     0m0.336s

Nice -- so native compilation yields a 2.3x speedup
versus the ets-based version (and 2.87x versus the
original). Have you tried setting that process to
priority high? (Can be dangerous in general, but in
the interest of science ... :-)

Parallelization yields 1.78x speedup on the first
version and 1.49x on the native code version. That's
pretty good too on two cores.

(I think the ultimate limit in most any language would
be a time of about 0.33 seconds, the sys time. But at
this point, we are actually within a factor 10 of
that.)

Okay, I just thought of something marginally more
clever when using ets. Maybe you did it this way,
even.

All the matching processes do this when they find a
match 'M':

case catch ets:update_counter(Tab, M, 1) of
   {'EXIT', Rsn} -> match_table_owner ! {init_key, M};
   _ -> ok
end

and the match_table_owner (which will be the sole
process inserting new matches into the table) does
something like

receive
  {init_key, M} ->
    case catch ets:update_counter(Tab, M, 1) of
       {'EXIT', _} -> ets:insert(Tab, {M, 1});
       _ -> ok
    end, ...;
  ...
end

This detects and handles the race condition by
funneling all initializations through the
match_table_owner and making the init there
idempotent. Furthermore, post-init updates will be
done by each matching process without sending
anything. Sounds potentially promising.

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com