[erlang-questions] ETS’s performances and limits?

[Valentin Micic]

On 01 Sep 2018, at 11:09 PM, ANTHONY MOLINARO wrote:

> 
> I’ve got a few ordered_sets with ~12 million entries in them which seem to work reasonably well under normal loads, but they tend to show abnormal performance under heavier concurrency.
> 

There is a good chance that performance degradation you are referring to may *not* be related to a "heavier concurrency" as much as it may be caused by a rate of inserts into a table of type "ordered_set". The manual indicates that ordered_set tables do behave differently "in come situations" (which I read as "insert operations when a table gets big").

Out of interest, could you please quantify "heavier concurrency" in terms of a number of processes running and interacting with the table?

My experience is with an ETS table of type "set", hosting more than 190 million entries, and a steady insert rate (which may be updating existing, but also adding new entries) of 200 entries per second, with concurrent lookups from more than a hundred short-lived processes at any point in time (well, rather, most of the time). 

Apart from requiring some time to load all that data from disk to the ETS memory (System Start-Up), and needing some time to release all the memory upon ETS table destruction (System Shutdown), the performance is quite consistent.

Kind regards

V/