[erlang-questions] update_counter on steroids

[Jay Nelson]

Anthony Ramine writes:

> I plan to implement ets:take_all_objects(Table) which would empty out a table and return all its rows. Would you have a use for such a thing too?

In https://github.com/duomark/epocxy I implement a FIFO queue using array
semantics in an ets table in the file ets_buffer.erl. Reading everything in the
queue could take advantage of ets:take_all_objects(Table, Key_Pattern).

I use metadata tagged entries so that I can have multiple FIFO queues in a single
ets table (or an ets table may hold just one queue, but have a single metadata row).
For example, an ets_buffer might contain the following rows:

{ets_buffer, {meta, my_fifo_queue}, …}   % Meta-info with pointers to FIFO queue
{buffer_data, {my_fifo_queue, 15}, timestamp, {mydata, 23}}
{buffer_data, {my_fifo_queue, 16}, timestamp, {mydata, 43}}
{buffer_data, {my_fifo_queue, 17}, …}

I would want to read all the #buffer_data{key={my_fifo_queue, ‘_’}} records and
have them removed to return all items in the queue. I use an ordered_set ets table.

[ For my case, this will not actually be sufficient, because I would need to reset the pointers
  in the metadata record and that operation would not be atomic with the take_all,
  but I suspect this sort of key partitioning is a common occurrence and would be
  needed in many other situations. ]

Off the top of my head though, using an ordered_set ets table as if it were an orddict
could be a handy way to sort windows of transactions:

1) Concurrently insert transactions from throughout a server
2) Periodically call ets:take_all_objects

Every period you get a sorted set of the transactions that occurred. Things like folsom
could probably benefit from this approach. If you insert different types of transactions
you would want to apply the match spec pattern trick to the take_all (and could then
use different period windows for different transaction types).

jay