[erlang-questions] ETS delete_trap

[James Aimonetti]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 04/26/2016 12:37 AM, Lukas Larsson wrote:
> On Tue, Apr 26, 2016 at 1:30 AM, James Aimonetti <james@REDACTED>
> wrote:
> 
>> 
>> The curious bit that I can't really figure out is the 
>> {current_function, {ets,delete_trap,1}} bit from the
>> process_info/1 call (output below). As far as I can tell, that is
>> buried in the VM's C code.
>> 
>> 
> The delete_trap function is used when you are calling either 
> ets:select_delete/2 or ets:delete/1 on a table with many many
> elements.
> 
> When working with ets tables that have many elements, erts takes
> smaller chunks of the table at a time and reschedules the process
> doing the processing.
> 
> So for ets:delete it would first delete the first X elements
> (calling delete), then reschedule, then delete the next X (calling
> delete_trap), and so on deleting X records per iteration. Most ets
> functions work this way in order to not hog the schedulers for too
> a long time when working with large tables.
> 
> Lukas
> 

This makes sense to a degree. We read the ETS table (protected, set)
from the worker process to see if there are any entries that should be
erased and cast into the gen_listener process to erase the individual
entries by key. The table is large-ish, maybe 1.5GB, but we are using
ets:delete(Tab, Key) for each Key found in the worker process (there
are associated callbacks so client code can be notified if the cached
entry is flushed/erased) which I assume should be fast. The only other
delete operation is to flush the whole table with
ets:delete_all_objects/1.

We haven't tried enabling the read/write concurrency flags yet; would
write_concurrency make a difference when calling ets:delete/2? This is
18.2 too, by the way. Sorry for not mentioning that first.

Any tips on profiling ETS usage and finding efficiencies?

Thanks for the information so far.

- -- 
James Aimonetti
Lead Systems Architect / Impressionable Scallywag
"If Dialyzer doesn't care, I don't care"

2600HzPDX | http://2600hz.com
sip:james@REDACTED
tel:415.886.7905
irc:mc_ @ freenode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBCAAGBQJXH32wAAoJENTKa+JPXCVgEcEIAIqug2b14Sq604f+Kkwd+SF1
oNhFC9ZN9pllzaEMzmEQ+k2gFVWdMnpUg4s2pPLnHiRPP24XcpAQzGXmU+R8hlTb
XmVrpQ6TFrTbVOi8lJlN3afMlK0XOgyjfd+QzbIMWIRLPyKnPq36bBbmD5Ybbrt1
/wNHVTxFJ/vJ7CxfaAXNNvli2DvRjiFYxzUn6t1b/C18ad0a+pFm1SZ1NdTOsHh9
Bcminbe8y3J/QJU/O62A8l2N40MvqD10O5O9RoHo9Wp3Uyz6NftDP/caDYKSbCLw
BYY0ziI7Wovk5IWZfaYAEY/5uJCH+DmCq6xdL+BKesKY/AMnlIWDH9O8jWq5RHI=
=z88N
-----END PGP SIGNATURE-----