[erlang-questions] Packets deduplication
Thu Feb 18 19:28:24 CET 2016
The right data structure is either bloom filter or scalable bloom filter.
I've played with standard bloom here
20k might require you to do tuning and reflect the filter to ETS.
The ETS as such or any cache based on ETS might help you as well. Cache is needed to implement TTL for your ID's
Sent from my iPhone
> On 18 Feb 2016, at 14:01, Alexander Petrovsky <askjuise@REDACTED> wrote:
> I have the stream of packets with ID (int), and I need to check is the packet is uniq (by ID) or not?
> Incoming rate is about 20k pps and ID is monotonically grows. What's the best way and data structure fit for this problem?
> Петровский Александр / Alexander Petrovsky,
> Skype: askjuise
> Phone: +7 914 8 820 815
> erlang-questions mailing list
More information about the erlang-questions