[erlang-questions] Abusing OTP-8623 for high-priority messages

[Peti Gömöri]

Hi

As far as I understand OTP-8623 how it works is it puts a marker in the
message queue when a reference is created (and other special conditions
apply) and in case of a selective receive which contains that reference
only the messages received after that marker are scanned. This makes sense
as messages that were sent before the reference was created cannot contain
that reference.

Considering this it makes no sense to use a reference that was created in
init since all (or almost all) messages will be sent after init.

The readme stating "constant time" is at least a bit misleading. It
probably only refers to the fact that no matter how long your message queue
already is if you create a reference now and you only expect to receive one
or few new messages after this it will be cheap to find that message in the
queue. It will still scan the new messages in sequence, no hashing or
anything.

I might be wrong of course

br
Peter


On Fri, Apr 11, 2014 at 7:50 AM, Michael Truog <mjtruog@REDACTED> wrote:

>  On 04/10/2014 08:56 PM, Dmitry Demeshchuk wrote:
>
> Well, actually, if Michael pointed me at a module that handles that and it
> was something similar, that could have been helpful :)
>
>  But, judging by the priorities from -128 to 127, it's probably a manual
> filter of some kind, probably with an intermediate mailbox-reducer process.
>
> The v1 UUID acts as the known reference with O(1) complexity here.  The
> priority is being handled by an Erlang priority queue data structure called
> pqueue4 since there has to be some queue structure for queuing service
> requests so that atomic transactions can occur for service requests.  This
> might not have been the priority you had in mind if you were thinking about
> various pattern matching cases on different priority values, but I have
> found the approach with a data structure more flexible and useful for the
> dynamic nature of a service, due to the need for a queue (which enforces
> the incoming limit, since it is easier than trying to do something
> error-prone with an Erlang pid directly, i.e., something that would modify
> the Erlang pid state or try to check it repeatedly).
>
>
>
>  Which is fine, actually, but my question wasn't "how to handle huge
> queues?", but "is using OTP-8623 for that an anti-pattern of some sort?".
>
>  And yeah, definitely not "what platform should I use?".
>
>
> On Thu, Apr 10, 2014 at 7:28 PM, zxq9 <zxq9@REDACTED> wrote:
>
>> On Thursday 10 April 2014 17:24:54 Michael Truog wrote:
>> > On 04/10/2014 05:02 PM, Dmitry Demeshchuk wrote:
>> > > Hi list,
>> > >
>> > > Today (yeah, just a couple years have passed since R14A) I learned
>> about
>> > > OTP-8623, which allows to do selective receive for messages that
>> contain
>> > > a known reference with O(1) complexity.
>>  ...
>> > > This could potentially help with sending managing messages to the
>> > > processes that may have overloaded inbox with thousands of messages.
>> Say,
>> > > we see that the process gets overloaded and we send an operational
>> > > message of some kind ("discard the entire inbox", for example).
>> > >
>> > > Thoughts?
>> >
>> > You could get this interaction by creating an internal service with
>> CloudI
>> > (http://cloudi.org).
>>
>>  Not so sure this is what the OP was getting at.
>>
>> "Is OTP8623 a lightweight way to overcome a clogged queue?" /= "Base
>> everything you do on some new service stack"
>>  _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
>
>  --
> Best regards,
> Dmitry Demeshchuk
>
>
> _______________________________________________
> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20140411/68fc6808/attachment.htm>