[erlang-questions] Where are the binaries in message queue but not part of process_info(Pid, binary) ?

[Lukas Larsson]

Hello,

On Tue, Oct 17, 2017 at 11:38 PM, Bekes, Andras G <
Andras.Bekes@REDACTED> wrote:

> - It seems the reliable way to measure the amount of data in the message
> queue is setting message_queue_data process flag to off_heap beforehand
> (ideally at spawn time with spawn_opt), and retrieve the full message queue
> content with process_info(Pid,messages) – which must be fully scanned. Is
> there a better solution? (I originally aimed at refc binaries only because
> I just wanted a quick estimation, and in my case anything else was
> negligible.)
>

The off_heap flag should not effect anything when you are using
process_info(Pid, messages). If you want the size of the data in the
message queue then there is no better option than process_info(Pid,
messages).

Do keep in mind though, that any message queue inspection tools only show
the messages that is part of what we call the internal message queue.
Messages that are part of the external queue are considered to be
in-transit to the process and thus not part of the message queue. In an
overloaded process the majority of the messages will most likely be in the
external message queue.


>
>
> - May I suggest to improve the documentation of erlang:process_info(Pid,
> binary) by the below facts, also referring to message_queue_data process
> flag?
>

Yes, good idea.


>
>
> - Is it possible to implement a new process_info_item() of
> erlang:process_info/2 that gives information about binaries currently in
> the message queue but off-heap?
>
>
It should be possible to do for messages with off_heap data, however for a
on_heap message queue it is only easy to get the messages that ended up
off-heap, the binaries of messages that are on-heap would not be part of
the result. This makes the API hard to explain which is not good.

Also this inspection would again only be for the internal message queue.


>
>
> - The documentation suggests using off_heap message_queue_data “if the
> process potentially can get many messages”, but this is rather vague. Does
> this refer to processes expected to have many **queued** messages? I
> guess a process receiving but not queuing many messages performs better
> with on_heap (“Performance of the actual message passing is however
> generally better when not using flag off_heap.”).
>

yes it does. I'll try to clarify that in the documentation.

Lukas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20171024/de95d61a/attachment.htm>