[erlang-questions] Configuring Messaging Back Pressure

[Andy Till]

I don't think you can rely on the scheduler alone if there are, say ten 
producers giving work to one consumer asynchronously.  Using 
gen_server:call from the producer to the consumer could help here?

Cheers

Andy

On 21/11/2014 12:23, Olivier BOUDEVILLE wrote:
> Hi,
>
> The reciprocal case (M producers and N consumers, with M >> N) may 
> also be a problem.
>
> I can share a simple program which seems to show that the retaliation 
> upon the producers is at least in some cases (where M is the number of 
> cores of the machine minus 1, and N=1) insufficient: the program will 
> crash and burn, exhausting the memory because of the mailbox of the 
> consumer becoming too large. Ideally the back-pressure would increase 
> steadily until the mailbox of the producer decreases below reasonable 
> limits. Currently, I think that the developer has either to tune M by 
> hand (which would depend on the hardware resources...), or to perform 
> some explicit yielding (with timer:sleep(1) on the producers). Neither 
> of them is very satisfactory.
>
> More generally, couldn't we hope that the scheduling logic could cope 
> with at least most of the M producers-N consumers scenarios, without 
> resorting to hints from the developer like specifying a process flag 
> or performing explicit yielding? Maybe (for the M >> N case) it is 
> just a matter of heavier punishing (ex: ultimately exponential) 
> whenever the receiving mailbox becomes too large.
>
> Best,
>
> Olivier.
> ---------------------------
> Olivier Boudeville
>
> EDF R&D : 1, avenue du Général de Gaulle, 92140 Clamart, France
> Département SINETICS, groupe ASICS (I2A), bureau B-226
> Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 
> 47 65 27 13
>
>
> *darach@REDACTED*
> Envoyé par : erlang-questions-bounces@REDACTED
>
> 21/11/2014 11:27
>
> 	
> A
> 	atill@REDACTED
> cc
> 	erlang-questions@REDACTED
> Objet
> 	Re: [erlang-questions] Configuring Messaging Back Pressure
>
>
>
> 	
>
>
>
>
>
> +1
>
> On 21 Nov 2014 04:53, "Andy Till" <_atill@REDACTED 
> <mailto:atill@REDACTED>> wrote:
> Short outline of how erts_use_sender_punish works, please comment if 
> there are any errors in the description.
>
> erts_use_sender_punish is a flag hard coded to 1 (true), when it is 
> true a process sending messages to another process will have its 
> reduction count reduced by the number of messages in the receivers 
> message queue multiplied by four.  Sending messages to processes with 
> zero messages in the queue is free in terms of reductions, but sending 
> messages to load queues is very expensive and will lead the scheduler 
> to context switch to another process more often.
>
> In the case where the the relationship between of producers and 
> consumers is one to one, this makes sense.  It provides back pressure 
> when part of the system is loaded.
>
> In the case where there is one producer to many consumers, if one 
> consumer gets a flood of messages and the producer is punished then 
> the punishment is not only on the producer but on all other consumers 
> because the producer cannot create enough work for them as it used all 
> of its reductions.
>
> This makes performance unpredictable when work cannot be spread evenly.
>
> Could this be made to be configurable using a process flag?
>
> Cheers
>
> Andy
>
> Code:
> _
> __https://github.com/erlang/otp/blob/a70f0ce9b34e4db61dacb8db24f9ab5671ed4c8a/erts/emulator/beam/erl_init.c#L694__
> __https://github.com/erlang/otp/blob/682a6082159568f40615f03d12d44ee70edd14c6/erts/emulator/beam/bif.c#L2058__
> __https://github.com/erlang/otp/blob/a8e12f7168c14cc765a63a51c838d065412795d7/erts/emulator/beam/erl_message.c#L504_
>
> Previous mentions and issues with this feature:
> _
> __http://erlang.org/pipermail/erlang-questions/2011-July/060314.html__
> __http://erlang.org/pipermail/erlang-questions/2011-July/060042.html_
>
> _______________________________________________
> erlang-questions mailing list_
> __erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>_
> __http://erlang.org/mailman/listinfo/erlang-questions________________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
> Ce message et toutes les pièces jointes (ci-après le 'Message') sont 
> établis à l'intention exclusive des destinataires et les informations 
> qui y figurent sont strictement confidentielles. Toute utilisation de 
> ce Message non conforme à sa destination, toute diffusion ou toute 
> publication totale ou partielle, est interdite sauf autorisation expresse.
>
> Si vous n'êtes pas le destinataire de ce Message, il vous est interdit 
> de le copier, de le faire suivre, de le divulguer ou d'en utiliser 
> tout ou partie. Si vous avez reçu ce Message par erreur, merci de le 
> supprimer de votre système, ainsi que toutes ses copies, et de n'en 
> garder aucune trace sur quelque support que ce soit. Nous vous 
> remercions également d'en avertir immédiatement l'expéditeur par 
> retour du message.
>
> Il est impossible de garantir que les communications par messagerie 
> électronique arrivent en temps utile, sont sécurisées ou dénuées de 
> toute erreur ou virus.
> ____________________________________________________
>
> This message and any attachments (the 'Message') are intended solely 
> for the addressees. The information contained in this Message is 
> confidential. Any use of information contained in this Message not in 
> accord with its purpose, any dissemination or disclosure, either whole 
> or partial, is prohibited except formal approval.
>
> If you are not the addressee, you may not copy, forward, disclose or 
> use any part of it. If you have received this message in error, please 
> delete it and all copies from your system and notify the sender 
> immediately by return message.
>
> E-mail communication cannot be guaranteed to be timely secure, error 
> or virus-free.
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20141121/4bda5be2/attachment.htm>