[erlang-questions] Erlang Moscow meetup: my speach about protecting system from overload

[Fred Hebert]

Hi Max,

Very interesting read. Really liked the following quote: "Deadlocked
system maintain good load (about zero) but it provides bad service
(about zero)".

One thing you seem to rely a lot on is the use of process_info/2. I
noticed you used it to get the message queue length, and also to read
the process dictionary by foreign processes.

Does accessing foreign data with process info have any noticeable cost?
How does it compare to using ETS tables? Do you use the process
dictionary due to better performances, or because it reduces the pain of
maintaining multiple tables?

I was wondering about using the message queue length as some arbitrary
measure of load to implement ideas described in
www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf 

Basically statistically optimal choices in [sequential] load-balancing
can be made by picking 2 random entries, checking the load, and picking
the most available one. Using message queue lengths for this would have
been useful, but I was afraid doing it on average twice per request
could have a prohibitive cost in a production system. I never tested it,
but your post reminded me of that.

Regarding the use of process dictionary to store states to debug
(put(status, write_frame_on_disk), etc.), would you still use that system
given newer OTP versions tell you what function they're currently
running? I'm assuming the process dictionary is a bit more fool proof to
refactoring when you come to rely on it.


On 01/14, Max Lapshin wrote:
> Meetup of erlang programmers was on 12 January in Moscow.
> 
> I was speaking about some useful tips for protecting erlang system from
> overload and crash.
> 
> Here they are in posts:
> 
> http://blog.erlyvideo.org/2013/01/01/overload-protection-1/  introducing
> feedback inside system parts
> http://blog.erlyvideo.org/2013/01/02/overload-protection-2/  early
> rejecting requests
> http://blog.erlyvideo.org/2013/01/03/deadlocks-1/  tips to workaround
> deadlocks
> http://blog.erlyvideo.org/2013/01/04/unobtrusive-read/  some notes about
> non-messaging reads
> http://blog.erlyvideo.org/2013/01/05/maintain-long-requests/  controlling
> long user tasks
> 
> It would be good if this will be helpful to someone. Any feedback on these
> ideas is welcome.

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions