[erlang-questions] process priority
Mon Jul 4 13:53:55 CEST 2011
My 2 cents.
Generally, writing straight to disk is a bad thing. You should have a
table (ets/mnesia) where you write your log lines and a process that
periodically flushes it to disk. Depending on your system load this
will help immensely.
In very rare cases you would tweak the process priorities and if you
do you should consider if your solution is "wrong" rather than the
priority being the bad guy.
On 4 July 2011 13:43, Martin Dimitrov <mrtndimitrov@REDACTED> wrote:
> Yes, the logging process writes to disk.
> I am not sure what thread pool means but our in our app only the logger
> does I/O operations. Can it block for more than 5 secs so the other
> processes to time out?
> On 7/4/2011 2:19 PM, Ulf Wiger wrote:
>> I assume the logging process writes to disk?
>> Are you running with a thread pool? If there is a lot of disk I/O, this can cause the emulator loop to block, unless there are enough I/O threads available.
>> I'm guessing the logging process has 'normal' priority right now? It is possible to set the priority of a gen_event process, e.g. by installing a handler (which gets to execute in the process context, thus being able to call process_flag(priority, P)). However, running a process at 'low' priority is generally not a good idea. It could cause the logging process to fall too much behind, and eventually running out of memory, or it could even in some cases lead to priority inversion, if the number of runnable normal-priority processes is significantly greater than the number of low-priority processes.
>> The runtime system does have one "feature", where it punishes the sender with extra reductions if the receiver's message queue is large. This could contribute to giving the logger process relatively more time than the other processes.
>> Fundamentally, though, you have to ensure that you don't produce more log output than your I/O subsystem can manage.
>> Ulf W
>> On 4 Jul 2011, at 12:31, Martin Dimitrov wrote:
>>> we have implemented a logging mechanism that calls gen_event:notify
>>> which returns immediately but nevertheless when performing extensive
>>> logging some genserver calls time out. I came to the conclusion that the
>>> erlang process manager is giving more time to the logging process and
>>> thus other processes never receive a chance to handle their queues.
>>> Is this conclusion possible or to seek the reason for the timeouts
>>> somewhere else? And if *yes* is it possible to give the logging process
>>> less priority?
>>> erlang-questions mailing list
>> Ulf Wiger, CTO, Erlang Solutions, Ltd.
> erlang-questions mailing list
More information about the erlang-questions