[erlang-questions] run_erl and fsync

[Damienuk Davis]

It's linux (Redhat ES 5.2) platform with following server configuration
DL370 with 2*6core and 12GB RAM 
Is it safe to define O_SYNC (this will cause to bybass fsync call)? What can be the impact to the system?




________________________________
 From: Dmitry Kolesnikov <dmkolesnikov@REDACTED>
To: Damienuk Davis <damienuk@REDACTED> 
Cc: "erlang-questions@REDACTED" <erlang-questions@REDACTED> 
Sent: Thursday, July 26, 2012 8:48 PM
Subject: Re: [erlang-questions] run_erl and fsync
 

I am curious what platform you are build the staff and what is the config?
fsync is behind USE_FSYNC but 


#ifndef O_SYNC
#define O_SYNC 0
#define USE_FSYNC 1
#endif

- Dmitry


On Jul 26, 2012, at 3:36 PM, Damienuk Davis wrote:

In my project there are fsms that communicates with c via erl driver (port). There are io:fwrite's in the erlang side. This program along works fine. But when you run some other process with high disk access, it causes erlang program to halt even if there is enough memory and processing. If I comment the fsync is run_erl.c file then program works fine even with higher load on hard disk. 
>static void write_to_log(int* lfd, int* log_num, char* buf, int len)
>{
>  int size;
>
>
>  /* Decide if new logfile needed, and open if so */
>
>
>  size = lseek(*lfd,0,SEEK_END);
>  if(size+len > log_maxsize) {
>    close(*lfd);
>    *log_num = next_log(*log_num);
>    *lfd = open_log(*log_num, O_RDWR|O_CREAT|O_TRUNC|O_SYNC);
>  }
>
>
>  /* Write to log file */
>
>
>  if (write_all(*lfd, buf, len) < 0) {
>    status("Error in writing to log.\n");
>  }
>
>
>#if USE_FSYNC
>  fsync(*lfd);//after commenting this line it works fine 
>#endif
>}
>
>>
>>
>>
>>Thanks
>>Damien
>
>
>
>
>
>________________________________
> From: dmitry kolesnikov <dmkolesnikov@REDACTED>
>To: Damienuk Davis <damienuk@REDACTED> 
>Cc: "erlang-questions@REDACTED" <erlang-questions@REDACTED> 
>Sent: Thursday, July 26, 2012 5:44 PM
>Subject: Re: [erlang-questions] run_erl and fsync
> 
>
>Does fsync block any VM scheduler? Or is it executed within async thread?
>Or none of them :-)
>
>
>I do not think that flush log is required for majority of Web systems. I believe this is a TeleCo legacy... Should it be configurable at least build time?
>
>Best Regards,
>Dmitry >-|-|-*>
>
>
>
>On 26.7.2012, at 9.10, Damienuk Davis <damienuk@REDACTED> wrote:
>
>
>Hi all,
>>
>>
>>In one of my projects I used run_erl to launch Erlang VM in daemon mode, rotate logs, etc.
>>But I discovered unexpected performance problem running production application with run_erl. Application caused high iowait.
>>After some investigation I found that run_erl does fsync on every log entry, causing disk IO load. Do we need fsync on every log entry.
>>Appreciate your valuable comments.
>>
>>
>>Thanks
>>Damien
>_______________________________________________
>>erlang-questions mailing list
>>erlang-questions@REDACTED
>>http://erlang.org/mailman/listinfo/erlang-questions
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20120726/c244412e/attachment.htm>