trailing a growing file

[Andrae Muys]

Shawn Pearce wrote:

> tail -f just does the same thing you are doing by reading the log,
> hitting EOF, and waiting a short while.
> 
> Classic UNIX/POSIX doesn't have a way to get new data from a file, or
> to find out when a file/directory is modified - applications just have
> to poll.
> 
> FreeBSD and IRIX (that I know of) have methods where an application
> can open a directory or file, and tell the OS it is interested in
> being notified when a change is made.  Then you read from it and
> carry on.  Under IRIX this is called FAM (File Activity Monitor).  I
> don't know what FreeBSD calls it; and to my knowledge Linux does not
> have support for this.
> 
> If the log writer is another Erlang node I would suggest just sending
> the log data to the target node, as well as write it to disk.  If the
> log writer is an external process, well, you are stuck.  If it is a
> syslogd maintained file, perhaps you could put an Erlang node into the
> list of targets syslogd will write the logs to.
> 
> If the log writer is anything else, you'll need to either patch the
> application (example would be with Apache build or reuse a mod_* that
> can send the logs over the network to an Erlang node), or just poll
> the log file...
> 
> If it was me, I'd be polling from within Erlang using a raw file (so
> no extra process is started) rather than using a port to tail -f.
> 
> 

Other options include:

Using a FIFO instead of a raw file, and using poll/select.
Using dnotify under Linux to achieve a similar effect as IRIX's FAM.

Note if the logfile is generated by syslogd, you can configure it to log 
to both a FIFO and a regular file.

Andrae

-- 
  Andrae Muys    <andrae.muys@REDACTED>
  Engineer       Braintree Communications
   "Now, allowing captured continuations to be inspected and altered at
    runtime (including binding mutation, complete rebinding of scopes,
    and call tree mutation)...  *that* is really evil.  And, I should
    point out, quite useful."        - Dan Sugalski