odd problem with distributed erlang system

[Raimo Niskanen]

What was annoying in this case was that even though I had:
-dumped amd and used aufofs instead
-forced rwsize an wsize to 8192
-used NFSv3
-mounted "hard,intr"
the kernel or the NFS filesystem screwed up the semantics of a write 
operation (writev in my opinion i just one write operation despite the 
write buffer being scattered) by not gathering the write data, doing 
1024 synchronous write operations instead.

The "sync" mount option is the default and the "safe bet". It works just 
fine with on Solaris, so I would say NFS on Linux is still sometimes a 
weak spot. This time it punished the Erlang emulator file driver that 
just wanted to be efficient.

Yes, tcpdump is now a friend of mine :-)

/ Raimo Niskanen, Erlang/OTP



Miguel Barreiro Paz wrote:
>>It was on Debian 3.0 (Linux 2.4.18 kernel) with OTP R9B and an
>>automounted NFS volume. I had lots of problems to make the automounter
>>use the mount options I wanted, partly because the automounter I used
>>(amd) seems to not use the mount command, but a library API, so when I
>>was done I had changed automounters, written some weird scripts, etc.
> 
> 
> NFS on Linux has traditionally been a weak spot. It's much much better
> today. My experience on nfs on linux:
> 
> - dump amd and use autofs instead
> - force rsize and wsize to 8192 if autofs doesn't do it for you
> - use NFSv3 (yes, it's marked "experimental", but has been in use here for
> a _long_ time without worry. Maybe we just were lucky)
> - mount "hard" or "hard,intr"
> - NFSv3 over TCP is still a little bit unreliable in my humble experience.
> - Async mounts are perfectly OK in a LAN for most purposes.
> - use jumbo frames if you're on 1000Base-* and hardware allows (1 ether
> frame per 8KB NFS datagram is nice)
> - tcpdump is your friend, as you already know :-)
> 
> 
>>The solution was to not mount the volume synchronously, i.e I added the
>>mount options: "async,actimeou=0". The "actimeout=0" flag sets attribute
>>caching timeout to 0, which is pretty syncronous, and will do for me.
> 
> 
> Not quite the same as "sync" but good enough unless someone runs a mail
> spool or something similarly insane on that fs.
> 
> 
>