[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

NFS, Performance, and Reliability


I posted this to misc@, but got no reply on this whole thing
for some days now. Below I "cross-post" to this list the last
of a series of other messages that state _massive_ problems
using NFS in a LAN. I don't know if it's to heavy, too lame,
or if it's only not interesting, or if I'm not the guy to
talk to, but I would like to _not_ need to pull out of
OpenBSD for my server(s), but can't get it sorted out
on my own now. I cut the personal part about my efforts
and no answers and such.

Thank you for listening,


----- Forwarded message from Toni Mueller <openbsd-misc_(_at_)_oeko_(_dot_)_net> -----

Date: Fri, 28 Jul 2000 16:13:18 +0200
From: Toni Mueller <openbsd-misc_(_at_)_oeko_(_dot_)_net>
To: misc_(_at_)_openbsd_(_dot_)_org
Subject: NFS, Performance, and Reliability
Reply-To: openbsd-misc_(_at_)_oeko_(_dot_)_net

[ ... cut ... ]

When talking about OpenBSD below, I always mean 2.7 on i386.
The NFS server is a plain CD install, and the Client is a
-stable from 18.7.2000. If anyone needs more info, I am
happy to provide the needed bits.

---- snip -----
Some of the results of my experimenting were the following:

- Linux 2.0.x probably makes a bad NFS client for an OpenBSD server.
- Linux 2.0.x probably makes an acceptable NFS server for an OpenBSD
  client, at least performance-wise.
- Linux 2.2.x (x >= 15) makes a medium NFS client for an OpenBSD server.
- OpenBSD 2.7 makes a better, but also medium NFS client for an
  OpenBSD server, with the point being that the client portion of
  NFS appears to be quite unstable to me.

To the latter I had several machine hangs using NFS I had not under
Linux. These were reproducible in that I could take some actions
that would crash the OpenBSD client. The server ran stable under
all circumstances.* The client was from -current as of 7/18/2000,
the server from CD. I'll copy in what I found:

* meaning, the server didn't crash or exhibit obvious problems.
  I don't currently believe that it worked ok, though, but see

------ start of excerpt -------------------------------
As far as I currently see it's like this (NIC: Intel Pro100 Mgmt where
not specified otherwise):
FTP either way: 7.5-9.5 MB/s            as printed out by FTP clients
NFS read OpenBSD->OpenBSD: 10MB/s
NFS read Linux->OpenBSD: 2-3MB/s	kernel 2.0.38, but new NFS server
NFS read OpenBSD->Linux: 7MB/s
NFS write Linux->OpenBSD: 4MB/s         kernel 2.2.15
NFS write Linux->OpenBSD: around 25KB/s kernel 2.0.38, 3com905TX
NFS write OpenBSD->OpenBSD: 600KB/s     (although the write rate on
					 the server hoovers around 5MB/s)

Most performance numbers are done with iostat -w1 on the server.

I'm not sure what that means, but it means at least that I can
feel a tiny relief that the server works somewhat as expected ...

* comparing what iostat -w1 and eg. df have to say on the disk write
  rate (df of course called often, eg in a loop) makes for a huge
  difference, the df number being a lot lower than expected.

On a side note, when doing the following on the OpenBSD client,
the client _hangs_ afterwards:

time /bin/tar cf - /usr |/bin/dd of=/mnt/test/w6.usr.tar obs=8192
(ie, starting that NFS operation and then hitting Control C shortly
afterwards, say, within a second). Repeated in similar fashions
several times now... OUCH!

So far I found only one way around it (ie, the following does
_NOT_ work:
mount -t nfs -o intr server:/export /import
!!! - so far for the OpenBSD software quality I was after =8*|   )

mount_nfs -i -T server:/export /import

------ end of excerpt -------------------------------

* I must cancel the statement above. It provided a tiny relief, and
  I could kill that tar processes with no machine hang, BUT the
  following made the job in question hang nonetheless.

On the client's console I see:

"NFS server not responding"...

after doing

mount_nfs -i -T  .... /mnt

cp /cdrom/* /mnt

* Since the individual CD reads were slow, and especially the
  CD in the server is slow, I started similar jobs on two
  NFS client machines, too, to have three CDs filling one
  directory that is exported to the two clients. Since the
  files on that CDs are disjoint apart from a few directories,
  I considered this to be no problem.

The job is in "IE+" state for about one hour now. Typing on that
shell yields "^C" or what key I press.

* It appears as if the "intr" option is not honoured anywhere.
  kill -9 doesn't help either.

Sorry to bug you, but I don't exactly know how to provoke these
errors and/or describe these errors in a way that would get them
heard. I will certainly also ask the list whether anyone maybe
actually uses the NFS stuff, but so far I can only say that a
lowly two-year-old Linux server and/or client hold up very well
against this. Bad performance, no locking, high CPU load and all,
but it didn't crash as far as I tried... It's a real shame!

* AND it provided some degree of data consistency if you could
  assure exclusive access to the individual files, eg. by
  convention ("I work with this file and you work with that one").

One other thing: I have a hard time getting a usable transfer
rate (say, at floppy speeds) from a locally installed SCSI CD
hanging off an Adaptec-2940u2w. When initially copying the
contents of an IDE disk (~ 3GB) to the RAID, it simply made
"swoosh" and was done in the blink of an eye. How to diagnose?

And yet one other thing: When I run iostat -w1 and also
while true ; do df ; sleep 1; done, I see a large difference
between these two meters. iostat gives _far_ higher numbers!

Any help is greatly appreciated!

Best Regards,

----- End forwarded message -----