[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: nfs stalls and speed issues

The data you did below shows that we can be excessively confident
that the machines are quick and are actually capable of speaking
to each other at full speed.

| All the machines performed almost identically.  Reading from the server
| was around 10MB/s, writing to the server was around 330KB/s.  Writing to
| the server with FTP was about 10MB/s.

so NFS reads from the server were 10MB/second.  And that's as
fast as you'll get (and pretty good really - little overhead,
fully laden packets).

WRITES were 330KB/second.  I might wonder about the far end's disk
speed, running around for different cyl groups, and writing dir data
(where softupdates are win), but for the ftp test.  One presumes pushing
few large files (which should be quicker than many small files).

nfsstat (as in "nfsstat -sw 1" for brief stats) will give you some
help you see which of the not-some-many rpc's it's spending its
time in.  There used to be a really useful alternative to nfsstate
that dug in pretty well.

The question for me is which SIDE is being bad (the client writing
or the server writing) and why?

(useful info summarized below with minor commentary, but mainly showing
that it's running full speed and NFS is the difference between fast and slow)

Quoting Duncan Martin (openbsd_(_at_)_codebunny_(_dot_)_org):
> On Fri, 20 Aug 2004 17:08:02 -0700
> Chuck Yerkes <chuck+obsd_(_at_)_2004_(_dot_)_snew_(_dot_)_com> wrote:

> > I'd be curious to see what ttcp shows (it's in ports, fairly good quick
> > bandwidth testing tool).

Full 100baseT speed.

>  - TCP
>  - Server receiving:
> drmatt: ttcp-t: 104857600 bytes in 11.04 real seconds = 9275.33 KB/sec +++
> lucas:  ttcp-t: 104857600 bytes in 8.99 real seconds = 11392.55 KB/sec +++
>  - Server transmitting
> drmatt: ttcp-t: 104857600 bytes in 9.98 real seconds = 10259.37 KB/sec +++
> lucas:  ttcp-t: 104857600 bytes in 9.56 real seconds = 10713.79 KB/sec +++
>  - UDP
>  - Server receiving
> drmatt: ttcp-t: 104857600 bytes in 10.42 real seconds = 9829.25 KB/sec +++
> lucas:  ttcp-t: 104857600 bytes in 12.38 real seconds = 8274.08 KB/sec +++
>  - Server transmitting
> drmatt: ttcp-t: 104857600 bytes in 12.40 real seconds = 8257.47 KB/sec +++
> lucas: ttcp-t:  104857600 bytes in 12.40 real seconds = 8261.15 KB/sec +++

> > As for writes, I'd also be curious how fast you can write to the
> > local machine. 
> [snip]
> 104857600 bytes transferred in 1.963 secs (53403303 bytes/sec)

(an allegeded 50MB/s (which is a single stream with almost no
head movement).  My experience is that 10-15MB/s is realistic with
most REAL data.)  But good enough.

> (this one has softdep)
> 104857600 bytes transferred in 4.047 secs (25909760 bytes/sec)
25MB/s.  Unsure why softdep would be SLOWER, but that may not
have been a factor anyhow.

> > What happens to writes when that NFS partition in a tmpfs partition?
> > (eg. eliminate the spindle from the game).
> I set up a mfs tmp partition on the server and exported it, but the client couldn't mount it correctly.  It shows in mount but not in df.  Trying to access the dir gives "input/output error".  I'm sure I've done that before, think I'm mising something obvious.

> Damn, I was afraid it might not work.  Not sure why it wouldn't, but...

> Other tests:
> ftp
> ---
> (100MB file)
> drmatt: 9.25 MB/s
> lucas: 11.13 MB/s
Full 100baseT

> scp
> ---
> (scp /tmp/100meg caleb:/tmp/out)
> drmatt: 3.2MB/s
> lucas: 5.0MB/s
And math gets in the way (the only reason I keep rsh around
on a secured network).