[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

no idle CPU ... system hogging it all ...



Its been looking like this pretty much all day ... top shows nothing
major, and the drive looks reaonably quiet ... there is nothing in
messages to indicate a problem that I can see (even those enclosure
messages have been reasonably quiet) ...

What consumes SYS CPU?  Stuff like apache and jakarta-tomcat use up USER
CPU, correct?

neptune# iostat 5
      tty           aacd0            pass0            pass1             cpu
 tin tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
   1  125  0.00   0  0.00   0.00   0  0.00   0.00   0  0.00  13  0 48  0 39
   0  108 17.39  88  1.49   0.00   0  0.00   0.00   0  0.00  16  0 83  0  0
   0   23 17.79  23  0.39   0.00   0  0.00   0.00   0  0.00   6  0 92  0  1
   0  294 15.99  28  0.43   0.00   0  0.00   0.00   0  0.00   8  0 92  0  0
   0   35 17.56 156  2.67   0.00   0  0.00   0.00   0  0.00  18  0 82  0  0
   0 1048 15.86  68  1.05   0.00   0  0.00   0.00   0  0.00  22  0 77  0  0
   3  981 14.63  79  1.13   0.00   0  0.00   0.00   0  0.00  10  0 89  1  0
   0  353 14.85  77  1.12   0.00   0  0.00   0.00   0  0.00  18  0 81  0  0
   0  728 14.75  31  0.45   0.00   0  0.00   0.00   0  0.00   8  0 91  0  0
   1  669 13.93  11  0.15   0.00   0  0.00   0.00   0  0.00  11  0 89  0  0
   0   15 10.42   4  0.04   0.00   0  0.00   0.00   0  0.00   6  0 93  0  0
   0   15 13.42  31  0.40   0.00   0  0.00   0.00   0  0.00   3  0 97  0  0
   0   15  8.52  25  0.21   0.00   0  0.00   0.00   0  0.00   3  0 97  0  0
   5  686 15.71  64  0.98   0.00   0  0.00   0.00   0  0.00   9  0 91  0  0
   5  463  8.94  10  0.09   0.00   0  0.00   0.00   0  0.00   6  0 93  0  0
   5   24 10.67   3  0.03   0.00   0  0.00   0.00   0  0.00  14  0 85  1  0
   6   41 16.95 128  2.12   0.00   0  0.00   0.00   0  0.00   9  0 90  0  0
   8  274 16.14 114  1.80   0.00   0  0.00   0.00   0  0.00   7  0 93  0  0
   8   33 15.13   8  0.11   0.00   0  0.00   0.00   0  0.00  11  0 89  0  0
   3   20 15.65  33  0.51   0.00   0  0.00   0.00   0  0.00   4  0 96  0  0

doing a ps, the 'server processes' don't look to be consuming much CPU,
other then the vmdaemon/syncer (that is 18 and 9 hrs respectively, right?)

USER        PID %CPU %MEM   VSZ  RSS  TT  STAT STARTED      TIME COMMAND
root          0  0.0  0.0     0    0  ??  DLs  Thu07AM   0:45.80  (swapper)
root          1  0.0  0.0   552  196  ??  SLs  Thu07AM   0:03.37 /sbin/init --
root          2  0.0  0.0     0    0  ??  DL   Thu07AM   0:00.26  (aac0aif)
root          3  0.0  0.0     0    0  ??  DL   Thu07AM   0:19.50  (pagedaemon)
root          4  0.4  0.0     0    0  ??  DL   Thu07AM  18:27.81  (vmdaemon)
root          5  0.0  0.0     0    0  ??  DL   Thu07AM   0:01.62  (bufdaemon)
root          6  0.0  0.0     0    0  ??  DL   Thu07AM   0:01.03  (vnlru)
root          7  0.0  0.0     0    0  ??  DL   Thu07AM   9:03.81  (syncer)
root         28  0.0  0.0   212    0  ??  IWs  -         0:00.00 adjkerntz -i
root        134  0.0  0.0   952  528  ??  Ss   Thu10AM   0:06.17 /usr/sbin/sysl
root        138  0.0  0.1  6304 5608  ??  Ss   Thu10AM   0:51.81 /usr/sbin/name
daemon      140  0.0  0.0   988  436  ??  Ss   Thu10AM   0:02.75 /usr/sbin/port
root        142  0.0  0.0   572  356  ??  Ss   Thu10AM   0:02.76 mountd -r
root        144  0.0  0.0   368    0  ??  IWs  -         0:00.00 nfsd: master (
root        145  0.0  0.0   360    0  ??  IW   -         0:00.00 nfsd: server (
root        146  0.0  0.0   360    0  ??  IW   -         0:00.00 nfsd: server (
root        148  0.0  0.0   360    0  ??  IW   -         0:00.00 nfsd: server (
root        149  0.0  0.0   360    0  ??  IW   -         0:00.00 nfsd: server (
root        150  0.0  0.0 263088    0  ??  Ss   Thu10AM   0:02.73 rpc.statd
daemon      152  0.0  0.0   908    0  ??  IWs  -         0:00.00 rwhod
root        158  0.0  0.0  1024    0  ??  IWs  -         0:00.00 /usr/sbin/cron
root        160  0.0  0.0  2592  928  ??  Ss   Thu10AM   0:03.87 /usr/sbin/sshd

Right now, kvm and vnodes are looking like:

vm.kvm_free: 809500672 - debug.numvnodes: 414708 - debug.freevnodes: 95815

which is pretty much standard for my servers ... and there are only ~1200
processes running on this one, which is a Dual PIII with 4Gb of RAM ...

Starting commands seems to take a long time ... top takes forever, and
pstat -s shows:

neptune# time pstat -s
Device          1K-blocks     Used    Avail Capacity  Type
/dev/aacd0s1b     8388480   119916  8268564     1%    Interleaved
0.245u 3.670s 0:27.39 14.2%     16+218k 0+0io 0pf+0w

And not much swap is being used for the # of processes ..

What else should I be looking at? :(