kernel panic (was Re: mailman, apache, chroot)

[Chad M Stewart <cms_(_at_)_balius_(_dot_)_com>]

After learning about mount_union I went ahead and tried the following

- removed mailman and python (they were installed in /var/www/usr/...
- installed mailman and python in their normal default locations
- used mount_union to mount /usr under /var/www/usr
- created the default mailman list

and then tried to access the listinfo page from a web browser, and poof 
machine gone.  I checked the console and found it sitting at ddb>


Here is the output from the reboot and subsequent attempt to reproduce 
the the panic, and it did it.  I'm running from the snapshot @ 
http://openbsd.secsup.org/snapshots/i386/ dated March 29th.


boot>
booting hd0a:/bsd: 4669908+843804 [58+213376+190976]=0x5a4f48
entry point at 0x100120

[ using 404824 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
        The Regents of the University of California.  All rights 
reserved.
Copyright (c) 1995-2004 OpenBSD. All rights reserved.  
http://www.OpenBSD.org

OpenBSD 3.5 (GENERIC) #34: Mon Mar 29 12:24:55 MST 2004
    deraadt_(_at_)_i386_(_dot_)_openbsd_(_dot_)_org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: AMD-K6(tm) 3D processor ("AuthenticAMD" 586-class) 451 MHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,MCE,CX8,PGE,MMX
real mem  = 469336064 (458336K)
avail mem = 428847104 (418796K)
using 4278 buffers containing 23568384 bytes (23016K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(06) BIOS, date 07/16/99, BIOS32 rev. 0 @ 
0xf0690
apm0 at bios0: Power Management spec V1.2 (BIOS mgmt disabled)
apm0: APM power management enable: unrecognized device ID (9)
apm0: APM engage (device 1): power management disabled (1)
apm0: AC on, battery charge unknown
pcibios0 at bios0: rev. 2.1 @ 0xf0000/0xd42
pcibios0: PCI IRQ Routing Table rev. 1.0 @ 0xf0ca0/160 (8 entries)
pcibios0: PCI Interrupt Router at 000:07:0 ("Acer Labs M1533 ISA" rev 
0x00)
pcibios0: PCI bus #1 is the last bus
bios0: ROM list: 0xc0000/0x10000
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Acer Labs M1541 PCI" rev 0x04
ppb0 at pci0 dev 1 function 0 "Acer Labs M5243 AGP/PCI-PCI" rev 0x04
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "Nvidia Riva TNT2 Ultra" rev 0x15
wsdisplay0 at vga1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
"Acer Labs M7101 Power Mgmt" rev 0x00 at pci0 dev 3 function 0 not 
configured
pcib0 at pci0 dev 7 function 0 "Acer Labs M1533 ISA" rev 0xc3
rl0 at pci0 dev 11 function 0 "Accton MPX 5030/5038" rev 0x10: irq 10 
address 00:04:e2:06:a8:14
rlphy0 at rl0 phy 0: RTL internal phy
rl1 at pci0 dev 12 function 0 "Accton MPX 5030/5038" rev 0x10: irq 11 
address 00:e0:29:62:03:b9
rlphy1 at rl1 phy 0: RTL internal phy
rl2 at pci0 dev 13 function 0 "Accton MPX 5030/5038" rev 0x10: irq 12 
address 00:e0:29:5b:30:84
rlphy2 at rl2 phy 0: RTL internal phy
pciide0 at pci0 dev 15 function 0 "Acer Labs M5229 UDMA IDE" rev 0xc1: 
DMA, channel 0 configured to compatibility, channel 1 configured to 
compatibility
wd0 at pciide0 channel 0 drive 0: <ST330630A>
wd0: 16-sector PIO, LBA, 29188MB, 59777640 sectors
wd1 at pciide0 channel 0 drive 1: <ST330630A>
wd1: 16-sector PIO, LBA, 29188MB, 59777640 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
wd1(pciide0:0:1): using PIO mode 4, Ultra-DMA mode 2
atapiscsi0 at pciide0 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <E-IDE, CD-ROM Max 50X, 6.40> SCSI0 
5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
sysbeep0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
lm0 at isa0 port 0x290/8: W83781D
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom0: console
pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
biomask c040 netmask dc40 ttymask dcc2
pctr: user-level cycle counter enabled
mtrr: K6-family MTRR support (2 registers)
dkcsum: wd0 matched BIOS disk 80
dkcsum: wd1 matched BIOS disk 81
root on wd0a
rootdev=0x0 rrootdev=0x300 rawdev=0x302
WARNING: / was not properly unmounted
Automatic boot in progress: starting file system checks.
/dev/rwd0a: 1570 files, 13423 used, 36908 free (12 frags, 4612 blocks, 
0.0% fragmentation)
/dev/rwd0a: MARKING FILE SYSTEM CLEAN
/dev/rwd0e: UNREF FILE  I=3  OWNER=root MODE=100600
/dev/rwd0e: SIZE=0 MTIME=Apr  8 13:00 2004  (CLEARED)
/dev/rwd0e: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED)
/dev/rwd0e: SUMMARY INFORMATION BAD (SALVAGED)
/dev/rwd0e: BLK(S) MISSING IN BIT MAPS (SALVAGED)
/dev/rwd0e: 1 files, 1 used, 250806 free (14 frags, 31349 blocks, 0.0% 
fragmentation)
/dev/rwd0e: MARKING FILE SYSTEM CLEAN
/dev/rwd1a: 2337 files, 973548 used, 13730852 free (3356 frags, 1715937 
blocks, 0.0% fragmentation)
/dev/rwd1a: MARKING FILE SYSTEM CLEAN
/dev/rwd0f: 61241 files, 173400 used, 12716576 free (328 frags, 1589531 
blocks, 0.0% fragmentation)
/dev/rwd0f: MARKING FILE SYSTEM CLEAN
/dev/rwd0d: LINK COUNT FILE I=61831  OWNER=root MODE=100644
/dev/rwd0d: SIZE=648 MTIME=Apr  8 13:02 2004  COUNT 2 SHOULD BE 1 
(ADJUSTED)
/dev/rwd0d: UNREF FILE  I=103047  OWNER=root MODE=100444
/dev/rwd0d: SIZE=2984 MTIME=Apr  7 10:38 2004  (RECONNECTED)
/dev/rwd0d: NO lost+found DIRECTORY (CREATED)
/dev/rwd0d: UNREF FILE  I=103055  OWNER=root MODE=100444
/dev/rwd0d: SIZE=3328 MTIME=Apr  7 14:51 2004  (RECONNECTED)
/dev/rwd0d: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED)
/dev/rwd0d: SUMMARY INFORMATION BAD (SALVAGED)
/dev/rwd0d: 35043 files, 468033 used, 538882 free (706 frags, 67272 
blocks, 0.1% fragmentation)
/dev/rwd0d: MARKING FILE SYSTEM CLEAN
setting tty flags
starting network
starting system logger
starting rpc daemons: portmap.
savecore: no core dump
checking quotas: done.
building ps databases: kvm dev.
clearing /tmp
starting pre-securelevel daemons:.
setting kernel security level: kern.securelevel: 0 -> 1
creating runtime link editor directory cache.
preserving editor files
starting network daemons: dhcpd sendmail httpd inetd sshd.
starting local daemons:.
standard daemons: cron.
Thu Apr  8 09:08:16 EDT 2004

OpenBSD/i386 (bia.amotken.com) (tty00)

login: panic: union: locking against myself
Stopped at      Debugger+0x4:   leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS 
PANIC!
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb> trace
Debugger(d0ef2280,200042,e7c33308,e75c7320,d0eabb60) at Debugger+0x4
panic(d048c700,e7d22978,e7bfb0c4,e7bfb184,e7c33308) at panic+0x87
union_lock(e7d22938,e7bfb184,e7ce8dc8,407552a2,200042) at 
union_lock+0x11b
VOP_LOCK(e7c33308,300042,e7ce8dc8,d02b251f) at VOP_LOCK+0x35
vn_lock(e7c33308,200002,e7ce8dc8,100) at vn_lock+0x3a
vn_rdwr(0,e7c33308,e7d22c00,34,0) at vn_rdwr+0x2c
elf32_read_from(e7ce8dc8,e7c33308,0,e7d22c00,34) at elf32_read_from+0x31
elf32_load_file(e7ce8dc8,d0edbc00,e7d22de0,d0ea2640,e7d22c84) at 
elf32_load_fil
e+0x125
exec_elf32_fixup(e7ce8dc8,e7d22de0,e75c7140,e7ce8dc8) at 
exec_elf32_fixup+0x4c
sys_execve(e7ce8dc8,e7d22f88,e7d22f7c,11,271cd960) at sys_execve+0x8c8
syscall() at syscall+0x21d
--- syscall (number 59) ---
0x7208f5d:
ddb> ps
   PID   PPID   PGRP    UID  S       FLAGS  WAIT       COMMAND
*19424   8922   8854     67  2    0x244104             listinfo
  6018  15339   6018      0  3      0x4086  kqread     tail
 15339  28099  15339    500  3      0x4086  pause      sh
 28099  30271  30271    500  3       0x184  select     sshd
 30271  11114  30271      0  3       0x184  netio      sshd
 13810   2750  13810    500  3      0x4086  ttyin      sh
  2750  31232  31232    500  3       0x184  select     sshd
 31232  11114  31232      0  3       0x184  netio      sshd
 25898      1  25898      0  3      0x4086  ttyin      getty
 22306      1  22306      0  3      0x4086  ttyin      getty
  3162      1   3162      0  3      0x4086  ttyin      getty
 29408      1  29408      0  3      0x4086  ttyin      getty
 11454      1  11454      0  3      0x4086  ttyin      getty
  4835      1   4835      0  3      0x4086  ttyin      getty
 21009      1  21009      0  3        0x84  select     cron
 26128   8854   8854     67  3       0x184  netcon     httpd
 23398   8854   8854     67  3       0x184  netcon     httpd
  6452   8854   8854     67  3       0x184  netcon     httpd
 12995   8854   8854     67  3       0x184  netcon     httpd
  8922   8854   8854     67  3       0x184  piperd     httpd
 11114      1  11114      0  3        0x84  select     sshd
 12108      1  12108      0  3       0x184  select     inetd
  8854      1   8854     67  3       0x184  select     httpd
 14800      1  14800      0  3     0x40184  select     sendmail
  9307      1   9307      0  3        0x84  poll       dhcpd
  7955      1   7955     28  3       0x184  poll       portmap
 23020  22704  22704     73  3       0x184  poll       syslogd
 22704      1  22704      0  3        0x84  netio      syslogd
     8      0      0      0  3    0x100204  apmev      apm0
     7      0      0      0  3    0x100204  crypto_wa  crypto
     6      0      0      0  3    0x100204  aiodoned   aiodoned
     5      0      0      0  3    0x100204  syncer     update
     4      0      0      0  3    0x100204  cleaner    cleaner
     3      0      0      0  3    0x100204  reaper     reaper
     2      0      0      0  3    0x100204  pgdaemon   pagedaemon
     1      0      1      0  3      0x4084  wait       init
     0     -1      0      0  3     0x80204  scheduler  swapper
ddb>


Am I using mount_union incorrectly?


-Chad-