Re: RAID questions...

[Otto Moerbeek <otto@drijf.net>]

On Sat, 1 Jan 2005, Chris Weiss wrote:

> I'm working on putting together an OpenBSD 3.6 box with a RAID5 array as a
> home media file server and have a few questions I couldn't find answers to on
> the various doc/howto pages. I was wondering if someone here could shed some
> insight... My hardware setup is 4 Seagate Barracuda 160gb IDE drives for the
> raid and a 40gb boot drive.
> 
> All of the examples I've seen for the raid.conf file show them using
> /dev/wd0e, /dev/wd1e, /dev/wd2e, etc. for devices, is there a particular
> reason they're not using the 'a' partitions? (IE /dev/wd0a, /dev/wd1a, etc..)
> I'll be using dedicated drives, so they'd only have one 'a' partition taking
> up the whole drive.

That should not be a problem.

> When doing the array verification ("raidctl -iv raid0"), I get about a hundred
> or so soft errors across all 4 devices.
> Jan  1 03:32:46 cartman /bsd: wd1a: DMA error reading fsbn 2800992 of
> 2800992-2801023 (wd1 bn 2801055; cn 2778 tn 13 sn 12), retrying
> Jan  1 03:32:46 cartman /bsd: wd1: soft error (corrected)
> There are some occasional DMA downgrades from UDMA mode 4 to UDMA 3. Is this
> at all acceptable? I remember reading somewhere that this can be due to
> interference on the hardware (cable crosstalk, etc.) and that it's just the
> nature of the beast.

Bad cabling is the prime candidate. I would try another set of high 
quality, 80 conductor cables.

> In my particular case, after I've gone through and created the raid array and
> verified it, I run "disklabel -E raid0" to set up the mount points and the
> array is only showing 24gb of available space (even after zeroing out the
> partition table). What could be causing this? There's some lines at the bottom
> of my dmesg (below), about disks being truncated - that sounds ominous, but
> I'm not sure what's causing it. Note that in the examples below, I've not yet
> set the disk label, so I'm OK with those error messages.

Carefully check the disklabels of your wd disks. It could be the sizes of 
your RAID partitions are not what you think they are.

	-Otto


> Again, I'd greatly appreciate any help on this matter... Thanks!
> 
> # disklabel -E raid0
> Jan  1 04:33:55 cartman /bsd: raid0: no disk label
> Jan  1 04:33:55 cartman /bsd: raid0: no disk label
> disklabel: Can't get bios geometry: Device not configured
> 
> Initial label editor (enter '?' for help at any prompt)
> > p G
> device: /dev/rraid0c
> type: RAID
> disk: raid
> label: fictitious
> bytes/sector: 512
> sectors/track: 96
> tracks/cylinder: 16
> sectors/cylinder: 1536
> cylinders: 32253
> total sectors: 49541760
> free sectors: 49541760
> rpm: 3600
> 
> 16 partitions:
> #        size   offset    fstype   [fsize bsize   cpg]
>  c:    23.6G     0.0G    unused        0     0         # (Cyl.    0 - 32253*)
>  i:     1.0G     1.0G    unused        0     0         # (Cyl. 1365*- 2687)
>  j:     1.0G     1.0G    unused        0     0         # (Cyl. 1365*- 2687)
>  k:     1.0G     1.0G    unused        0     0         # (Cyl. 1365*- 2687)
>  l:     1.0G     1.0G    unused        0     0         # (Cyl. 1365*- 2687)
> >
> 
> #raidctl -s raid0
> Jan  1 04:39:13 cartman /bsd: raid0: no disk label
> Jan  1 04:39:13 cartman /bsd: raid0: no disk label
> raid0 Components:
>           /dev/wd0a: optimal
>           /dev/wd1a: optimal
>           /dev/wd2a: optimal
>           /dev/wd3a: optimal
> No spares.
> Parity status: clean
> Reconstruction is 100% complete.
> Parity Re-write is 100% complete.
> Copyback is 100% complete.
> 
> #dmesg
> OpenBSD 3.6-stable (GENERIC_RAID) #0: Fri Dec 31 10:29:52 PST 2004
>    root@cartman.casadelove.com:/usr/src/sys/arch/i386/compile/GENERIC_RAID
> cpu0: AMD Athlon(tm) XP 2100+ ("AuthenticAMD" 686-class) 1.74 GHz
> cpu0:
> FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE
> real mem  = 536383488 (523812K)
> avail mem = 481951744 (470656K)
> using 4278 buffers containing 26923008 bytes (26292K) of memory
> mainbus0 (root)
> bios0 at mainbus0: AT/286+(8a) BIOS, date 11/06/02, BIOS32 rev. 0 @ 0xfb440
> apm0 at bios0: Power Management spec V1.2
> apm0: AC on, battery charge unknown
> pcibios0 at bios0: rev 2.1 @ 0xf0000/0xdf84
> pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdee0/160 (8 entries)
> pcibios0: PCI Exclusive IRQs: 5 10 11
> pcibios0: PCI Interrupt Router at 000:17:0 ("VIA VT82C596A ISA" rev 0x00)
> pcibios0: PCI bus #1 is the last bus
> bios0: ROM list: 0xc0000/0x9000 0xcc000/0x8000! 0xd4000/0x2800
> cpu0 at mainbus0
> pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
> pchb0 at pci0 dev 0 function 0 "VIA VT8366 PCI" rev 0x00
> ppb0 at pci0 dev 1 function 0 "VIA VT8366 AGP" rev 0x00
> pci1 at ppb0 bus 1
> vga1 at pci1 dev 0 function 0 "Matrox MGA G550 AGP" rev 0x01
> wsdisplay0 at vga1: console (80x25, vt100 emulation)
> wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
> re0 at pci0 dev 9 function 0 "Realtek 8169" rev 0x10: irq 5, address
> 00:09:5b:be:4d:5e
> rgephy0 at re0 phy 7: RTL8169S/8110S media interface, rev. 0 PHY
> re0: diagnostic failed, failed to receive packet in loopback mode
> re0: attach aborted due to hardware diag failure
> pciide0 at pci0 dev 10 function 0 "Promise PDC20269" rev 0x02: DMA, channel 0
> configured to native-PCI, channel 1 configured to native-PCI
> pciide0: using irq 10 for native-PCI interrupt
> wd0 at pciide0 channel 0 drive 0: <ST3160023A>
> wd0: 16-sector PIO, LBA48, 152627MB, 312581808 sectors
> wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
> wd1 at pciide0 channel 1 drive 0: <ST3160023A>
> wd1: 16-sector PIO, LBA48, 152627MB, 312581808 sectors
> wd1(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 5
> pciide1 at pci0 dev 12 function 0 "Promise PDC20271" rev 0x02: DMA, channel 0
> configured to native-PCI, channel 1 configured to native-PCI
> pciide1: using irq 10 for native-PCI interrupt
> wd2 at pciide1 channel 0 drive 0: <ST3160023A>
> wd2: 16-sector PIO, LBA48, 152627MB, 312581808 sectors
> wd2(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5
> wd3 at pciide1 channel 1 drive 0: <ST3160023A>
> wd3: 16-sector PIO, LBA48, 152627MB, 312581808 sectors
> wd3(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5
> pcib0 at pci0 dev 17 function 0 "VIA VT8366 ISA" rev 0x00
> pciide2 at pci0 dev 17 function 1 "VIA VT82C571 IDE" rev 0x06: ATA100, channel
> 0 configured to compatibility, channel 1 configured to compatibility
> wd4 at pciide2 channel 0 drive 0: <IC35L040AVVN07-0>
> wd4: 16-sector PIO, LBA, 39266MB, 80418240 sectors
> wd4(pciide2:0:0): using PIO mode 4, Ultra-DMA mode 5
> atapiscsi0 at pciide2 channel 1 drive 1
> scsibus0 at atapiscsi0: 2 targets
> cd0 at scsibus0 targ 0 lun 0: <LITE-ON, DVD SOHD-167T, 9S15> SCSI0 5/cdrom
> removable
> cd0(pciide2:1:1): using PIO mode 4, Ultra-DMA mode 2
> uhci0 at pci0 dev 17 function 2 "VIA VT83C572 USB" rev 0x1b: irq 11
> usb0 at uhci0: USB revision 1.0
> uhub0 at usb0
> uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub0: 2 ports with 2 removable, self powered
> uhci1 at pci0 dev 17 function 3 "VIA VT83C572 USB" rev 0x1b: irq 11
> usb1 at uhci1: USB revision 1.0
> uhub1 at usb1
> uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub1: 2 ports with 2 removable, self powered
> uhci2 at pci0 dev 17 function 4 "VIA VT83C572 USB" rev 0x1b: irq 11
> usb2 at uhci2: USB revision 1.0
> uhub2 at usb2
> uhub2: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
> uhub2: 2 ports with 2 removable, self powered
> auvia0 at pci0 dev 17 function 5 "VIA VT8233 AC97" rev 0x30: irq 10
> ac97: codec id 0x414c4710 (Avance Logic ALC200)
> ac97: codec features headphone, 18 bit DAC, 18 bit ADC, Realtek 3D
> audio0 at auvia0
> isa0 at pcib0
> isadma0 at isa0
> pckbc0 at isa0 port 0x60/5
> pckbd0 at pckbc0 (kbd slot)
> pckbc0: using irq 1 for kbd slot
> wskbd0 at pckbd0: console keyboard, using wsdisplay0
> pcppi0 at isa0 port 0x61
> midi0 at pcppi0: <PC speaker>
> sysbeep0 at pcppi0
> lpt0 at isa0 port 0x378/4 irq 7
> lm0 at isa0 port 0x290/8: W83697HF
> npx0 at isa0 port 0xf0/16: using exception 16
> pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
> pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
> fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
> biomask ff45 netmask ff65 ttymask ffe7
> pctr: user-level cycle counter enabled
> mtrr: Pentium Pro MTRR support
> Kernelized RAIDframe activated
> dkcsum: wd0 matched BIOS disk 81
> dkcsum: wd1 matched BIOS disk 82
> dkcsum: wd2 had no matching BIOS disk
> dkcsum: wd3 had no matching BIOS disk
> dkcsum: wd4 matched BIOS disk 80
> root on wd4a
> rootdev=0x40 rrootdev=0x340 rawdev=0x342
> uhub3 at uhub0 port 2
> uhub3: ATEN International product 0x7000, class 9/0, rev 1.10/1.00, addr 2
> uhub3: 4 ports with 4 removable, self powered
> uhidev0 at uhub3 port 1 configuration 1 interface 0
> uhidev0: Logitech Logitech USB Keyboard, rev 1.10/15.00, addr 3, iclass 3/1
> ukbd0 at uhidev0: 8 modifier keys, 6 key codes
> wskbd1 at ukbd0 mux 1
> wskbd1: connecting to wsdisplay0
> uhidev1 at uhub3 port 1 configuration 1 interface 1
> uhidev1: Logitech Logitech USB Keyboard, rev 1.10/15.00, addr 3, iclass 3/0
> uhidev1: 3 report ids
> uhid0 at uhidev1 reportid 1: input=2, output=0, feature=0
> uhid1 at uhidev1 reportid 2: input=1, output=0, feature=0
> ums0 at uhidev1 reportid 3: 0 buttons and Z dir.
> wsmouse0 at ums0 mux 0
> uhidev2 at uhub3 port 2 configuration 1 interface 0
> uhidev2: Logitech USB Mouse, rev 1.10/6.10, addr 4, iclass 3/1
> ums1 at uhidev2: 3 buttons and Z dir.
> wsmouse1 at ums1 mux 0
> raid0: Component /dev/wd0a being configured at row: 0 col: 0
>         Row: 0 Column: 0 Num Rows: 1 Num Columns: 4
>         Version: 2 Serial Number: 5150 Mod Counter: 48
>         Clean: Yes Status: 0
> raid0: Component /dev/wd1a being configured at row: 0 col: 1
>         Row: 0 Column: 1 Num Rows: 1 Num Columns: 4
>         Version: 2 Serial Number: 5150 Mod Counter: 48
>         Clean: Yes Status: 0
> raid0: Component /dev/wd2a being configured at row: 0 col: 2
>         Row: 0 Column: 2 Num Rows: 1 Num Columns: 4
>         Version: 2 Serial Number: 5150 Mod Counter: 48
>         Clean: Yes Status: 0
> raid0: Component /dev/wd3a being configured at row: 0 col: 3
>         Row: 0 Column: 3 Num Rows: 1 Num Columns: 4
>         Version: 2 Serial Number: 5150 Mod Counter: 48
>         Clean: Yes Status: 0
> WARNING: truncating disk at r 0 c 0 to 16513937 blocks.
> WARNING: truncating disk at r 0 c 1 to 16513937 blocks.
> raid0 (root)raid0: no disk label