[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: nic hang, irq's, memory space, right way?



Ben Goren wrote:
> On 2004 Oct 10, at 12:51 PM, Nick Holland wrote:
> 
>> For the record, I recently set up a machine with 20 dc(4) ports (five
>> 4-port cards).
> 
> Dang. What is this, some sort of managed switch? Rackmount, whitebox...?

guess a few people are wondering about this. :)
No, no super app, though, I'll admit I did have seven ports on my home
firewall at one point (quad port NIC,  on-board port,  wireless,  coax
port for a "legacy" segment), though I never got around to using all the
ports.

It was an artificial test of a problem someone reported with a large
number of gig NICs.  Well, I had a five PCI slot machine (Dell
desktop!), and a pile of four port dc(4) cards I've been hoarding, so I
stuffed this machine full.  While my test proved to be non-productive
(dc(4)'s didn't show the problem), it was resolved for 3.6, and you can
now put metric snootloads of gig NICs in an OpenBSD box.  With bgpd, I'd
imagine that putting a lot of NICs in a machine and using it as a
massive router is becoming a likely application...

Observations:
1) NICs are not all created equal.  dc(4)s are too well mannered to
cause problems, at least of the type I was after.  Other than a few
gasps of astonishment from a few developers, the 20 ports didn't
accomplish anything.  Ok, a gasp of astonishment from some of these
people WAS worth it, at least over something other than me saying
something stupid. 8-)
2) While it looks logical in retrospect, it didn't strike me as a likely
problem point: the more NICs, the larger the memory consumption.  32M is
fine for two or three NICs, but put 20 in the machine and you will cause
a panic around dc8 or dc9.  As I recall, 64M didn't get me all the way
through 20 NICs, but don't quote me on that one (might have been only
48M.  And yes, I managed to put 48M in a machine with five PCI slots
(32M DIMM and 16M DIMM.  And yes, 16M DIMMs are a crime against
humanity)).  One chews through a lot more memory per NIC than I would
have guessed.
3) Identifying the ports is anything but trivial, and as they were on
multiple PCI busses, as I recall, far from intuitive...esp. when the
machine is a "sideways slot desktop"...one more way for the dyslexic
mind to get confused. :)

btw: I since found a machine with six PCI slots, and another with
probably even more..and I still had a few quad dc(4) cards in the pile.
 Should max that machine out and post the dmesg, it's impressive as all
heck: quad PPro, 2G RAM...  Unfortunately, it's only going to get
powered up for tests, running 24hrs a day, rough calcs indicate it would
cost $700 a year in electricity, IGNORING I'd have to upgrade my
basement AC (thing was rated for a heat output of 4000 BTUs, my basement
AC is only 5000 BTUs!).

Nick.
-- 
http://www.holland-consulting.net



Visit your host, monkey.org