Re: my multipath routing questions...

[andrew fresh <andrew_(_at_)_mad-techies_(_dot_)_org>]

On Fri, Dec 02, 2005 at 04:08:13PM -0700, andrew fresh wrote:
> I am getting 3 different DDB's.  Mostly "kernel: page fault trap,
> code=0" and "Panic: rtfree 2".  I have also gotten some "Panic: sbdrop",
> but not since I got the serial console attached.  When I got the sbdrop,
> trace showed calls to pf_* but I did not write it down as I thought I
> would see it again with the
> serial console.
> 
> It seems to DDB anywhere from 5 minutes to 90 minutes after a reboot.
> Once I got 6.5 hours, but mostly closer to 10 minutes.  The only thing
> that seems to make a difference is disabling pf, I am up 17.5 hours now
> with pf disabled.
> 
> DMESG and the trace/ps from the DDBs are below.

They are actually available in the archives so as not to waste
bandwidth.
http://marc.theaimsgroup.com/?l=openbsd-misc&m=113356535818065&w=2

> > > or something with 'route-to' in pf?

It appears that it is the route-to that is causing it to crash.  

(czn is the interface group of all of the T1s) 

This works:
  set skip on { lo }
  scrub in all fragment reassemble
  block drop log all
  pass on czn all
  pass on fxp0 all

as does this (although this makes data transfers VERY slow):
  set skip on { lo }
  block drop log all
  pass on czn all
  pass in on san0 reply-to (san0 216.190.36.133) inet all keep state
  pass in on san1 reply-to (san1 216.190.36.137) inet all keep state
  pass in on san2 reply-to (san2 216.190.36.141) inet all keep state
  pass in on san3 reply-to (san3 216.190.36.145) inet all keep state
  pass on fxp0 all

But it crashes when my rules are this:
  set skip on { lo }
  block drop log all
  pass on czn all
  pass out on czn route-to { \
    (san0 216.190.36.133),   \
    (san1 216.190.36.137),   \
    (san2 216.190.36.141),   \
    (san3 216.190.36.145)    \
  } round-robin inet all
  pass on fxp0 all

or this:
  set skip on { lo }
  scrub in all fragment reassemble
  block drop log all
  pass on czn all
  pass out on san0 route-to { \
    (san0 216.190.36.133),    \
    (san1 216.190.36.137),    \
    (san2 216.190.36.141),    \
    (san3 216.190.36.145)     \
  } round-robin inet all
  pass out on san1 route-to { \
    (san0 216.190.36.133),    \
    (san1 216.190.36.137),    \
    (san2 216.190.36.141),    \
    (san3 216.190.36.145)     \
  } round-robin inet all
  pass out on san2 route-to { \
    (san0 216.190.36.133),    \
    (san1 216.190.36.137),    \
    (san2 216.190.36.141),    \
    (san3 216.190.36.145)     \
  } round-robin inet all
  pass out on san3 route-to { \
    (san0 216.190.36.133),    \
    (san1 216.190.36.137),    \
    (san2 216.190.36.141),    \
    (san3 216.190.36.145)     \
  } round-robin inet all
  pass on fxp0 all

Is there something that I am obviously doing wrong?

Is there some way I can load balance outbound traffic across these 4
interfaces in a different wat?

Should I submit this as a bug report?

l8rZ,
-- 
andrew - ICQ# 253198 - JID: afresh1_(_at_)_jabber_(_dot_)_org
     Proud member: http://www.mad-techies.org

BOFH excuse of the day: Sticky bits on disk.