bgpd nexthop/gateway weirdness (was: Re: bgpd: "send_rtmsg: action foo, prefix bar/baz: (some error)")

[Arvid Grøtting <arvidg_(_at_)_netfonds_(_dot_)_no>]

Henning Brauer <lists-openbsd_(_at_)_bsws_(_dot_)_de> writes:

> I don't fully remember what was in 3.5, but nexthop handling in 
> -current should be 100% sane, we haven't had changes in the nexthop 
> verification code since quite some time now.

Well, I'm afraid the nexthop verification doesn't look quite sane to
me.

If I boot without a default route, things are fine: The selected
gateways match the actual shortest path to the nexthop gateway once
internal routing ensures that the nexthop gateway becomes reachable.

If, however, I have a default route in place (in /etc/mygate) when
booting, bgpd may[1] still think it suitable to use default route to
reach the bgp nexthop.

When internal routes to the nexthops do show up, the bgpd fib is not
updated with the corresponding gateway address.  Neither is the kernel
routing table.

This applies, of course, to nexthops reachable through other ibgp
peers.  The ebgp peer is directly reachable; I don't have any ebgp
multihop peers at the moment.

FWIW, the internal routing is at the moment done with an old version
of gated running ospf on all interfaces.  The symptoms disappear if
the ospf routing deamon can run for a minute before bgp is started.
Oh, and if memory serves me right, zebra too likes to have ospfd
started a bit of time[2] before its bgpd.

So, I have several ways to work around this problem as long as I keep
my internal routing stable, but I'm a bit curious about what would
happen if the internal route to one of the nexthops were to change.
(I can't test that at the moment; sorry about that.)


[1] I'm not sure that this is 100% reproducible; this time, when
testing, it happened to routes through one ibgp peer, but not those
through the other ibgp peer.
[2] enough to get the ospf routes set up
-- 

Arvid