[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Help with pf + pfsync + carp


I am evaluating OpenBSD as a potential replacement for some checkpoint fw equipment and am having some difficulty getting pf + pfsync + carp to reliably handle transparent failover. Im sure its just my lack of experience with the system and have most likely goofed some configuration parameter somewhere. My setup is simple as it is a test case in our lab. I have two firewalls configured to simulate failover by shutting down switchports randomly. The hardware consists of two Dual AMD64 3+GZ Processor with 4GB of Ram, LSI Controllers w/ RAID5 and Intel Quad Port Gigabit cards ( all overkill for firewall I know ) running the stock 3.6 UP i386 kernel and with both boxes configured like so ...

em0 - public
em1 - private
em5 - pfsync interconnect
carp0 - public
carp1 - public
pfsync0 - for state replication

fw1 interface configs ...

hostname.em0 = NONE
hostname.em1 = NONE
hostname.em5 = NONE
hostname.carp0 = NONE vhid 1 advskew 1 ...
hostname.carp1 = NONE vhid 2 advskew 1 ...
hostname.pfsync0 = up syncif em5

fw2 interface configs ...

hostname.em0 = NONE
hostname.em1 = NONE
hostname.em5 = NONE
hostname.carp0 = NONE vhid 1 advskew 100 ...
hostname.carp1 = NONE vhid 1 advskew 100 ...
hostname.pfsync0 = up syncif em5

my pf.conf on both firewalls look like this ...

nat on carp0 inet from !em0 -> carp0
pass quick on em0 inet proto carp
pass quick on em1 inet proto carp
pass quick on em5 inet
pass quick log-all inet keep state

The CARP failover portion works great. Its a bit confusing to configure since the man page is a bit non-verbose, but when an active interface is unplugged, the secondary firewalls carp interface transitions to a MASTER state and everything looks groovy. After the failover, pf passes traffic well when new connections are open by a client in the private network to the public network. However, if I have an open TCP connection, ( like an ftp download ) it will stall and eventually fail. Using tcpdump to monitor the em0 interface on the backup firewall reveals that the ftp tcp connection is alive and packets are being forward to it as soon as its carp interface transitions to MASTER. But when tcpdump is used to monitor the pflog0 interface, it does not show any matched rules. It seems to me that the NAT and rule state should be present on the backup firewall due to pfsync and the traffic should first get translated and then get passed to the client endpoint of the tcp stream. At first I though the NAT rule was to blame but the tcpdump output from em0 shows the packets addressed as ...

[ftp server address] >

... which looks like the packet should look post translation. I tried changing the rule ...

nat on carp0 inet from !em0 -> carp0

... to ...

nat on em0 inet from !em0 -> em0

... for grins but it did not appear to help. I can see that state info is being replicated to the backup firewall via pfsync. For example, when use pfctl -s state to view the ftp connections state entries ...

self tcp -> [ftp server address]:63703 ESTABLISHED:ESTABLISHED
self tcp [ftp server address]:63703 <- ESTABLISHED:ESTABLISHED

... shows up on both the primary and backup firewall before the failover happens. Should I be able to glean any NAT related state info from this commands output? Anyhow, when I try the same sort of test with a more interactive connection ( like ssh ), the connection will stall until the client generates a bit of outbound traffic. At that point all the inbound traffic ( like if you had started a ping on the remote machine via the ssh terminal session to generate constant screen activity ) pours in at once.

So, am I missing something fundamental or doing something obviously wrong? Any suggestions would be greatly appreciated. Please include me in responses as I do not subscribe to this list.

Matthew Grooms

Visit your host, monkey.org