pf modulate state decreases throughput

[Josh Carroll <josh_(_dot_)_carroll_(_at_)_gmail_(_dot_)_com>]

Hello,

I recently installed OpenBSD 3.5 on a Soekris net4801 and I'm
currently using it as an ethernet bridge and pf to filter on said
bridge.

The problem I am encountering is with "modulate state". When I use
"keep state" my throughput on my 6Mbit DSL connection is normal -
roughly 620KB/s or so. However, when I change "keep state" to
"modulate state" for one particular rule, the throughput drops to
500-550KB/s, jumping up and down, assumedly due to retransmits and/or
duplicate ACKs (which I noticed with ethereal on the Linux box doing
the downloading behind the bridge). This speed drop happens whether
I'm downloading on the Linux box or the Windows XP box being nat'd on
the OpenBSD box or the Linux box.

The pf rule in question is:

pass out quick on $wan proto { tcp, udp, icmp } from any to any \
        flags S/SA keep state

Below you will find my full pf.conf (the last rule is the one in question).

My network topology is the following:

DSL--(sis0)Soekris(sis1)--(eth0)Linux(eth1)--switch--WinXP

My question is whether the state modulation code has a problem here,
or does the soekris 4801 just not have enough horsepower to do state
modulation? It is my understanding that "modulate state" simply
generates a strong initial sequence number, which implies the overhead
of the randomization is supposed to happen for only the first packet
to match the rule. I should note that the load on the box does not
increase abnormally or differently when using modulate state versus
keep state.

Additionally, if I look at a tcp sequence number graph, I see the
sequence linearity "break" every 3 seconds or so, so it's happening on
a regular interval.

The "good" sequence graph looks something like this:
   /
  /
 /
/

whereas the sequence graph with modulate state looks like this:

        _/
     _/
  _/
/

I had a friend test keep state versus modulate state on his lan and
his throughput dropped from 5MB/s to 3.8MB/s, on a Pentium 233MMX box.

Thanks in advance,
Josh

pf.conf follows:
#############################
# interfaces
wan = "sis0"
lan = "sis1"
wlan = "sis2"
                                                  
# ports setup
# allowed ports to the linux box
allowed_linux_tcp = "{ 22, 25, 53, 80, 6667, 2222, 113, 33333:65535 }"
allowed_linux_udp = "{ 53 }"
always_block_ports_tcp = "{ 12345, 31337, 49152, 65506 }"

# linux IPs
table <linux> persist { a.b.c.2, a.b.c.4, a.b.c.5, a.b.c.6 }
table <windows> persist { a.b.c.3 }
table <block_ips> persist file "/etc/pf/block_ips"
table <bogons> persist file "/etc/pf/bogons"

# normalization
scrub in all fragment reassemble
scrub out all fragment reassemble

# black list of ips and bogons
block in log quick on $wan from <block_ips> to any
block out log quick on $wan from any to <block_ips>
block in log quick on $wan from <bogons> to any
block out log quick on $wan from any to <bogons>

## default policy - block
block out log on $wan all
block in log on $wan all
block return-rst out log on $wan proto tcp all
block return-rst in log on $wan proto tcp all
block return-icmp out log on $wan proto udp all
block return-icmp in log on $wan proto udp all
pass quick on lo0 inet label "local traffic"

antispoof for { lo0, $wan, $lan, $wlan }

# nmap blocking
block in log quick on $wan inet proto tcp from any to any flags FUP/FUP
block in log quick on $wan inet proto tcp from any to any flags SF/SFRA
block in log quick on $wan inet proto tcp from any to any flags /SFRA

# common trojan ports/etc that are > 33333
block in log quick on $wan proto tcp from any to any \
    port $always_block_ports_tcp

pass in on $wan inet proto icmp from any to <linux>

pass in quick on $wan proto tcp from any to <linux> port $allowed_linux_tcp \
    flags S/SA keep state

pass in quick on $wan proto udp from any to <linux> port $allowed_linux_udp \
    keep state

block in log quick on $wan from any to <windows>

pass out quick on $wan proto { tcp, udp, icmp } from any to any \
        flags S/SA keep state