[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

strange problems : ipsec tunnel



Folks

	I am having some strange problems getting nfs mounts to work from one side
of an ipsec tunnel to the other.

What seems to be happening is that the mounting of filesystems works ok but
if I try to ls a large directory then the whole terminal session on the
client hangs until I kill -9 ls from another session.

After spending some time playing with tcp dump I have an idea what is
happening but I am getting nowhere figuring out why it is happening.

What I have found seems to be that not all of the packets are getting
through (specifically fragmented packets ... ls of large directory).
Having tracked the packet path through their whole route it now seems that
something is being lost in the ipsec tunnel.

My setup is as follows

My private net := 10.128.1.0/24
My nfs client (psiren - Redhat linux 6.1) := 10.128.1.8
Internal ip of ipsec local :=10.128.1.2
External ip of ipsec local (hilly - OpenBSD 2.6 snapshot) :=63.a.b.c/29

Office private net = 10.0.0.0/9
Nfs server (kryten - Redhat 6.1) = 10.p.q.r

There is a cisco router between 10 net and dmz net
The router is configured with a static route for 10.128.0.0/9 via the
office ipsec machine.

Ipsec Office machine (queeg - same snapshot and kernel as hilly) is at net
address d.m.z.x

There two vpn machines are configured with flows between the 10.128.1/24
net and the d.m.z/24 and 10.0/9 net.

On queeg (ipsec - office end)

	"tcpdump host kryten " shows

01:00:07.687654 10.128.1.8.12f019d1 > kryten.talarian.com.nfs: 140 getattr
[|nfs]
01:00:07.690309 kryten.talarian.com.nfs > 10.128.1.8.12f019d1: reply ok 96
getattr [|nfs]
01:00:07.736345 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:07.739730 kryten.talarian.com > 10.128.1.8: (frag 21228:888_(_at_)_2960)
01:00:07.741270 kryten.talarian.com > 10.128.1.8: (frag 21228:1480_(_at_)_1480+)
01:00:07.742073 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21228:1480_(_at_)_0+)
01:00:08.433535 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:08.436904 kryten.talarian.com > 10.128.1.8: (frag 21229:888_(_at_)_2960)
01:00:08.438186 kryten.talarian.com > 10.128.1.8: (frag 21229:1480_(_at_)_1480+)
01:00:08.439437 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21229:1480_(_at_)_0+)
01:00:09.832946 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:09.836322 kryten.talarian.com > 10.128.1.8: (frag 21230:888_(_at_)_2960)
01:00:09.837605 kryten.talarian.com > 10.128.1.8: (frag 21230:1480_(_at_)_1480+)
01:00:09.838856 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21230:1480_(_at_)_0+)
01:00:12.634103 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:12.637477 kryten.talarian.com > 10.128.1.8: (frag 21231:888_(_at_)_2960)
01:00:12.638761 kryten.talarian.com > 10.128.1.8: (frag 21231:1480_(_at_)_1480+)
01:00:12.640015 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21231:1480_(_at_)_0+)
01:00:18.238531 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:18.241902 kryten.talarian.com > 10.128.1.8: (frag 21233:888_(_at_)_2960)
01:00:18.243185 kryten.talarian.com > 10.128.1.8: (frag 21233:1480_(_at_)_1480+)



"tcpdump proto esp" shows 

01:00:07.687455 esp hilly.talarian.com > queeg.talarian.com spi 0x00001000
seq 4713 len 204
01:00:07.690458 esp queeg.talarian.com > hilly.talarian.com spi 0x00001001
seq 9765 len 156
01:00:07.698945 esp hilly.talarian.com > queeg.talarian.com spi 0x00001000
seq 4714 len 100
01:00:07.702020 esp queeg.talarian.com > hilly.talarian.com spi 0x00001001
seq 9766 len 196
01:00:07.736174 esp hilly.talarian.com > queeg.talarian.com spi 0x00001000
seq 4715 len 212
01:00:07.740241 esp queeg.talarian.com > hilly.talarian.com spi 0x00001001
seq 9767 len 940
01:00:07.742063 queeg.talarian.com > hilly.talarian.com: (frag
23405:1480_(_at_)_1024+)
01:00:07.742080 queeg.talarian.com > hilly.talarian.com: (frag
23405:52_(_at_)_2504)
01:00:07.742872 esp queeg.talarian.com > hilly.talarian.com spi 0x00001001
seq 9769 len 1480 (frag 22523:1480_(_at_)_0+)
01:00:07.742881 queeg.talarian.com > hilly.talarian.com: (frag
22523:52_(_at_)_1480)
01:00:08.433351 esp hilly.talarian.com > queeg.talarian.com spi 0x00001000
seq 4716 len 212
01:00:08.437416 esp queeg.talarian.com > hilly.talarian.com spi 0x00001001
seq 9770 len 940
01:00:08.438973 queeg.talarian.com > hilly.talarian.com: (frag
18684:1480_(_at_)_1024+)
01:00:08.438983 queeg.talarian.com > hilly.talarian.com: (frag
18684:52_(_at_)_2504)
01:00:08.440228 esp queeg.talarian.com > hilly.talarian.com spi 0x00001001
seq 9772 len 1480 (frag 28383:1480_(_at_)_0+)
01:00:08.440238 queeg.talarian.com > hilly.talarian.com: (frag
28383:52_(_at_)_1480)
01:00:08.527589 esp hilly.talarian.com > queeg.talarian.com spi 0x00001000
seq 4717 len 100
01:00:08.564577 esp hilly.talarian.com > queeg.talarian.com spi 0x00001000
seq 4718 len 100


on hilly "tcpdump -i xl1 host kryten" shows


00:59:59.195181 10.128.1.8.12f019d1 > kryten.talarian.com.nfs: 140 getattr
[|nfs]
00:59:59.243281 kryten.talarian.com.nfs > 10.128.1.8.12f019d1: reply ok 96
getattr [|nfs]
00:59:59.244409 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
00:59:59.306872 kryten.talarian.com > 10.128.1.8: (frag 21228:888_(_at_)_2960)
00:59:59.331305 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21228:1480_(_at_)_0+)
00:59:59.941931 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:00.004243 kryten.talarian.com > 10.128.1.8: (frag 21229:888_(_at_)_2960)
01:00:00.027931 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21229:1480_(_at_)_0+)
01:00:01.342118 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:01.403672 kryten.talarian.com > 10.128.1.8: (frag 21230:888_(_at_)_2960)
01:00:01.426621 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21230:1480_(_at_)_0+)
01:00:04.142526 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:04.204975 kryten.talarian.com > 10.128.1.8: (frag 21231:888_(_at_)_2960)
01:00:04.228170 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21231:1480_(_at_)_0+)
01:00:09.743367 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:09.807832 kryten.talarian.com > 10.128.1.8: (frag 21233:888_(_at_)_2960)
01:00:09.831518 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21233:1480_(_at_)_0+)
01:00:11.143519 10.128.1.8.13f019d1 > kryten.talarian.com.nfs: 148 readdir
[|nfs]
01:00:11.207740 kryten.talarian.com > 10.128.1.8: (frag 21236:888_(_at_)_2960)
01:00:11.230698 kryten.talarian.com.nfs > 10.128.1.8.13f019d1: reply ok
1472 readdir [|nfs] (frag 21236:1480_(_at_)_0+)


It looks to me that the tunnel (or at least the tunnel or one of the
machines at the ends) is only passing through the first "installment" of
the fragmented packet.

I have no idea where to go next.  Can anybody make any suggestions.

Thanks 

	Peter

PS. I didnt state which 2.6 snapshot as I am not totally certain (but I am
sure I installed it mid to late October).  If someone can tell me an easy
way to find out what the snapshot date was I can provide that info too.