Re: ipsec scalability?

["Daniel de Young" <daniel@velvetsea.com>]

> On Tue, 1 Jul 2003, Daniel de Young wrote:
> ...
>> A quick search of google results turned up little that seemed
>> relevant.
>>
>> What are list members experiences with running dozens ++ of tunnels.
>> Any hard limits or known/documented limits based on ram/proc etc.?
>
> While I have not run dozens++ myself in any production environment (only
> up to 10-15 there), I have run many more than that during testing. I
> don't quite remember, this was 2-3 years ago, maybe 5.000 SAs?
>
> One of the design specs for isakmpd, and thus the relevant ipsec parts
> of the OpenBSD kernel, was the ability to manage somehing on the order
> of 20.000 SAs. Niklas(@openbsd.org) did some testing with such numbers.
>
> The main limiting factor is kernel RAM (mostly for the keying data),
> unless your lifetimes are small, in which case CPU becomes an issue.
> Also, which that many SAs to renegotiate, you want to distribute the
> renegotiations over time to avoid "peaks".
>
> For 20.000 SAs, I imagine you'd need something like 15-20MB RAM
> available to the kernel. If you renegotiate them once every hour, it
> means approx 2-3 negotiations per second (SAs are always negotiated in
> pairs), which should be little problem for any modern hardware.
>
> The CPU you need to do the actual data transfer, i.e using the tunnels,
> is normally dependent on the bandwidth. A system (with 20.000 SAs)
> connected on 100mbit/s needs to be able to manage 100mbit/s throughput.
> (Or possibly 200mbit/s, in case it's bidirectional.)
>
> For most situations, I imagine administering it will be the real
> problem. (Ex. "we need to change tunnel 4321 to use 3des")
>
>>
>> Impressions of the experienced?
>
> Uh.. no problem? :)
>
> /H

Thanks!  I really appreciate it.  I managed to dug up some decent
information also.

I found a usenix presentation that covered original design goals of
isakmpd, project justification and a lot more.

http://www.usenix.org/publications/library/proceedings/usenix2000/freenix/full_papers/hallqvist/hallqvist_html/ikepaper.html
http://www.openbsd.org/papers/ikeslides.ps

The design-notes doc in the isakmpd src tree has some great stuff in it too.

http://openbsd.secsup.org/src/sbin/isakmpd/DESIGN-NOTES

I also found some discussion about porting isakmpd to linux which offered
some great insight into the differences in architecure and philosophy of
f***swan.

http://ringstrom.mine.nu/pipermail/ipsec_tunnel/2002/000039.html

Thanks to everybody who gave real world feedback.

-Daniel