> Hi,
>
> I have two ISPs and a LAN. I need to load balance outgoing connections
> from the LAN between the ISPs.
>
> My router is running OpenBSD 4.6-stable, up-to-date.
>
> -----------------
> | |
> ISP 1 ---(isp1 box)---+-ext1_if |
> | |
> | int_if-+--- LAN
> | |
> ISP 2 ---(isp2 box)---+-ext2_if |
> | |
> -----------------
> OpenBSD
>
> int_if -> interface to LAN
> int_net -> 192.168.0.0/24
>
> ext1_if -> interface to ISP 1 (192.168.1.2)
> ext1_gw -> ISP 1 gateway (192.168.1.1)
>
> ext2_if -> interface to ISP 2 (192.168.2.2)
> ext2_gw -> ISP 2 gateway (192.168.2.1)
>
> There is no default route (empty /etc/mygate).
>
> Following
http://www.openbsd.org/faq/pf/pools.html#outgoing I ended up
> with the following :
>
> ---sysctl.conf---
> net.inet.ip.forwarding=1
> ---sysctl.conf---
>
> ---pf.conf---
> set debug loud
> set block-policy return
> #set optimization aggressive
> #set timeout src.track 300
> set skip on lo
>
> nat log on $ext1_if from $int_net -> ($ext1_if)
> nat log on $ext2_if from $int_net -> ($ext2_if)
>
> block log
>
> pass in log on $int_if route-to \
> { ($ext1_if $ext1_gw), ($ext2_if $ext2_gw) } \
> from $int_net
>
> pass in log on $int_if from $int_net to $int_if
>
> pass out log
> pass out log on $ext1_if route-to ($ext2_if $ext2_gw) from $ext2_if
> pass out log on $ext2_if route-to ($ext1_if $ext1_gw) from $ext1_if
> ---pf.conf---
>
> Everything works fine with this setup but https and some ftp servers
> are very sensitive to IP changes, so I did add sticky-address to bind
> a connection to an ISP and fix this issue :
>
> pass in log on $int_if route-to \
> { ($ext1_if $ext1_gw), ($ext2_if $ext2_gw) } sticky-address \
> from $int_net
>
> This works when only one PC open a connection. When another PC open
> a connection the first one has no longer access to the internet. After
> some time in tcpdump and /var/log/messages, it seems because packets
> are routed to the wrong interface.
>
> /var/log/messages analysis :
>
> 192.168.0.78 tries a connection :
> /bsd: pf_map_addr: selected address 192.168.2.1
> /bsd: pf_map_addr: selected address 192.168.2.2
> /bsd: pf_map_addr: src tracking maps 192.168.0.78 to 192.168.2.1
> /bsd: pf_map_addr: src tracking maps 192.168.0.78 to 192.168.2.2
> /bsd: pf_map_addr: src tracking maps 192.168.0.78 to 192.168.2.1
> /bsd: pf_map_addr: src tracking maps 192.168.0.78 to 192.168.2.2
>
> Looks ok, now 192.168.0.29 tries a connection too :
> /bsd: pf_map_addr: selected address 192.168.1.1
> /bsd: pf_map_addr: selected address 192.168.1.2
> /bsd: pf_map_addr: src tracking maps 192.168.0.29 to 192.168.1.1
> /bsd: pf_map_addr: src tracking maps 192.168.0.29 to 192.168.1.2
> /bsd: pf_map_addr: src tracking maps 192.168.0.29 to 192.168.1.1
> /bsd: pf_map_addr: src tracking maps 192.168.0.29 to 192.168.1.2
>
> Looks good, now 192.168.0.78 retries a connection :
> /bsd: pf_map_addr: src tracking maps 192.168.0.78 to 192.168.2.1
> /bsd: pf_map_addr: selected address 192.168.1.2
> [and that's all]
>
> This is confirmed by tcpdump (on pflog0, bge0, em0 and em1) : the
> packets are routed to the wrong interface. Three SYN are sent, then
> client bail out. Tests were quickly done, so there is no state/track
> timeout here.
>
> Is there something I am doing wrong ?
>
