Routing and Multi-Wan
Not surprisingly, advanced routing is required on a multi-WAN system. Like most seasoned system administrators, the route command is part of the standard tool set. With multi-WAN, you need to forget about using route and start becoming friends with the ip route command! Every WAN interface on a multi-WAN system has its own routing table. The default table (shown in the “route” command) is just not relevant for WANs in multi-WAN. We will start to dig a bit deeper below.
Simple ClearOS Gateway
The following is the output from running the old route command on a simple ClearOS gateway:
# route Destination Gateway Genmask Flags Metric Ref Use Iface 206.248.154.122 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0 192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2 0.0.0.0 206.248.154.122 0.0.0.0 UG 0 0 0 ppp0
There's a LAN network on eth2/192.168.2.x and a PPPoE/DSL connection on ppp0. Here is the same routing table using the ip route command:
# ip route 206.248.154.122 dev ppp0 proto kernel scope link src 69.165.220.249 192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.1 default via 206.248.154.122 dev ppp0
ClearOS Gateway with 2 WANs
As soon as we have more than two WANs, routing becomes a little more complicated. So where do routes for each WAN interface go? The powerful (and sparsely documented) ip command is the key. With 2 WANs running, you can list all the routing tables using ip rule:
# ip rule 0: from all lookup 255 50: from all lookup 50 100: from all fwmark 0x8000 lookup 100 101: from all fwmark 0x8001 lookup 101 200: from 69.165.220.249 lookup 200 201: from 69.165.220.248 lookup 201 250: from all lookup 250 32766: from all lookup main 32767: from all lookup default
For comparison, take a look at the output from the ip rule command on a simple 1-WAN gateway:
# ip rule 0: from all lookup 255 32766: from all lookup main 32767: from all lookup default
The first column is the table number. For now, we are interested in the tables below:
Table | Description |
---|---|
100 | WAN #1 routing details |
101 | WAN #2 routing details |
250 | default route |
Using the ip command again, you can examine the routing table for WAN #1 / ppp0:
# ip route show table 100 206.248.154.122 dev ppp0 proto kernel scope link src 69.165.220.249 192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.1 default via 206.248.154.122 dev ppp0
The routing table for WAN #2 looks similar, but ppp1 information is used:
# ip route show table 101 206.248.154.122 dev ppp1 proto kernel scope link src 69.165.220.248 192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.1 default via 206.248.154.122 dev ppp1
Table 250 is an important one. It shows the default route including the weight for each of the WANs:
# ip route show table 250 default proto static nexthop via 206.248.154.122 dev ppp0 weight 2 nexthop via 206.248.154.122 dev ppp1 weight 1
Multi-WAN and VPN Routing
When you add routes to a multi-WAN system, it is necessary to add routes to the routing table for each WAN interface. Fortunately, the ClearOS firewall simplifies the process. In our example, we are going to add a route for 192.168.55.0/24 on ppp1:
# ip route add 192.168.55.0/24 dev ppp1
Our default routing table looks like:
# ip route 206.248.154.122 dev ppp0 proto kernel scope link src 69.165.220.249 206.248.154.122 dev ppp1 proto kernel scope link src 69.165.220.248 192.168.55.0/24 dev ppp1 scope link 192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.1 default via 206.248.154.122 dev ppp0
Sure enough, our route has been added to the default table. Unfortunately, the routing table for each of the WAN interfaces has not been updated:
# ip route show table 100 206.248.154.122 dev ppp0 proto kernel scope link src 69.165.220.249 192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.1 default via 206.248.154.122 dev ppp0
For all intents and purposes, our 192.168.55.0/24 route is not active. If you restart the firewall, it will automatically rebuild the multi-WAN routing tables:
# service firewall restart # ip route show table 100 206.248.154.122 dev ppp0 proto kernel scope link src 69.165.220.249 192.168.55.0/24 dev ppp1 scope link 192.168.2.0/24 dev eth2 proto kernel scope link src 192.168.2.1 default via 206.248.154.122 dev ppp0
Multi-WAN and Static Routes
If you are adding static routes to your ClearOS system, please make sure you follow the configuration the Managing Static Routes document.