4.4. Route Load Balancing

Overview

cOS Core provides the option to perform Route Load Balancing (RLB). This is the ability to distribute traffic over multiple alternate routes using one of a number of distribution algorithms.

The purpose of this feature is to provide the following:

Enabling RLB

RLB is enabled on a routing table basis and this is done by creating an RLB Instance object. This object specifies two parameters: a routing table and an RLB algorithm. A table may have only one Instance object associated with it.

One of the algorithms from the following list can be specified in an RLB Instance object:

[Important] Important: ALGs affect which algorithm can be used

If traffic is subject to an ALG for any protocol, either through an IP rule or IP policy, then only the Destination algorithm can be used. This is because protocols using multiple connections will typically use the same source address for NATed clients.

In addition, Round Robin or Spillover should not be used in combination with any ALG because the behavior may not be as expected.

Note also these restrictions do not apply to non-ALG traffic scanning, such as that done by the application control and IDP subsystems.

Disabling RLB

Deleting a routing table's Instance object has the effect of switching off RLB for that table.

RLB Operation

When RLB is enabled for a routing table through an RLB Instance object, the sequence of processing steps is as follows:

  1. Route lookup is done in the routing table and a list of all matching routes is assembled. The routes in the list must cover the exact same IP address range (further explanation of this requirement can be found later in this section).
  2. If the route lookup finds only one matching route then that route is used and balancing does not take place.
  3. If more than one matching route is found then RLB is used to choose which one to use. This is done according to which algorithm is selected in the table's RLB Instance object:
    • Round Robin

      Successive routes are chosen from the matching routes in a "round robin" fashion provided that the metric of the routes is the same. This results in route lookups being spread evenly across matching routes with the same metric. If the matching routes have unequal metrics then routes with lower metrics are selected more often and in proportion to the relative values of all metrics (this is explained further below).

      The RLB Round Robin Algorithm

      Figure 4.5. The RLB Round Robin Algorithm

    • Destination

      This option provides "stickiness" so that unique destination IP addresses always get the same route from a lookup. The importance of this is that a particular destination application can see all traffic coming from the same source IP address.

    • Spillover

      Spillover is not similar to the previous algorithms. With spillover, the first matching route's interface is repeatedly used until the Spillover Limits of that route's interface are continuously exceeded for the Hold Timer number of seconds.

      Once this happens, the next matching route is then chosen. The Spillover Limits for an interface are set in the RLB Algorithm Settings along with the Hold Timer number of seconds (the default is 30 seconds) for the interface.

      When the traffic passing through the original route's interface falls below the Spillover Limits continuously for the Hold Timer number of seconds, route lookups will then revert back to the original route and its associated interface.

      The RLB Spillover Algorithm

      Figure 4.6. The RLB Spillover Algorithm

      Spillover Limits are set separately for ingoing and outgoing traffic with only one of these typically being specified. If both are specified then only one of them needs to be exceeded continuously for Hold Timer seconds for the next matching route to be chosen. The units of the limits, such as Mbps, can be selected to simplify specification of the values.

Using Route Metrics with Round Robin

An individual route has a metric associated with it, with the default metric value being zero.

With the Round Robin and the associated Destination algorithms, the metric value can be set differently on matching routes to create a bias towards the routes with lower metrics. Routes with lower metrics will be chosen more frequently than those with higher metrics and the proportion of usage will be based on the relative differences between the metrics of matching routes.

In a scenario with two ISPs, if the requirement is that the bulk of traffic passes through one of the ISPs then this can be achieved by enabling RLB and setting a low metric on the route to the preferred ISP. A relatively higher metric is then set on the route to the other ISP.

Using Route Metrics with Spillover

When using the Spillover algorithm, a number of points should be noted regarding metrics and the way alternative routes are chosen:

The Requirement for Matching IP Ranges

As explained above, when RLB is assembling a list of matching routes from a routing table, the routes it selects must have the same range. Balancing between routes will not take place if their ranges are not exactly the same.

For instance, if one matching route has an IP address range of 10.4.16.0/24 and there is a second matching route with an address range 10.4.16.0/16 (which is a range that includes 10.4.16.0/24) then RLB will not take place between these routes. The ranges are not exactly the same so RLB will treat the routes as being different.

It should also be remembered that route lookup will select the route that has the narrowest range that matches the destination IP address used in the lookup. In the above example, 10.4.16.0/24 may be chosen over 10.4.16.0/16 because the range is narrower with 10.4.16.0/24 for an IP address they both contain.

RLB ALgorithm Resets and Algorithm Behavior

All RLB algorithms will be reset to their initial state when any of the following occurs:

In all these cases, the chosen route will revert to the one selected when the algorithm began operation, provided that all the inputs to the algorithm are the same. In other words, the algorithms will always behave in the same way if all the inputs are the same. An example of an input that might change algorithm behavior is if the metric of a route changes.

RLB Limitations

It should be noted that the selection of different alternate routes occurs only when the route lookup is done and it is based on the algorithm being used with the routing table used for the lookup and the algorithm's state.

RLB cannot know how much data traffic will be related to each lookup. The purpose of RLB is to be able to spread route lookups across alternatives on the assumption that each lookup will relate to a connection carrying some assumed amount of traffic.

An RLB Scenario

Below is an illustration which shows a typical scenario where RLB might be used. Here, there is a group of clients on a network connected via the LAN interface of the Clavister firewall and these will access the Internet.

Internet access is available from either one of two ISPs, whose gateways GW1 GW2 are connected to the firewall interfaces WAN1 and WAN2. RLB will be used to balance the connections between the two ISPs.

A Route Load Balancing Scenario

Figure 4.7. A Route Load Balancing Scenario

We first need to define two routes to these two ISPs in the main routing table as shown below:

Route No. Interface Destination Gateway Metric
1 WAN1 all-nets GW1 100
2 WAN2 all-nets GW2 100

We will not use the spillover algorithm in this example so the routing metric for both routes should be the same, in this case a value of 100 is selected.

By using the Destination RLB algorithm we can ensure that clients communicate with a particular server using the same route and therefore the same source IP address. If NAT was being used for the client communication, the IP address seen by the server would be WAN1 or WAN2.

In order to flow, any traffic requires both a route and an allowing IP rule set entry. The following rules will allow traffic to flow to either ISP and will NAT the traffic using the external IP addresses of interfaces WAN1 and WAN2.

Rule No. Action Src Interface Src Network Dest Interface Dest Network Service
1 Allow/NAT lan lan_net WAN1 all-nets all_services
1 Allow/NAT lan lan_net WAN2 all-nets all_services

The service all_services is used in the above IP rule set entries but this should be further narrowed to a specific service or service group that covers specific protocols.

Example 4.9. Setting Up RLB

In this example, the details of the RLB scenario described above will be implemented. The assumption is made that the various IP address book objects needed have already been defined.

The IP objects WAN1 and WAN2 represent the interfaces that connect to the two ISPs and the IP objects GW1 and GW2 represent the IP addresses of the gateway routers at the two ISPs.

Step 1. Set up the routes in the main routing table

CLI

First, change the category context to be the main routing table:

Device:/> cc RoutingTable main

Add the route to the first ISP:

Device:/main> add Route
			Interface=WAN1
			Network=all-nets
			Gateway=GW1
			Metric=100

Add the route to the second ISP:

Device:/main> add Route
			Interface=WAN2
			Network=all-nets
			Gateway=GW2
			Metric=100

Return to the original context:

Device:/main> cc
		
Device:/> 

Web Interface

  1. Go to: Network > Routing > Routing Tables
  2. Select the main routing table
  3. Select Add > Route
  4. The dialog for a new route will appear. For the first route, enter:
    • Interface: WAN1
    • Network: all-nets
    • Gateway: GW1
    • Local IP address: Leave as None
    • Metric: 100
    • Click OK
  5. Select Add > Route again to add the second route
  6. The dialog for a new route will appear. For the second route, enter:
    • Interface: WAN2
    • Network: all-nets
    • Gateway: GW2
    • Local IP address: Leave as None
    • Metric: 100
    • Click OK

Step 2. Create an RLB Instance object

A Route Load Balancing Instance object is now created which uses the Destination algorithm will be selected to achieve stickiness so the server always sees the same source IP address (WAN1 or WAN2) from a single client.

Command-Line Interface

Device:/> add RouteBalancingInstance main Algorithm=Destination

InControl

Follow similar steps to those used for the Web Interface below.

Web Interface

  1. Go to: Network > Routing > Instances > Add > Route Balancing Instance
  2. The route balancing instance dialog will appear. Now select:
    • Routing Table: main
    • Algorithm: Destination
    • Click OK

Step 3. Create IP rule set entries to allow traffic to flow

Lastly, IP rule set entries need to be created that allow the traffic to flow. The detailed steps for this are not included here but the created rules would follow the pattern described above.

RLB with VPN

When using RLB with VPN, a number of issues need to be overcome.

If we were to try and use RLB to balance traffic between two IPsec tunnels, the problem that arises is that the Remote Endpoint for any two IPsec tunnels in cOS Core must be different. The solutions to this issue are as follows:

Connecting to External Active-Active Network Equipment

Some network equipment may operate in an active-active mode where the Clavister firewall can communicate using alternate links to the equipment. It is possible for RLB to be used so that cOS Core can take advantage of the active-active feature so traffic can flow through both links to increase throughput.

Setting this up is described by an article in the Clavister Knowledge Base at the following link:

https://kb.clavister.com/324735687