Overview
cOS Core provides the option to perform Route Load Balancing (RLB). This is the ability to distribute traffic over multiple alternate routes using one of a number of distribution algorithms.The purpose of this feature is to provide the following:
Enabling RLB
RLB is enabled on a routing table basis and this is done by creating an RLB Instance object. This object specifies two parameters: a routing table and an RLB algorithm. A table may have only one Instance object associated with it.One of the algorithms from the following list can be specified in an RLB Instance object:
Matching routes are used equally often by successively going to the next matching route.
This is an algorithm that provides destination IP "stickiness" so that the same destination IP address gets the same route.
This uses the next route when specified interface traffic limits are exceeded continuously for a given time.
RLB Operation
When RLB is enabled for a routing table through an RLB Instance object, the sequence of processing steps is as follows:Round Robin
Successive routes are chosen from the matching routes in a "round robin" fashion provided that the metric of the routes is the same. This results in route lookups being spread evenly across matching routes with the same metric. If the matching routes have unequal metrics then routes with lower metrics are selected more often and in proportion to the relative values of all metrics (this is explained further below).
Destination
This option provides "stickiness" so that unique destination IP addresses always get the same route from a lookup. The importance of this is that a particular destination application can see all traffic coming from the same source IP address.
Spillover
Spillover is not similar to the previous algorithms. With spillover, the first matching route's interface is repeatedly used until the Spillover Limits of that route's interface are continuously exceeded for the Hold Timer number of seconds.
Once this happens, the next matching route is then chosen. The Spillover Limits for an interface are set in the RLB Algorithm Settings along with the Hold Timer number of seconds (the default is 30 seconds) for the interface.
When the traffic passing through the original route's interface falls below the Spillover Limits continuously for the Hold Timer number of seconds, route lookups will then revert back to the original route and its associated interface.
Spillover Limits are set separately for ingoing and outgoing traffic with only one of these typically being specified. If both are specified then only one of them needs to be exceeded continuously for Hold Timer seconds for the next matching route to be chosen. The units of the limits, such as Mbps, can be selected to simplify specification of the values.
Using Route Metrics with Round Robin
An individual route has a metric associated with it, with the default metric value being zero.With the Round Robin and the associated Destination algorithms, the metric value can be set differently on matching routes to create a bias towards the routes with lower metrics. Routes with lower metrics will be chosen more frequently than those with higher metrics and the proportion of usage will be based on the relative differences between the metrics of matching routes.
In a scenario with two ISPs, if the requirement is that the bulk of traffic passes through one of the ISPs then this can be achieved by enabling RLB and setting a low metric on the route to the preferred ISP. A relatively higher metric is then set on the route to the other ISP.
Using Route Metrics with Spillover
When using the Spillover algorithm, a number of points should be noted regarding metrics and the way alternative routes are chosen:Route metrics should always be set.
With spillover, cOS Core always chooses the route in the matching routes list that has the lowest metric. The algorithm is not intended to be used with routes having the same metric so the administrator should set different metrics for all the routes to which spillover applies.
Metrics determine a clear ordering for which route should be chosen next after the interface traffic limits for the chosen route have been exceeded.
There can be many alternative routes.
Several alternative routes can be set up, each with their own interface limits and each with a different metric. The route with the lowest metric is chosen first and when that route's interface limits are exceeded, the route with the next highest metric is then chosen.
When that new route's interface limits are also exceeded then the route with the next highest metric is taken and so on. As soon as any route with a lower metric falls below its interface limit for its Hold Timer number of seconds, then it reverts to being the chosen route.
If there is no alternative route, the route does not change.
If the spillover limit is reached but all alternative routes have also reached their limit then the route will not change.
The Requirement for Matching IP Ranges
As explained above, when RLB is assembling a list of matching routes from a routing table, the routes it selects must have the same range. Balancing between routes will not take place if their ranges are not exactly the same.For instance, if one matching route has an IP address range of 10.4.16.0/24 and there is a second matching route with an address range 10.4.16.0/16 (which is a range that includes 10.4.16.0/24) then RLB will not take place between these routes. The ranges are not exactly the same so RLB will treat the routes as being different.
It should also be remembered that route lookup will select the route that has the narrowest range that matches the destination IP address used in the lookup. In the above example, 10.4.16.0/24 may be chosen over 10.4.16.0/16 because the range is narrower with 10.4.16.0/24 for an IP address they both contain.
RLB ALgorithm Resets and Algorithm Behavior
All RLB algorithms will be reset to their initial state when any of the following occurs:In all these cases, the chosen route will revert to the one selected when the algorithm began operation, provided that all the inputs to the algorithm are the same. In other words, the algorithms will always behave in the same way if all the inputs are the same. An example of an input that might change algorithm behavior is if the metric of a route changes.
RLB Limitations
It should be noted that the selection of different alternate routes occurs only when the route lookup is done and it is based on the algorithm being used with the routing table used for the lookup and the algorithm's state.RLB cannot know how much data traffic will be related to each lookup. The purpose of RLB is to be able to spread route lookups across alternatives on the assumption that each lookup will relate to a connection carrying some assumed amount of traffic.
Below is an illustration which shows a typical scenario where RLB might be used. Here, there is a group of clients on a network connected via the LAN interface of the Clavister firewall and these will access the Internet.Internet access is available from either one of two ISPs, whose gateways GW1 GW2 are connected to the firewall interfaces WAN1 and WAN2. RLB will be used to balance the connections between the two ISPs.
We first need to define two routes to these two ISPs in the main routing table as shown below:
Route No. | Interface | Destination | Gateway | Metric |
---|---|---|---|---|
1 | WAN1 | all-nets | GW1 | 100 |
2 | WAN2 | all-nets | GW2 | 100 |
We will not use the spillover algorithm in this example so the routing metric for both routes should be the same, in this case a value of 100 is selected.
By using the Destination RLB algorithm we can ensure that clients communicate with a particular server using the same route and therefore the same source IP address. If NAT was being used for the client communication, the IP address seen by the server would be WAN1 or WAN2.
In order to flow, any traffic requires both a route and an allowing IP rule set entry. The following rules will allow traffic to flow to either ISP and will NAT the traffic using the external IP addresses of interfaces WAN1 and WAN2.
Rule No. | Action | Src Interface | Src Network | Dest Interface | Dest Network | Service |
---|---|---|---|---|---|---|
1 | Allow/NAT | lan | lan_net | WAN1 | all-nets | all_services |
1 | Allow/NAT | lan | lan_net | WAN2 | all-nets | all_services |
The service all_services is used in the above IP rule set entries but this should be further narrowed to a specific service or service group that covers specific protocols.
Example 4.9. Setting Up RLB
In this example, the details of the RLB scenario described above will be implemented. The assumption is made that the various IP address book objects needed have already been defined.
The IP objects WAN1 and WAN2 represent the interfaces that connect to the two ISPs and the IP objects GW1 and GW2 represent the IP addresses of the gateway routers at the two ISPs.
Step 1. Set up the routes in the main routing table
CLI
First, change the category context to be the main routing table:
Device:/>
cc RoutingTable main
Add the route to the first ISP:
Device:/main>
add Route
Interface=WAN1
Network=all-nets
Gateway=GW1
Metric=100
Add the route to the second ISP:
Device:/main>
add Route
Interface=WAN2
Network=all-nets
Gateway=GW2
Metric=100
Return to the original context:
Device:/main> cc
Device:/>
Web Interface
Step 2. Create an RLB Instance object
A Route Load Balancing Instance object is now created which uses the Destination algorithm will be selected to achieve stickiness so the server always sees the same source IP address (WAN1 or WAN2) from a single client.
Command-Line Interface
Device:/>
add RouteBalancingInstance main Algorithm=Destination
InControl
Follow similar steps to those used for the Web Interface below.
Web Interface
Step 3. Create IP rule set entries to allow traffic to flow
Lastly, IP rule set entries need to be created that allow the traffic to flow. The detailed steps for this are not included here but the created rules would follow the pattern described above.
If we were to try and use RLB to balance traffic between two IPsec tunnels, the problem that arises is that the Remote Endpoint for any two IPsec tunnels in cOS Core must be different. The solutions to this issue are as follows:
Use two ISPs, with one tunnel connecting through one ISP and the other tunnel connecting through the other ISP. RLB can then be applied as normal with the two tunnels.
In order to get the second tunnel to function in this case, it is necessary to add a single host route in the main routing table that points to the secondary ISPs interface and with the secondary ISPs gateway.
This solution has the advantage of providing redundancy should one ISP link fail.
Use VPN with one tunnel that is IPsec based and another tunnel that is uses a different protocol.
If both tunnels must be, for example, IPsec connects, it is possible to wrap IPsec in a GRE tunnel (in other words, the IPsec tunnel is carried by a GRE tunnel). GRE is a simple tunneling protocol without encryption and therefore involves a minimum of extra overhead. See Section 3.4.7, GRE Tunnels for more about this topic.
Connecting to External Active-Active Network Equipment
Some network equipment may operate in an active-active mode where the Clavister firewall can communicate using alternate links to the equipment. It is possible for RLB to be used so that cOS Core can take advantage of the active-active feature so traffic can flow through both links to increase throughput.Setting this up is described by an article in the Clavister Knowledge Base at the following link: