Path MTU Discovery is a method by which the MTU size of packets sent across the Internet can be adjusted to meet the MTU limits of traversed network equipment thus avoiding fragmentation. When a packet exceeds an MTU limit, ICMP messages are used to adjust the MTU size sent. It is defined by RFC 1981 (for IPv6) and RFC 1191 (for IPv4), and makes use of ICMP messages to communicate MTU requirements between equipment.

When external equipment receives traffic with an MTU size greater than the next hop MTU size, an ICMP message is sent back to the source so the sender's MTU can be adjusted accordingly. The Clavister NetShield Firewall can act both as the endpoint in these exchanges as well as forwarding messages between endpoints.

Path MTU discovery is enabled by default for IPv6 on all interfaces. For IPv4, it must be explicitly enabled. The current MTU used by cOS Stream for a destination can be viewed using the CLI flow command but only for locally initiated or terminated traffic.

The Clavister NetShield Firewall path MTU behavior is controlled by a set of advanced settings which are described in this section. They include settings from both the IPSettings and ICMPSettings object categories.

IP4PathMTUMin

This is the smallest size which path MTU discovery can decrease the MTU to for IPv4 traffic.

System:/> set Settings IPSettings IP4PathMTUMin=576

This setting is only relevant when the firewall acts as a traffic endpoint and controls whether cOS Stream uses the path MTU information received from another hop.

cOS Stream determines the next-hop MTU by using the MTU property value of the Route object in its configuration for the next hop. If this is not set (has a value of zero), the next-hop MTU defaults to the MTU property value of the interface associated with the next hop.

For RFC compatibility, this value should be set to 68 and this is the smallest value that should be used. Other common values are 576 and 65535. The value 576 is the smallest IP packet that a host is required to handle and is commonly used as the MTU size.

Default: 576

IP6PathMTUMin

This is the IPv6 equivalent of IP4PathMTUMin and has the same function as described above.

System:/> set Settings IPSettings IP6PathMTUMin=1280

The minimum allowed value for an IPv6 network is 96. If path discovery finds smaller than 1280 is needed, 1280 is used anyway but a specialized header is added to fragments. This can result in a fragment that is both the first and the last.

Default: 1280

IP4PathMTULifetime

This controls how long a path MTU value should be considered valid in minutes for IPv4. A value of zero means infinite time.

System:/> set Settings IPSettings IP4PathMTULifetime=10

If somewhere along the path still requires a lower MTU after this many minutes, the data packet will be dropped and a new ICMP error will be received. In other words the process is restarted as if the path MTU is unknown.

Having a lower lifetime value can be advantageous if the network is unstable and the path MTU between source and destination fluctuates. The cost of a lower lifetime is that the MTU must be tested more often with multiple potential packet losses and retransmissions occurring each time the lifetime expires. A higher lifetime can be appropriate in stable networks where the path MTU changes infrequently.

Default: 10

IP6PathMTULifetime

This is the IPv6 equivalent of IP4PathMTULifetime and has the same function as described above.

System:/> set Settings IPSettings IP6PathMTULifetime=10

Default: 10

IP4OnPktTooBigAndDFSet

This setting is only used when processing a packet that is larger than the next hop and the Do Not Fragment (DF) flag is set. It enables or disables participation of the firewall in path MTU discovery and enables/disables the sending of discovery information to other hosts.

System:/> set Settings IPSettings  IP4OnPktTooBigAndDFSet=SendICMPNeedFrag

The default of SendICMPNeedFrag means that participation is enabled. An ICMP error message will be sent back to the source when an attempt is made to forward a packet that is too big for the next hop and the packet itself is being dropped. For IPv4, the DF flag is honored (the packet is fragmented if the flag is unset). This will only expose minimal information (the MTU used inside the network and the IP of the firewall).

Participation can be turned off (and the protected network's MTU hidden) by using the setting StripDFAndFragment to disable DF flag when the packet size goes below a given value.

This setting may break protocols that rely on a rejected packet really being dropped, or very security aware operating systems that constantly verify error messages against the packets that has been received at the other end-point (for example, the OpenBSD TCP stack).

Default: SendICMPNeedFrag

IP6OnPacketTooBig

This is the equivalent of IP4OnPktTooBigAndDFSet for IPv6 and has the same function as described above.

System:/> set Settings IPSettings IP6OnPacketTooBig=SendICMPPktTooBig

Default: SendICMPPktTooBig

StripDFOnSmall

This controls how small IPv4 packets can be to participate in the path discovery process and determines how to handle the DF flag when forwarding. Packets smaller than this value will have their DF flag cleared.

If a packet is smaller than this value, and a hop decides that it is too big, the packet will be fragmented and will not participate in the discovery process.

System:/> set Settings IPSettings StripDFOnSmall=65535

This setting only affects traffic being forwarded by cOS Stream and not traffic where the firewall is the traffic endpoint. It should be set to 65535 in order to block discovery one hop beyond the firewall and set to zero in order to fully support discovery.

In order to block discovery and fully hide the network topology beyond the firewall, StripDFOnSmall is set to 65535 and IP4OnPacketTooBig is set to the value Fragment.

Default: 65535

IP4FragmentationNeeded

This controls whether the error messages involved in discovery are allowed to be forwarded from the network beyond the firewall and back to the source.

System:/> set Settings ICMPSettings IP4FragmentationNeeded=ObeyServiceLog

This setting must be set to AllowAlways or ObeyServiceLog to fully support path discovery and to be RFC compliant. Dropping these error messages can silently create severe network problems. The Drop option should only be used when the consequences are understood.

If the setting's value is ObeyService or ObeyServiceLog then the Service object used with the IP rule for the relevant flow must have the property PassICMPReturn enabled (it is disabled by default). By disabling the PassICMPReturn property it is also possible to disable ICMP message forwarding for a particular IP rule instead of disabling it globally.

Default: ObeyServiceLog

IP6PacketTooBig

This is the IPv6 equivalent of IP4FragmentationNeeded and has the same function as described above.

System:/> set Settings ICMPSettings IP6PacketTooBig=AlwaysAllow

Default: AlwaysAllowLog

Path MTU Discovery Setting Combinations

The following are recommended combinations of settings for enabling and disabling path discovery:

Path MTU Discovery Forwarding

The Clavister NetShield Firewall can forward any path discovery ICMP messages that need to traverse the firewall. Forwarding depends on the following two settings:

The following example shows how the advanced settings are used with and without forwarding.

Path MTU Discovery Examples

To better understand the use of the various advanced settings to control MTU path forwarding, consider the following two examples.

Example 1: The System is not the Endpoint

Shown below is a scenario where a client tries to connect to a server via a Clavister NetShield Firewall, the public Internet and a router.

The path MTU discovery exchanges that take place are as follows:

Example 2: The System as the Endpoint

In this example, the Clavister NetShield Firewall is the endpoint and a client tries to open a flow to it across the public Internet using the SSH protocol.

The path MTU discovery exchanges that take place are as follows: