	Note: This document is also available in other formats
A PDF version of this document along with all current and older documentation in PDF format can be found at https://my.clavister.com. It is also available in a framed HTML version.

Note: This document is also available in other formats

A PDF version of this document along with all current and older documentation in PDF format can be found at https://my.clavister.com.

It is also available in a framed HTML version.

By using virtualization, it is possible to have a single computer running multiple virtual firewalls, each running a separate software copy. This technique is referred to as virtualization and each virtual firewall can be said to be running in its own virtual machine. This is the basis for the Clavister Virtual Series of products and includes running under VMware™ or the open source KVM virtual environment.

Not only can cOS Stream run in its own virtual machine, the management workstation that is used to administer cOS Stream can also run as a virtual machine on the same or on different hardware. This workstation could be communicating using a CLI console which connects using a Secure Shell (SSH) client.

Supported Virtual Environments

The following virtual environments can host a Clavister Firewall:

VMware ESXi version 5.5 or later.
QEMU/KVM version 1.0.0 or later.

The following is required for the virtual CPU:

Intel VT-d or AMD-Vi technology.
SSE4.2 (Intel Streaming SIMD Extensions 4.2).

For a further discussion of resource allocation for all virtual machines, including memory requirements, see Chapter 5, Resource Allocation Guidelines.

Installation Documentation Structure

A general description of installation is given in Chapter 2, Installation. The installation steps in that section are common to all virtual environments. The difference between installation in different virtual environments is the creation and customizing of a virtual machine. This is described in the following parts of this document:

Chapter 3, VMware Virtual Machine Creation.
Chapter 4, KVM Virtual Machine Creation.

Licensing

No license is supplied with the default firewall configuration and the only function that is enabled without a license is management access via one of the Ethernet interfaces or the directly connected console. Retrieving and installing a license is described in Chapter 2, Installation as part of the overall installation process.

Referencing Hypervisor Documentation

This guide describes the steps involved when installing and running cOS Stream in a virtualized environment, as well as covering many of the issues that may be encountered when running in such as environment.

The guide tries to deal specifically with the subject of running in the virtual environment and unless relevant, does not go into depth about the hypervisors themselves. The administrator should refer to documentation that is publicly available over the Internet.

Chapter 2: Installation

This section describes the installation steps in a virtual environment. It includes details of customer registration and license installation. The steps are organized into the following stages:

Register as a User and Download
Create a Virtual Machine
Setting Up Management Access
Register a License and Bind It

Register as a User and Download

Go to the URL https://my.clavister.com in a web browser.

The MyClavister login page is presented. If you are already registered, log in and skip to step 8. If you are a new customer accessing MyClavister for the first time, click the Create Account link.

The registration page is now presented. The required information should be filled in. In the example below, a user called John Smith is registering.

When the registration details are accepted, an email is sent to the email address given so that the registration can be confirmed.

Below is an example of the heading in the email that would be received.

The confirmation link in the email leads back to the Clavister website to show that confirmation has been successful and logging in is now possible.

After logging in, the customer name is displayed with links for changing settings and logging out.

To download cOS Stream for VMware, select Downloads and then cOS Stream.

Press the Download button next to the desired product and version number to get a list in a popup window of all the different distributions available for that version. The button for the latest version is always at the top.

Select and download the relevant files. Disk images are available in the .ova format (for VMware) and qcow2 format (for KVM). Alternatively, an upgrade package can be downloaded which can be applied to an existing virtual firewall in any environment.

Create a Virtual Machine

After downloading the relevant disk image file, follow the instructions for creating a virtual machine using the instructions in one of the following sections:

Chapter 3, VMware Virtual Machine Creation.
Chapter 4, KVM Virtual Machine Creation.

Setting Up Management Access

The hypervisor's console for the virtual machine will now be available for entering CLI commands. If this was not a virtual environment, the console would be directly connected to a port on system hardware. It allows the administrator to issue any CLI command and can be used to configure cOS Stream.

SSH Management access via a network connection to an Ethernet interface is not yet possible because management access must be configured for one of these interfaces. On initial startup, cOS Stream will detect the available Ethernet interfaces. By default, the interfaces if1 and if2 will be present.

The following list of steps is used for setting up network management access:

Configure SSH management access via one of the Ethernet interfaces. In this example, if1 is used. First, enable the predefined RemoteMgmtSSH rule:
```
System:/> set RemoteManagement RemoteMgmtSSH RemoteMgmtSSH -enable
```
Then the allowed interface and network must be set:
```
System:/> set RemoteManagement RemoteMgmtSSH RemoteMgmtSSH
			SourceInterface=if1
			SourceNetwork=if1_net
```
The default SSH login credentials for RemoteMgmtSSH are:
- Username: admin
- Password: admin
The system's address book is automatically filled with address objects for all the available Ethernet interfaces. For network access, the if1 interface will need an IP address and IP network assigned to it. First, assign the IP address. Suppose that 203.0.113.10 will be used, then the command would be:
```
System:/> set Address IPAddress if1_ip Address=203.0.113.10
```
Secondly, assign the network to which the IP address belongs:
```
System:/> set Address IPAddress if1_net Address=203.0.113.0/24
```
Note that the interface address of the connecting SSH client computer must belong to the network if1_net (in this example, 203.0.113.0/24). If a Windows computer is used as the client, the console command ipconfig could be used to check the interface IP address.
For public Internet access, an all-nets default route needs to be added to the main routing table which includes the gateway address of a router. Unless there is a narrower route that matches for outgoing traffic, this route will always be used. To add the route, the CLI context needs to be changed to be the main routing table:
```
System:/> cc RoutingTable main
```
The command prompt will change to show that the current context is the main routing table:
```
System:/RoutingTable/main> 
```
Now, routes can be added to the main table. Assuming that the if1 interface is connected to a router with the IPv4 address 203.0.113.1 then a default route is added with the following command:
```
System:/RoutingTable/main> add Route
			Interface=if1
			Network=all-nets-ip4
			Gateway=203.0.113.1
```
Next, restore the CLI context to the default:
```
System:/RoutingTable/main> cc
System:/> 
```
Finally, all configuration changes must be now activated with the following command:
```
System:/> activate
```
Following activation, the changes must be committed permanently within 30 seconds otherwise the configuration will revert back to the original configuration and the changes will be lost. This is a check by cOS Stream that the administrator has not been locked out by the changes:
```
System:/> commit
```

Now, an SSH console to the IP address of the if1 interface can be used to perform further administration tasks. SCP uploads can also be made via the same IP address.

Register a License and Bind It

A license must be associated with a MAC address on the virtual machine. To get a MAC address, use the following CLI command:

System:/> ifstat if1

Below, are the first few lines of some typical output from this command, displaying an example MAC address:

System:/> ifstat if1
Interface if1: 
  IP Address     : 203.0.113.10 
  MAC            : 51:54:00:90:64:75 
  Device         : if1 
  PCI address    : 00:03.0 
  Driver         : virtio 
  MTU            : 1500 
  Routing Table  : <all> 
  Status         : 10 Gbps full duplex

Now, log into the MyClavister website and select the Register License menu option.

Select the NetShield option.

The registration fields will be displayed. After selecting the product type as Virtual Model, enter the License Number and the MAC Address. The license number will be supplied by the product reseller and the MAC address was noted in an earlier step.

After the license is registered and associated with the MAC address, select the Licenses menu, then the License List option and select the newly registered license from the displayed list.

Now, select the Download option and download the license file to the local disk.

Finally, upload the license using SCP. After uploading, the license must be activated with the following command:
```
System:/> license activate <license-filename>
```

The management access only restriction will be removed and cOS Stream will be restricted only by the capabilities defined by the license file.

Chapter 3: VMware Virtual Machine Creation

This section covers installation of the Clavister Firewall with VMware ESXi (version 5.5 or later) using the vSphere client. It assumed that the required software distribution files have been downloaded. File downloading is described in Chapter 2, Installation.

The steps for virtual machine creation as as follows:

Open the vSphere client and select File > Deploy OVF Template...

A vSphere wizard will start that allows the OVA file provided in the product distribution package to be selected. Press Next at each step as the default settings will be used. The final wizard step will show the summarized settings. Press Finish to close the wizard and create the virtual machine.

In vSphere, press the inventory button to see all the available virtual machines. The new virtual machine will be listed.

Right click on the new virtual machine and select Edit Settings.

	Important: Adjust the RAM memory to suit the requirements
	The default VMware virtual machine provides 1024 Mbytes of RAM memory. This is the absolute minimum value needed for cOS Stream to run. Many scenarios may need a higher amount of memory so the RAM size should be adjusted according to the requirements. It is recommended to have 2 cores allocated but cOS Stream can run on a single core.

The settings will show the current memory allocated to the virtual machine and the two virtual Ethernet interfaces that are included in the virtual machine image supplied and that cOS Stream will use. These virtual interfaces should each be assigned to a real Ethernet network adapter. For the Clavister Firewall, they will have the default logical names if1 and if2.

Now, power on this new virtual machine and cOS Stream will start up. Without an installed license, cOS Stream will only allow management access.

Switch to the Console tab to see the system console. If this was an actual Clavister hardware product, the console would be directly connected to a port on the hardware box. It allows the administrator to issue any CLI command and can be used to configure cOS Stream.

Selecting the Interface Driver Type

If only a single virtual CPU core is available to the virtual machine then the interface NIC type should be set to E1000E because interrupt mode will be used by cOS Stream. Selecting E1000 on all NICs will minimize data plane usage and make it possible to share a single CPU core with other tasks.

When running with multiple CPU cores, poll mode will be used instead by cOS Stream. This will provide higher performance and any supported interface type can be used.

Use a Single Socket with Multiple Cores

As described in Chapter 5, Resource Allocation Guidelines, cOS Stream should be allocated a single socket with at least two cores (although cOS Stream can run on a single core). With VMware, this may be configured in one of two ways:

In newer management interfaces

In newer management interfaces, configuration is done using the settings Total CPU Cores and Cores per Socket. The number of sockets is therefore the first setting divided by the second. Since only a single socket should be used with cOS Stream, these two settings should always be the same number.
In older management interfaces

In older management interfaces, configuration is simpler since the setting Number of virtual sockets specifies the number of sockets and this should be set to a value of one. The setting Cores per Socket is then used to specify the number of cores to be assigned to that socket.

VM Separation is Needed With HA Clusters

When deploying a high availability cluster with two firewalls, the master and slave in the cluster should be running on different VMware compute nodes so that one can still be active should a compute node fail.

When using VMware DRS, this can be achieved by specifying the appropriate DRS separation rules.

Chapter 4: KVM Virtual Machine Creation

This section describes the creation of a virtual machine in the KVM environment after downloading the required files. File downloading is described in Chapter 2, Installation.

It is assumed that the Virtual Machine Manager software (version 0.9.1 or later) is installed on the local management workstation. The following components are required on the server:

QEMU/KVM version 1.0.0 or later.
libvirt version 0.9.8 or later.

The steps for virtual machine creation are as follows:

Download the distribution for KVM after logging in as a user at the Clavister website.

The KVM distribution is distributed as a .qcow2 file which is a disk image of the virtual machine. The filename will have the following form:
```
clavister-cos-stream-<version>-virtual-x64-generic.qcow2
```
Where <version> is the version number of the release.

Start the Virtual Machine Manager (virt-manager) software and connect to KVM.

Select the New Virtual Machine option in Virtual Machine Manager and the new virtual machine wizard will start.

Enter a name for the new virtual machine, select Import existing disk image and then select Forward.

Select Browse and choose the downloaded .qcow2 file.

Select the amount of RAM memory and the number of CPUs required. Note that cOS Stream cannot run in less than 1024 Mbytes of RAM and it is recommended to have at least 2 processor cores allocated. However, cOS Stream can run on a single core.

In the next wizard step, make sure that the Virt Type is kvm and the Architecture is x86_64. If the Bridged NAT option is to be used, this can be configured here. Usually, this is set to Bridge Network.

When the wizard is finished, the new virtual machine configuration will now be displayed.

The virtual machine created can now be edited to meet specific requirements. The virtual machine will have no Ethernet interfaces and these should be added using the virtual machine manager.

Selecting the Interface Driver Type

If only a single virtual CPU core is available to the virtual machine then the interface NIC type should be set as E1000 because interrupt mode will be used by cOS Stream. Selecting E1000 on all NICs will minimize data plane usage and make it possible to share a single CPU core with other tasks.

When running with multiple CPU cores, poll mode will be used instead by cOS Stream. This will provide higher performance and any supported interface type can be used.

Specific Resource Allocation for KVM

The general guidelines for resource allocation are detailed in Chapter 5, Resource Allocation Guidelines. The following should be specifically noted about correctly allocating resources with KVM:

Only a single socket is supported so the number of sockets should always be set to one.
The number of cores can be set to the maximum number available.
The number of threads should be set to a value of 2.
Each core should be pinned to a hyperthread sibling in a one-to-one relationship.

An example extract from a KVM configuration is found below which uses the guidelines given in the list above.

<vcpu placement='static' cpuset='1-4,17-20'>28</vcpu>
<cputune>
  <vcpupin vcpu='0' cpuset='1'/>
  <vcpupin vcpu='1' cpuset='17'/>
  <vcpupin vcpu='2' cpuset='2'/>
  <vcpupin vcpu='3' cpuset='18'/>
  <vcpupin vcpu='4' cpuset='3'/>
  <vcpupin vcpu='5' cpuset='19'/>
  <vcpupin vcpu='6' cpuset='4'/>
  <vcpupin vcpu='7' cpuset='20'/>
</cputune>
<os>
  <type arch='x86_64' machine='pc-i440fx-xenial'>hvm</type>
  <boot dev='hd'/>
</os>
<features>
  <acpi/>
  <apic/>
</features>
<cpu mode='host-passthrough'>
  <topology sockets='1' cores='4' threads='2'/>
</cpu>

Chapter 5: Resource Allocation Guidelines

This section gives general guidelines for resource allocation for cOS Stream virtual machines for both VMware and KVM. Hypervisor specific guidelines can be found in the setup sections for VMware and KVM.

Minimum Memory and Disk Requirements

All hypervisor image files supplied by Clavister have a predefined RAM memory allocation of 1024 Mbytes. This is the minimum amount of memory required for cOS Stream to run and it should never be reduced. This default allocation may need to be increased depending on the license purchased and the number of connections/tunnels that will be open simultaneously. There is no higher memory limit.

If the allocated memory is insufficient during operation, cOS Stream will output console messages indicating this while trying to reduce the number of open connections/tunnels. Eventually, the system will enter safe mode where only management access is possible.

The minimum disk space allocated should be 1 GByte.

Overview of Memory and CPU Core Allocation

By default, cOS Stream will automatically distribute the allocated memory for the control and data planes in a virtual instance. To do this, it uses an algorithm that assigns the majority of the memory to the data plane since this is where the memory is most likely needed to handle flows and other state information that will grow with the amount of throughput. However, in some situations it may be required to change the memory distribution. This can be done by changing the ControlPlaneMemory system setting to the amount of megabytes that should be assigned to the control plane. The remaining memory will be assigned to the data plane. The system must be restarted for the change to take effect.

By default, cOS Stream will also automatically distribute the allocated CPU core resources for the control and data planes by using an algorithm that assigns the majority of CPU cores to the data plane since, in most cases, this plane is handling the majority of traffic. However, in some situations, it may be required to change the CPU core allocation and allocate more cores to the control plane. This can be done by configuring the ControlPlaneCPUCores system setting to be the number of cores to allocate to the control plane. The remaining cores will be assigned to the data plane. The system must be restarted for the changed setting to take effect.

Allocating Sockets, Cores and Threads to a Virtual Machine

The following guidelines should be noted when allocating CPU sockets, cores and threads to the virtual machine running cOS Stream:

Allocating CPU Sockets

cOS Stream only supports running on a single socket. Running on multiple sockets is possible but will degrade performance.

If multiple sockets are configured, cOS Stream will assign its data plane to run on one of the sockets and the control plane will run on the reminder. This then makes it impossible to redistribute CPU resources, either automatically (so the data plane gets most of the resources), or using the cOS Stream configuration setting ControlPlaneCPUCores.
Allocating CPU Cores

It is recommended to assign at least two CPU cores to cOS Stream (although it is possible to run cOS Stream on a single core). Having at least two allows the control plane and data plane to run on separate cores. If higher performance is required, more than two cores can be assigned and the data plane will make use of the additional cores.
Allocating Core Threads

cOS Stream supports one or two threads per core. These are sometimes referred to as hyper-threads or logical cores. However, there is no requirement to use multiple threads since cOS Stream performance is equally good if resources are allocated in the form of CPU cores.

If the underlying hardware has hyper-threading, some optimization of performance is possible if the CPU topology, including threads, is correctly exposed to the virtual machine. However, there is only a gain if the virtual thread pairs match actual hardware thread pairs. Without a match, performance can be degraded instead of improved. When in doubt, it is better to expose all hyper-threads as separate CPU cores rather than exposing them as incorrectly mapped threads.

As a final comment, if cOS Stream is presented with more than two threads per core (meaning an incorrect setup as no CPU has more than two threads per core) then this will give the same result as a multi-socket system. In other words, the data plane will be assigned to run on only one of them and the control plane will run on the remaining.

Chapter 6: Separating VLANs

The Problem

In virtualized installations, there may be a number of virtual machines set up which we want to connect together on a single virtual network. To do this, we define them as belonging to a single port group on a virtual switch with the port group having a particular VLAN ID. The virtual machines now act as though they are connected together on a single virtual network.

Suppose that we now have a second set of virtual machines similarly connected to the same virtual switch through another port group.

All the virtual machines from both groups may also need to communicate with a virtual Clavister Firewall. That firewall would also be connected to the virtual switch through another port group on the switch. This arrangement is illustrated in the diagram below. The boxes labeled "VM" represent the various virtual machines.

Figure 6.1. Connecting VLANs

The virtual switch will normally allow the two groups of virtual machines to communicate with each other. However, what is often required is that they should communicate with each other only through the Clavister Firewall so all traffic can be under the control of cOS Stream.

The VLAN ID Solution

The way to achieve separation is by using unique VLAN IDs for the two groups of virtual machines and a third VLAN for connection to cOS Stream. The diagram below illustrates this arrangement.

Figure 6.2. Separating VLANs

The key points for this solution are:

Each port group for the two groups of virtual machines must be given a unique VLAN ID.

One of the two networks of virtual machines in the illustration is set up to be a VLAN called TestVLAN_15 with the VLAN ID 15. The other is set up to be a VLAN called TestVLAN_16 with the VLAN ID 16. Using different IDs means that the two VLANs cannot communicate with each other.
The virtual machine port group on the virtual switch that connects to the firewall should allow all VLAN IDs to exist in this port group. This is done by specifying the VLAN ID to be 4095 in the infrastructure client (this ID is displayed by the client as the VLAN ID ALL). Only the connected interface of the firewall should exist in this group and this acts as a VLAN trunk (all VLAN IDs can exist on the trunk).
Each VLAN ID on the virtual switch requires a corresponding VLAN Interface object defined for the connecting interface and with the same VLAN ID.

In other words, create one VLAN Interface object for the VLAN TestVLAN_15 with the ID 15 and create a second for VLAN TestVLAN_16 with the ID 16. Both VLAN Interfaces are configured on the interface that connects to the virtual switch. This allows cOS Stream to communicate with these VLANs.

Advantages of the VLAN Approach

The key advantage of this approach of using VLANs is that all traffic flow between the virtual machines occurs inside the virtual network setup and none needs to leave the virtual environment to enter the "real world". This has clear benefits in terms of performance and control.

If Internet access is through the Clavister Firewall then that traffic would obviously leave the virtual environment.

Implementation with VMware

In the VMware infrastructure client, the virtual switch will contain two virtual machine port groups and these are shown in the partial screenshots below. The first port group is for TestVLAN_15:

The second port group is for TestVLAN_16:

VMware References

VMware themselves discuss this approach under the paragraph heading Fully Collapsed DMZ in a VMware document entitled DMZ Virtualization with VMware Infrastructure. The approach is described by VMware as "virtualizing the entire DMZ".

Chapter 7: High Availability Setup

This section provides the information needed to correctly set up a high availability (HA) cluster in a virtualized environment. An HA cluster consists of two Clavister Firewalls, one is the master, the other the slave. Each of these firewalls will run its own separate virtual machine. The interfaces of the two firewalls in a cluster need to be connected together in matching pairs through switches. It is the creation and connection of virtual switches for the cluster that is described in this section.

7.1. HA Setup with VMware

This section provides the extra information needed to correctly set up an HA cluster under VMware, where both Clavister Firewalls in the cluster are running in separate VMware virtual machines.

The initial setup of the two separate Clavister Firewalls is done as normal so they are initially working as separate units. Before performing the setup steps to create the HA cluster, it is first necessary to correctly configure the VMware virtual networking to emulate the hardware connections that would normally be present between the master and slave units.

To achieve this, create VMware separate virtual switches so that the pairs of matching interfaces from the firewalls in the cluster are connected together via a group in a virtual switch. Such switches must be set to operate in promiscuous mode.

In promiscuous mode, interfaces will not ignore a MAC address which is not the MAC address of the interface. Instead, all MAC addresses are recognized and the packets passed to cOS Stream. This is critical in HA since traffic destined for the shared MAC address will be dropped if promiscuous mode is not enabled.

Promiscuous mode is enabled automatically by cOS Stream on physical Ethernet interfaces. However, it must be enabled manually on virtual VMware interfaces since, by default, it is set to the Reject option.

Below is a screenshot which shows the setup in the Configuration section of the VMware infrastructure client for an ESXi server:

Figure 7.1. VMware Virtual Switch Setup with HA

The image shows the setup for virtual switches number 1 to 3. Virtual switch 0 is not shown since this is for the management workstation. The purpose of the 3 virtual switches is described next:

Switch 1

If we look at Switch 1 in the screenshot, there are two groups defined within the switch:

The first is the LAN group which connects the normal networks outside the Clavister Firewall to the LAN interface of the cluster.
The second group is the LAN-Promiscuous group and this connects together the LAN interfaces on the two firewalls. As the group name indicates, this group must operate in promiscuous mode which means that the switch does not use ARP requests to determine which host is found on which interface. Instead, traffic is sent to all connected interfaces.

Figure 7.2. Setting Promiscuous Mode in VMware

Switch 2

The structure of Switch 2 is the same as Switch 1 but this time it is the DMZ interfaces of the two firewalls which are being connected together in the second promiscuous group. The first group, again, is used for connection of external networks which will connect to the firewall via the DMZ interface of the cluster.

Switch 3

Switch 3 is a virtual switch with only one group. This is used to link together the Sync interfaces of each firewall.

7.2. HA Setup with KVM

Open vSwitch Installation

HA setup with KVM requires that Open vSwitch is installed on the Linux system. Open vSwitch will be used to provide virtual switches so that matching interfaces of the master and slave in the cluster can be connected together. The installation of Open vSwitch itself will not be discussed further here. Refer to the software's own documentation for help with installation.

Open vSwitch is open source software that can be used in situations other than high availability to implement various networking solutions with KVM.

	Note: The bridge-utils package must be removed
	Before installing Open vSwitch, the package bridge-utils must be removed from the Linux system.

A Single Physical Server is Assumed

This section assumes that both the virtual firewalls in the HA cluster are installed on the same hardware server. In practice, two servers will probably be used for hardware redundancy and both will have KVM and Open vSwitch installed on them.

The configuration of the connections between two separate servers will not be discussed in this section and it is up to the administrator to choose the most appropriate way of doing this. One approach is to use VLAN tagging with Open vSwitch so internal bridge traffic can pass between the physical servers that make up the HA cluster.

Setup

The initial setup of the two separate virtual firewalls is done as normal so they are initially working as separate firewalls. Before creating the HA cluster, it is necessary to first correctly configure the virtual networking to emulate the hardware connections that would normally be present between the master and slave units.

Configuring Open vSwitch for HA

Assuming Open vSwitch has been installed, it is necessary to create separate virtual switches so that the pairs of matching interfaces from the firewalls in the cluster are connected together on each switch.

This is done with the following steps:

A. Define an Open vSwitch bridge for each interface pair.
B. Configure the HA master and slave virtual machines to connect the interfaces to the relevant bridge.

These two steps are described next.

A. Define an Open vSwitch bridge for each interface pair.

Assuming that all of the default virtual interfaces (if1 and if2) on each firewall are to be connected together, two Open vSwitch bridges must be created:

br1-internal - Connecting together the if1 interfaces. This is created using the Linux command:
```
[root@linux]# ovs-vsctl add-br br1-internal
```
br2-external - Connecting together the if2 interfaces and also connected to the public Internet via a physical interface called eth0 (this name will vary between configurations). This is created using the Linux commands:
```
[root@linux]# ovs-vsctl add-br br2-external
		[root@linux]# ovs-vsctl add-port br2-external eth0
```

B. Connect interface pairs to the relevant bridge.

It is assumed that Virtual Machine Manager (virt-manager) will be used to configure each of the two virtual machines in the HA cluster.

Assume that the interface if1 is to be associated with Open vSwitch bridge br1-internal on both master and slave firewalls. The intuitive approach is to select the NIC entry in the navigation menu that corresponds to the if1 interface and enter the Bridge name:

However, if this is now applied and the virtual machine started, it will give an error message:

To get around this issue, allocate the Open vSwitch bridge using the following steps:

Open the properties of the HA cluster's master firewall in Virtual Machine Manager (virt-manager). Change the Source device to be something using macvtap so that the Type of the Virtual port can be set:

Set the Type to be openvswitch.

Save this setting by selecting the Apply button.

Now, change the Source device setting back to Specify shared device name and set it to be the Open vSwitch bridge connected to the interface. In this case, br1-internal.

Now select the Apply button and repeat the process with the remaining interface if2, connecting it to the bridge br2-external.

Repeat the process for the slave firewall.

	Note: Predefined bridges can simplify the above process
An alternative to the above is much simpler but assumes that the network bridges have already been defined using virsh. If that is the case then the OVS bridges will be present in the virtual Machine Manager drop-down box.

Note: Predefined bridges can simplify the above process

An alternative to the above is much simpler but assumes that the network bridges have already been defined using virsh. If that is the case then the OVS bridges will be present in the virtual Machine Manager drop-down box.

The networking for an HA cluster on a single hardware server is now complete. When the firewalls are on different servers, the procedure is similar. However, the administrator should then decide how they want to connect the Open vSwitch bridges on each server together. VLAN tagging can be used to separate the internal bridges on each server. Each pair of cluster interfaces uses a different VLAN ID to separate its traffic from the other pairs of interfaces.

Chapter 8: SR-IOV Setup

8.1. Overview

cOS Stream supports Single Root I/O Virtualization (SR-IOV). SR-IOV allows a single physical PCI Ethernet interface to be divided into multiple Virtual Function (VF) PCI Ethernet interface devices. When running in a virtual environment, the firewall can be assigned such VF interfaces, allowing it to have direct access to a dedicated part of the physical PCI Ethernet interface.

Just as with PCI passthrough, the direct access provided by SR-IOV can give dramatically higher traffic throughput capability for a Clavister Firewall since it circumvents the overhead involved with normal virtual interfaces.

Prerequisites for SR-IOV

In order to make use of SR-IOV, at least the following is required:

Hardware support for IOMMU and SR-IOV.
Both IOMMU and SR-IOV enabled in the BIOS.
An Intel or NVIDIA ConnectX Ethernet Network device that has SR-IOV capabilities. cOS Stream supports multiple models from both vendors.

Setup of the hardware platform for virtualization is not discussed further here. For details on this subject, please consult the documentation for the relevant hardware platforms, operating systems and/or hypervisors.

Using Assigned MAC Addresses in an HA cluster

By default, an HA cluster will use synthetic MAC addresses on its Ethernet interfaces. However, such MAC addresses may not be allowed by the constraints for the virtual machine. If this is the case, the actual interface MAC addresses must be used instead. This is done by setting the property HAEthernetAddressMode on the relevant EthernetInterface objects to the value InterfaceMAC (the default value is PrivateSharedMAC). For example:

System:/> set Interface EthernetInterface if1
			HAEthernetAddressMode=InterfaceMAC

If this feature is to be used, it is recommended to read the full description of its correct setup in the HA setup section of the separate Administration Guide for cOS Stream.

Achieving Maximum Throughput

Once the SR-IOV interfaces exist as logical interfaces in the system configuration they can be used for both receiving and sending traffic as well as being part of cOS Stream rule sets and other configuration objects.

In order to reach much higher throughput speeds, traffic must both enter and leave the firewall via SR-IOV interfaces. Having the traffic enter or leave on a normal virtual interface will create a bottleneck, reducing throughput back to non-SR-IOV speeds.

8.2. SR-IOV Setup with VMware

The following are the steps for SR-IOV interface setup when running under VMware using the VSphere™ client:

If it is running, stop the virtual machine.

In the vSphere client, select Edit virtual machine settings.

When the properties dialog appears, select the Add function.

In the list of device types, choose PCI Device .

Now, select the PCI device itself.

Select Finish when the addition is complete.

Repeat the above to add an additional PCI device.

Start cOS Stream again and enter the following console CLI command:
```
System:/> ethupdate -cfgupdate
```
The system will scan for the available interfaces and include the added PCI interfaces into the configuration.

Finally, save the configuration changes by entering the following commands:
```
System:/> activate
			
System:/> commit
```

8.3. SR-IOV Setup with KVM

Adding SR-IOV Interfaces

The following are the steps for SR-IOV interface setup:

If it is running, stop the virtual machine.

In Virtual Machine Manager (virt-manager), select Add Hardware.

Select PCI Host Device and the correct virtual function.

The same is repeated if a second PCI device is added.

The new adapters are now listed in virt-manager.

Start cOS Stream again and enter the following CLI command:
```
System:/> ethupdate -cfgupdate
```
The system will scan for the available interfaces and include the added PCI interfaces into the configuration.

Finally, save the configuration changes by entering the following commands:
```
System:/> activate
			
System:/> commit
```

SR-IOV with KVM Troubleshooting

The possibilities of configuration options are large with KVM. However, in certain circumstances the following can solve some common issues:

For VLANs to work with a VF device, VLAN offloading may need to be manually disabled on the physical function (PF) device. This is done by running the following command on the KVM host:
```
> ethtool -K <PF-device> rx-vlan-hw-parse off tx-vlan-hw-insert off
```
For multicast reception to work on the VF device, reception of "all multicast" may have to be enabled on the physical function (PF) device by running the following command on the KVM host:
```
> ip link set dev <pf-device> allmulticast on
```

Chapter 9: Running in OpenStack

The Clavister Firewall can be deployed in an OpenStack environment that uses KVM on Intel x86_64 as nova compute nodes. This section covers deployment and looks at the steps needed using the Horizon implementation of the OpenStack dashboard.

It is assumed that OpenStack has already been installed in a Linux environment and that the Horizon dashboard is also available.

OpenStack Prerequisites

To install the Clavister Firewall with Horizon, the following is required:

A Clavister Firewall image should be imported into OpenStack.
An OpenStack flavor should exist that provides a minimum of 1024 MB ram, at least 1 GByte of disk space and preferably at least 2 vCPUs (although cOS Stream can run with a single vCPU).

Setup Steps

The following steps are required for setup:

A. Create a Security Group.

B. Deploy an instance of cOS Stream.

C. Configure cOS Stream.

These steps will now be described in detail.

A. Create a Security Group

A Security Group needs to be created. This can be done through Horizon but here it is done with the OpenStack Neutron utility using the following steps:

Define the security group:

root@controller:~# neutron security-group-create
		-description 'my security group'
		netguard -security-group

Add the rule:

root@controller:~# neutron security-group-rule-create
		-direction ingress -remote_ip_prefix 0.0.0.0/0
		netguard-security-group

Verify that the group exists:

root@controller:~# neutron security-group-list

B. Deploy an instance

Before launching a new instance, a disk volume should be created from the imported image so it has permanency. This is done in Horizon with the following steps:

Select Volumes under the Compute tab.
Select Create Volume.
Select the imported image as the Volume Source and press Create Volume.
Now, cOS Stream can be started by pressing Launch Instance under the Instances tab.
Select a suitable Name and Flavor and choose Boot from volume as the Boot Source.
Under the Access & Security tab, select the previously create security group called netguard-security-group.
Select which networks to use under Networking.
Press Launch.

C. Configure the system

Once the system is up and running, cOS Stream will prompt for the activate and commit CLI commands to be entered. This will permanently add the detected Ethernet interfaces to the configuration.

Next, the CLI should be used to configure the IP addresses and networks assigned to these interfaces. These should then be visible in Horizon under Instance Overview.

The form of commands for configuring the interfaces can be found in the separate Clavister Firewall Administration Guide.

	Note: Using GRE tunneling in OpenStack
	If GRE tunneling is used in the OpenStack environment, the added overhead for the GRE headers may require a lower MTU size configuration on cOS Stream interfaces. Alternatively, OpenStack may require that jumbo frames are enabled.

Chapter 10: Template Based Deployment

10.1. Overview

Heat implements a method to launch multiple applications in a cloud environment based on coded templates. The template format supported by cOS Stream is the Heat Orchestration Template (HOT) format. This consists of human readable text files.

This chapter describes how to deploy a Clavister Firewall as an OpenStack OS::Nova::Server instance using the HOT format. It assumes a working knowledge of Heat and templates.

10.2. Heat Deployment Steps

This section describes the steps required for Heat deployment in the typical use case.

Actors in the Use Case

The actors assumed to be involved in the deployment are the following:

FW - The Clavister Firewall
USER - The Clavister Firewall & OpenStack administrator
OSE - The OpenStack Environment

Prerequisites

OSE must be configured and running, and the USER must have the correct privileges to instantiate Heat resources and stacks.

Setup Sequence

The following is the sequence required to deploy the FW.

USER first prepares a HOT template file and, optionally, a HOT template environment file to provide input parameter values. These are YAML text files.
USER prepares a FW image file and saves it to OSE internal storage.
USER prepares a FW flavor and saves it to OSE internal storage. The flavor should be a virtual computer template that supports at least the minimum FW system requirements.
USER optionally prepares an SSH RSA key-pair and saves it to OSE internal storage.

The USER also stores their private key locally as part of this step.
USER defines an OS::Nova::Server FW resource in the HOT template file.
USER configures the FW resource to enable the config drive data source (config_drive: true). This is required to make the config drive accessible to the FW resource instance.
USER configures the FW resource to use the FW image file.
USER configures the FW resource to use the FW flavor.
USER configures the FW resource to attach to at least one pre-existing, (or HOT template defined) network, port or subnet.
USER optionally configures the FW resource with a name that the FW will use as its hostname. Note that names longer than 31 characters will be truncated by the FW before assignment.
USER optionally configures the FW resource to use the SSH RSA public key. The FW will use the public key to configure access to the local user administration account with the username admin at deployment.
USER optionally configures the FW resource to use a plain text password. The FW will use the plain text password to configure access to the local user administration account with username admin at deployment.
USER optionally configures the FW resource to supply user-data in the form of a CLI script file. In order for the CLI script to be accepted by the FW, the file must begin with the following first line:
```
#cli-config
```
USER creates a new stack through the OSE using the defined HOT template file and optionally a HOT template environment file. Launching a new stack will launch instances of all resources defined by these templates.

This can be done graphically through the OSE dashboard (Horizon). Alternatively, this can be done through the OSE controller node with the following command:
```
openstack stack create -t <template.yaml> -e <envir.yaml> <stack-name>
```
USER should now review the status of the created stack. This can be done through the OSE dashboard (Horizon). Alternatively, this can be done through the OSE controller node with the following command:
```
openstack stack list
```
USER should also review the console log output of the FW resource instance in order to verify that the deployment was successful. The easiest way to do this is graphically through the OSE dashboard (Horizon).
USER can optionally delete the stack using the OSE. This will shut down all resource instances created when the stack was created. Deleting a stack can be done through the OSE dashboard (Horizon). Alternatively, this can be done through the OSE controller node with the following command:
```
openstack delete <name-of-stack>
```

10.3. Example Template Files

This section list the contents of two example .yaml files that could be used for deployment.

An Example Environment File

The following is an example environment .yaml file.

# Virtual Deployment - Example
parameter_defaults:
  image:                         fw_image
  flavor:                        fw_flavor
  key_name:                      fw_admin_key

  external_net_id:               ext-net
  external_subnet_id:            ext-net-subnet
  external_subnet_static_ip:     10.200.1.4

  dmz_net_id:                    dmz
  dmz_subnet_cidr_v4:            192.168.50.0/24
  dmz_subnet_gateway_ip_v4:      192.168.50.1
  dmz_subnet_static_ip_v4:       192.168.50.22

  dmz2_net_id:                   dmz2
  dmz2_subnet_cidr:              192.168.51.0/24
  dmz2_subnet_gateway_ip:        192.168.51.1
  dmz2_subnet_static_ip:         192.168.51.5

  user_net_id:                   user
  user_subnet_cidr:              192.168.52.0/24
  user_subnet_gateway_ip:        192.168.52.1
  user_subnet_pool_start:        192.168.52.7
  user_subnet_pool_end:          192.168.52.253

An Example HOT File

The following is an example HOT .yaml file that is paired with the preceding environment file.

# Virtual Deployment - Example 
heat_template_version: 2015-04-30
description: Virtual Deployment - Example

parameters:
  image:
    type: string
    default: fw_image
    description: Instance image ID
  flavor:
    type: string
    default: fw_flavor
    description: Instance flavor
  key_name:
    type: string
    default: fw_admin_key
    description: SSH public key

  external_net_id:
    type: string
    description: External network ID
  external_subnet_id:
    type: string
    description: External subnet ID
  external_subnet_static_ip:
    type: string
    description: Fixed IP on external subnet

  dmz_net_id:
    type: string
    description: DMZ network ID
  dmz_subnet_cidr_v4:
    type: string
    description: DMZ subnet IPv4 CIDR
  dmz_subnet_gateway_ip_v4:
    type: string
    description: DMZ subnet gateway IPv4 address
  dmz_subnet_static_ip_v4:
    type: string
    description: FW instance fixed IPv4 address on DMZ subnet

  dmz2_net_id:
    type: string
    description: DMZ2 network ID
  dmz2_subnet_cidr:
    type: string
    description: DMZ2 subnet CIDR
  dmz2_subnet_gateway_ip:
    type: string
    description: DMZ2 subnet gateway IP
  dmz2_subnet_static_ip:
    type: string
    description: FW instance fixed IP on DMZ2 subnet

  user_net_id:
    type: string
    description: User network ID
  user_subnet_cidr:
    type: string
    description: User subnet CIDR
  user_subnet_gateway_ip:
    type: string
    description: User subnet gateway IP
  user_subnet_pool_start:
    type: string
    description: User subnet IP pool start
  user_subnet_pool_end:
    type: string
    description: User subnet IP pool end

resources:
  dmz_net:
    type: OS::Neutron::Net
    properties:
      name: { get_param: dmz_net_id }

  dmz_subnet:
    type: OS::Neutron::Subnet
    properties:
      ip_version:  4
      enable_dhcp: False
      network_id:  { get_resource: dmz_net }
      cidr:        { get_param: dmz_subnet_cidr_v4 }
      gateway_ip:  { get_param: dmz_subnet_gateway_ip_v4 }

  dmz2_net:
    type: OS::Neutron::Net
    properties:
      name: { get_param: dmz2_net_id }

  dmz2_subnet:
    type: OS::Neutron::Subnet
    properties:
      ip_version:  4
      enable_dhcp: False
      network_id:  { get_resource: dmz2_net }
      cidr:        { get_param: dmz2_subnet_cidr }
      gateway_ip:  { get_param: dmz2_subnet_gateway_ip }

  user_net:
    type: OS::Neutron::Net
    properties:
      name: { get_param: user_net_id }

  user_subnet:
    type: OS::Neutron::Subnet
    properties:
      ip_version:       4
      enable_dhcp:      True
      network_id:       { get_resource: user_net }
      cidr:             { get_param: user_subnet_cidr }
      gateway_ip:       { get_param: user_subnet_gateway_ip }
      allocation_pools:
        - start:        { get_param: user_subnet_pool_start }
          end:          { get_param: user_subnet_pool_end }
        
  fw-inst_port0:
    type: OS::Neutron::Port
    properties:
      network_id: { get_param: external_net_id } 
      security_groups:
        - default
      fixed_ips:
        - ip_address: { get_param: external_subnet_static_ip } 

  fw-inst_port1:
    type: OS::Neutron::Port
    properties:
      network_id: { get_resource: dmz_net }
      mac_address: "0a:bc:de:ed:cb:a0"
      security_groups:
        - default
      fixed_ips:
        - ip_address: { get_param: dmz_subnet_static_ip_v4 }

  fw-inst_port2:
    type: OS::Neutron::Port
    properties:
      network_id: { get_resource: dmz2_net }
      security_groups:
        - default
      fixed_ips:
        - ip_address: { get_param: dmz2_subnet_static_ip }

  fw-inst_port3:
    type: OS::Neutron::Port
    properties:
      network_id: { get_resource: user_net } 
      security_groups:
        - default
      fixed_ips:
        - subnet_id: { get_resource: user_subnet } 
        
  fw-instance:
    type: OS::Nova::Server
    properties:
      name:         fw-inst
      admin_pass:   "plaintextpassword"
      config_drive: true
      image:        { get_param: image }
      flavor:       { get_param: flavor }
      key_name:     { get_param: key_name }
      networks:
        - port:     { get_resource: fw-inst_port0 }
        - port:     { get_resource: fw-inst_port1 }
        - port:     { get_resource: fw-inst_port2 }
        - port:     { get_resource: fw-inst_port3 }
      user_data_format: RAW
      user_data: |
        #cli-config
        echo "\r\n --- Configuring SSH remote management if ---\r\n"
        set System Name=assigned_by_userdata
        set RemoteManagement RemoteMgmtSSH RemoteMgmtSSH -enable
        set RemoteManagement RemoteMgmtSSH RemoteMgmtSSH
(line continued)             SourceInterface=if1 SourceNetwork=if1_net
        
outputs:
  fw-inst_external_subnet_ip:
    description: IP addresses assigned to FW for external subnet
    value: { get_attr: [ fw-instance, networks, private, 1 ] }
  fw-inst_dmz_subnet_ip:
    description: IP addresses assigned to FW for DMZ subnet
    value: { get_attr: [ fw-instance, networks, dmz, 0 ] }
  fw-inst_dmz2_subnet_ip:
    description: IP addresses assigned to FW for DMZ2 subnet
    value: { get_attr: [ fw-instance, networks, dmz2, 0 ] }
  fw-inst_user_subnet_ip:
    description: IP addresses assigned to FW for User subnet
    value: { get_attr: [ fw-instance, networks, user, 0 ] }

Successful Console Output

The console output shows the progressive of the deployment. Below is an example of console output from successful deployment of the configuration described in the preceding .yaml files. This output is shortened so it begins immediately after the initial system startup.

Note that some lines in this output have been folded to fit inside the width of the document page.

**************************************************************************
**************************************************************************
Applicable for cloud configuration deployment.
**************************************************************************
**************************************************************************

**************************************************************************
Setup:
**************************************************************************
Created Config Drive mount point at '/os_config_drive'.
Mounted Config Drive on ISO9660 format at '/os_config_drive'.

**************************************************************************
Network Data:
**************************************************************************
Buffered '/os_config_drive/openstack/2015-10-15/network_data.json' contents
Parsed '/os_config_drive/openstack/2015-10-15/network_data.json'
contents into Network Data DOM.
Successfully validated Network Data DOM.
Created ethernet device node 'if1'.
Created ethernet interface node 'if1'.
Configured ethernet interface 'if1' for DHCPv4. { DHCPEnabled = True }
Created ethernet device node 'if2'.
Created ethernet interface node 'if2'.
Created IP address node 'if2_ip'. { Address = '192.168.50.22' }
Configured ethernet interface 'if2' to use address 'if2_ip'.
Created IP address node 'if2_net_v4'. { Address = '192.168.50.0/24' }
Created IP address node 'if2_broadcast'. { Address = '192.168.50.255' }
Configured ethernet interface if2 to use broadcast address 'if2_broadcast'.
Created ethernet device node 'if3'.
Created ethernet interface node 'if3'.
Created IP address node 'if3_ip'. { Address = '192.168.51.5' }
Configured ethernet interface 'if3' to use address 'if3_ip'.
Created IP address node 'if3_net_v4'. { Address = '192.168.51.0/24' }
Created IP address node 'if3_broadcast'. { Address = '192.168.51.255' }
Configured ethernet interface if3 to use broadcast address 'if3_broadcast'.
Created ethernet device node 'if4'.
Created ethernet interface node 'if4'.
Configured ethernet interface 'if4' for DHCPv4. { DHCPEnabled = True }
Successfully configured system using Network Data DOM.

**************************************************************************
Meta Data:
**************************************************************************
Buffered '/os_config_drive/openstack/2015-10-15/meta_data.json' contents.
Parsed '/os_config_drive/openstack/2015-10-15/meta_data.json'
contents into Meta Data DOM.
Successfully validated Meta Data DOM.
Configured System name. { Name = 'fw-inst' }
Created SSHClientKey node 'fw_admin_key'.
{ PublicKey = 'ssh-rsa AAAAB3NzaC1yc2EAAA...'  }
Configuring local user 'admin' with plain-text password.
{ Password = <hidden> }
Configuring local user 'admin' with PSKs. { SSHKeys = 'fw_admin_key' }
Successfully configured system using Meta Data DOM.

**************************************************************************
User Data:
**************************************************************************
Buffered user_data file contents from
'/os_config_drive/openstack/2015-10-15/user_data'.
Created temporary CLI script file '/tmp/cfn-userdata.sgs' from user_data.
==========================================================================
Executing user data CLI script
==========================================================================
cfn-userdata.sgs(2):
echo "\r\n --- Configuring SSH remote management interface ---\r\n"

 --- Configuring SSH remote management interface (example 1) ---
cfn-userdata.sgs(3): set System Name=assigned_by_userdata
Modified System.
cfn-userdata.sgs(4): set RemoteManagement RemoteMgmtSSH RemoteMgmtSSH
                            -enable
Enabled RemoteMgmtSSH/RemoteMgmtSSH.
cfn-userdata.sgs(5): set RemoteManagement RemoteMgmtSSH RemoteMgmtSSH
                            SourceInterface=if1 SourceNetwork=if1_net
Modified RemoteMgmtSSH/RemoteMgmtSSH. Object no longer has errors.
==========================================================================
CLI script execution complete
==========================================================================

**************************************************************************
Activate:
**************************************************************************

Config notice  vsinit:3396  2016-12-01 14:37:23
  - Beginning system reconfigure. Activating new configuration.
Ethernet Devices:
   Name     HW identity string             Autoneg MAC
   -------- ------------------------------ ------- ----------------
                                                   00:00:00:00:00:00 UNUSED
        if1                    pci=00:03.0 X       fa:16:3e:03:9e:68 
        if2                    pci=00:04.0 X       0a:bc:de:ed:cb:a0 
        if3                    pci=00:05.0 X       fa:16:3e:b4:6c:82 
        if4                    pci=00:06.0 X       fa:16:3e:7f:eb:5f 
Ethernet Interfaces:
    if1: on device if1
        MAC:fa:16:3e:03:9e:68   MTU: 1500
        IP: 0.0.0.0   Routing Table membership: <all>
    if2: on device if2
        MAC:0a:bc:de:ed:cb:a0   MTU: 1500
        IP: 192.168.50.22   Routing Table membership: <all>
    if3: on device if3
        MAC:fa:16:3e:b4:6c:82   MTU: 1500
        IP: 192.168.51.5   Routing Table membership: <all>
    if4: on device if4
        MAC:fa:16:3e:7f:eb:5f   MTU: 1500
        IP: 0.0.0.0   Routing Table membership: <all>

Config warning  vsinit:3787  2016-12-01 14:37:24
  - License: Lockdown is in effect because the license is invalid.
    Only access from admin nets to the firewall itself is allowed,
    everything else is dropped.

Config notice  vsinit:301  2016-12-01 14:37:24
  - System: Bootloader configuration will be updated on commit;
    please restart the device at your earliest convenience, after the
    commit, for the changes to take effect. Once the device has been
    restarted please review the output of the 'cfglog' command to see
    if there were any problems with the new configuration.

Config notice  vsinit:3950  2016-12-01 14:37:24
  - Reconfigure completed successfully.

Successfully activated the configuration.

**************************************************************************
Commit:
**************************************************************************
2016-12-01 14:37:24  INTERNAL: The kernel generated a new message
[reason=<7>[   23.948290] ISO 9660 Extensions: Microsoft Joliet Level 3]

Config notice  vsinit:4556  2016-12-01 14:37:24
  - Configuration changes committed.

Successfully committed the configuration.
2016-12-01 14:37:24  INTERNAL: The kernel generated a new message
[reason=<7>[   23.951352] ISO 9660 Extensions: RRIP_1991A]

**************************************************************************
Unmounted Config Drive.
Removed Config Drive mount point.

**************************************************************************
**************************************************************************
Cloud configuration deployment successful.
**************************************************************************
**************************************************************************

10.4. Troubleshooting

An error in the .yaml files used for deployment can result in unsuccessful deployment

There is a template error in the .yaml files used for deployment.
There is an error in the CLI scripting included in the template.

Deployment Troubleshooting

To troubleshoot deployment problems, examine the system console.

Suppose that the set RemoteManagement RemoteMgmtSSH RemoteMgmtSSH line in the previous template was missing the SourceInterface and SourceNetwork properties. The resulting console output would contain lines that look similar to the following.

Activate:
**************************************************************************
Error: In RemoteMgmtSSH RemoteMgmtSSH:
  In property SourceNetwork: A reference to an object is required.
  In property SourceInterface: A reference to an object is required.

ERROR: Unable to activate new configuration.

**************************************************************************
Unmounted Config Drive.
Removed Config Drive mount point.

**************************************************************************
**************************************************************************
Cloud configuration deployment failed!
**************************************************************************

Chapter 11: FAQ

This section collects together answers to a selection of Frequently Asked Questions that can be helpful in solving various issues with cOS Stream running under a hypervisor.

Question Summary

1. Do all my virtual interfaces have to be configured as E1000 NICs?
2. How do I manage multiple virtual firewalls?
3. How do I separate different virtual networks?
4. Where is the Ethernet interface that was added to the virtual machine?
5. How much increase in throughput can SR-IOV provide?

Questions and Answers

1. Do all my virtual interfaces have to be configured as E1000 NICs?

No. The system will work with other virtual interface types. In fact, it is recommended to use the type Virtio with KVM when more than one virtual CPU is available to achieve maximum performance.

However, if only a single virtual CPU core is available with KVM then E1000 should be used for maximum performance. This topic is discussed further at the end of Chapter 4, KVM Virtual Machine Creation.

2. How do I manage multiple virtual firewalls?

The IP address of the management virtual Ethernet interface for cOS Stream must be different for the different virtual firewalls running under a single hypervisor.

3. How do I separate different virtual networks?

If different sets of virtual machines are connected to a virtual switch through different port groups, they can be separated by making them VLANs with different VLAN IDs. This is described further in Chapter 6, Separating VLANs.

4. Where is the Ethernet interface that was added to the virtual machine?

After adding a virtual Ethernet interface to the virtual machine, cOS Stream needs to scan for interfaces before it is added to the system configuration. This is done by entering the following firewall CLI command:

System:/> ethupdate –cfgupdate

This will scan for any new interfaces. The updated configuration must then be deployed using the command:

System:/> activate

Followed by:

System:/> commit

5. How much increase in throughput can SR-IOV provide?

The performance increase provided by SR-IOV can be dramatic if traffic both enters and leaves via SR-IOV interfaces. A quadrupling of maximum throughput is possible.

Appendix A: Supported ConnectX Ethernet Adapters

When using a physical ConnectX Ethernet adapter from NVIDIA® Technologies with virtualized Clavister Firewall, it is important to note that only the models listed below are supported. The recommended firmware version for each model is the officially supported version. While newer firmware versions might work, their compatibility is not guaranteed.

ConnectX®-3 Pro

Part No.	Link Speed	Ports	Firmware
MCX354A-FCC_Ax	40GbE	2	2.42.5000

ConnectX®-4

Part No.	Link Speed	Ports	Firmware
MCX4111A-XCAT	10GbE	1	12.18.1000
MCX4121A-XCAT	10GbE	2	12.18.1000
MCX4111A-ACAT	25GbE	1	12.18.1000
MCX4121A-ACAT	25GbE	2	14.32.0570
MCX4131A-BCAT	40GbE	1	12.18.1000
MCX413A-BCAT	40GbE	1	12.18.1000
MCX415A-BCAT	40GbE	1	12.18.1000
MCX4131A-GCAT	50GbE	1	14.32.0570
MCX413A-GCAT	50GbE	1	12.18.1000
MCX414A-BCAT	50GbE	2	12.18.1000
MCX415A-GCAT	50GbE	2	12.18.1000
MCX416A-BCAT	50GbE	2	12.18.1000
MCX416A-GCAT	50GbE	2	12.18.1000
MCX415A-CCAT	100GbE	1	12.18.1000
MCX416A-CCAT	100GbE	2	12.18.1000

ConnectX®-4 Lx

Part No.	Link Speed	Ports	Firmware
MCX4121A-XCAT	10GbE	2	14.18.1000
MCX4121A-ACAT	25GbE	2	14.18.1000

ConnectX®-5

Part No.	Link Speed	Ports	Firmware
MCX556A-ECAT	100GbE	2	16.32.0570

ConnectX®-5 Ex EN

Part No.	Link Speed	Ports	Firmware
MCX516A-CDAT	100GbE	2	16.32.0570

ConnectX®-6 Lx

Part No.	Link Speed	Ports	Firmware
MCX631432AN-ADAB (OCP3.0)	25GbE	2	26.32.0570
MCX631102AN-ADAT	25GbE	2	26.32.0570

	Important: Firmware versions must be correct
	Note that it is important that each device has the correct firmware version installed.

Appendix B: Troubleshooting Management Connection

This section deals with connection problems that might occur when connecting a management workstation to a Clavister Firewall.

If the management interface does not respond after the Clavister Firewall has powered up, there are a number of simple steps to troubleshoot basic connection problems:

1. Check that the correct interface is being used.

The most obvious problem is that the wrong interface has been used for the initial connection to the management workstation. The management interface is configured during installation and this is described in Chapter 2, Installation.

2. Check that the management workstation IP is configured correctly.

The second most obvious problem is if the IP address of the management workstation is not configured correctly.

3. Using the ifstat CLI command.

To investigate a connection problem further, use the virtual machine's console after cOS Stream starts. When you press the enter key with the console, The system should respond with the standard CLI prompt. Now enter the following CLI command once for each Ethernet interface:

System:/> ifstat <if-name>

Where <if-name> is the name of the management interface. This command will display a number of counters for that interface. The ifstat command on its own can list the names of all cOS Stream Ethernet interfaces.

If the Input counters in the hardware section of the output are not increasing then the error is likely to be in the cabling. However, it may simply be that the packets are not getting to the Clavister Firewall in the first place. This can be confirmed with a packet sniffer if it is available.

If the Input counters are increasing, the management interface may not be attached to the correct physical network. There may also be a problem with the routing information in any connected hosts or routers.

4. Using the arpsnoop CLI command.

A final diagnostic test is to try using the console command:

System:/> arpsnoop all

This will show the ARP packets being received on the different interfaces and confirm that the correct connections have been made to the correct interfaces.

Appendix C: Windows 10/11 IP Setup

If a PC running Microsoft Windows is being used as the management workstation, the computer's Ethernet interface connected to the Clavister Firewall must be configured with an IPv4 address which belongs to the network 192.168.1.0/24 and is different from the firewall's address of 192.168.1.1.

The IPv4 address 192.168.1.30 will be used for this purpose and the steps to set this up with Windows 7 are as follows:

Open the Windows Control Panel.
Select Network & Internet from the control panel.
Then, select the Network & Sharing Center option.
Now, select the Change adapter settings option.
A list of adapters will appear and will include the Ethernet interfaces. Select the interface that will connect to the firewall.
The properties for the selected interface will appear.

Select and display the properties for Internet Protocol Version 4 (TCP/IPv4).
In the properties dialog, select the option Use the following IP address and enter the following values:
- IP Address: 192.168.1.30
- Subnet mask: 255.255.255.0
- Default gateway: 192.168.1.1
DNS addresses can be entered later once Internet access is established.
Click OK to close this dialog and close all the other dialogs opened since step (1).

Appendix D: Apple Mac IP Setup

An Apple Mac can be used as the management workstation for initial setup of a Clavister Firewall. To do this, a selected Ethernet interface on the Mac must be configured correctly with a static IP. The setup steps for this with Mac OS X are:

Go to the Apple Menu and select System Preferences.
Click on Network.
Select Ethernet from the left sidebar menu.
Select Manually in the Configure pull down menu.
Now set the following values:
- IP Address: 192.168.1.30
- Subnet Mask: 255.255.255.0
- Router: 192.168.1.1
Click Apply to complete the static IP setup.