2.7. Maintenance

Home Prev	cOS Core 14.00.17 Administration Guide	Next

2.7.1. Software Upgrades

cOS Core Upgrades Require a Support Agreement

Clavister routinely releases minor and major upgrades to cOS Core software. However, it should be noted that these are only available to customers who have a current cOS Core support agreement.

Every Release Has Release Notes

Every cOS Core upgrade is accompanied by a release notes document which details the new features and changes which the release provides as well as any issues it addresses. The release notes also detail any considerations an administrator should be aware of before proceeding with an upgrade. In general, there should be no problems upgrading an older version of cOS Core to a newer version, but the release notes (and the change log, discussed later) will highlight any that might occur.

cOS Core Components

cOS Core consists of two upgradeable software components: the Core and the Loader. Usually it is the Core which is the main concern of the administrator since this is an executable binary and contains all of cOS Core's principal functional components. The Loader is a low level firmware loader which makes up the interface with the underlying hardware platform and this is updated only with some major revisions of cOS Core.

Figure 2.6. cOS Core Components

The Core Executable

Whenever new functionality is added to cOS Core, or when defects have been found and corrected, a new Core is produced which is the cOS Core executable binary. The new Core is packaged as a single file which is digitally signed and made available for download from the Clavister website https://www.clavister.com.

The Naming of Upgrade Files

In the distribution of software upgrades, a cOS Core upgrade filename can be one of the following types:

The filename will indicate a particular hardware. If there is an upgrade file specific for a particular Clavister hardware model, that file should always be used for upgrades on that model.
An x86 version which should be used for virtual environments such as VMware, KVM or Hyper-V.

The Types of cOS Core Upgrade

The following types of cOS Core releases may be available when performing an upgrade:

Maintenance Releases

These often have bug fixes only but may also contain minor feature additions. They are freely available to all customers who are licensed to run the base version involved in the upgrade. The last two digits in the version number will change for this type of upgrade. For example, version 12.00.00 might become version 11.00.01.

Feature Releases

These usually have both feature additions and bug fixes. They are freely available to all customers who are licensed to run the base version involved in the upgrade. The middle two digits in the version number will change for this type of upgrade. For example, version 12.00.03 might become version 12.01.00.

Major Releases

These involve the addition of major new functionality to cOS Core as well as bug fixes. They are available to customers who have a valid software subscription service contract. The first two digits of the version number will change for this type of upgrade. For example, version 12.mm.nn might become version 13.mm.nn.

	Warning: Firmware Downgrades and Full System Backup
Downgrades between firmware versions are not officially supported, and performing a downgrade is done at your own risk. Objects may be disabled or deleted, and errors may occur during the downgrade process. In some cases, if the configuration in the newer version has changed significantly, the firewall may fail to start. If possible, consider using a full system backup from the older version instead. It is recommended to always save a full system backup before initiating a firmware upgrade.

Warning: Firmware Downgrades and Full System Backup

Downgrades between firmware versions are not officially supported, and performing a downgrade is done at your own risk. Objects may be disabled or deleted, and errors may occur during the downgrade process. In some cases, if the configuration in the newer version has changed significantly, the firewall may fail to start.

If possible, consider using a full system backup from the older version instead. It is recommended to always save a full system backup before initiating a firmware upgrade.

The release types and the numbering system for cOS Core is also discussed in an article in the Clavister Knowledge Base at the following link:

https://kb.clavister.com/324735653

	Caution: The cOS Core license must allow upgrading
	If the New upgrades until date parameter in the cOS Core license has passed so the license is no longer valid, cOS Core will enter lockdown mode following an upgrade and only management access will be possible.

The Upgrade Procedure for Administrators Connected Via a Network

Upgrading to a new version of cOS Core as an administrator connected to a firewall via a network is a simple procedure. A cOS Core upgrade is packaged as a single file with filetype .upg and this can be downloaded first from the Clavister website by logging into the relevant MyClavister user account and searching for the required .upg file. This file is then uploaded to the firewall by using either of the following methods:

In the Web Interface, go to: Status > Maintenance > Upgrade. Then choose the option Upgrade Unit's Firmware to select and upload the relevant .upg file.
Upload the file using Secure Copy (SCP). When cOS Core receives the file, it automatically recognizes that it is an upgrade file.

When the upload of a .upg file is complete using any of the above methods, cOS Core will automatically initiate a full system restart. Any live traffic will therefore be interrupted and connections lost while this takes place.

	Important: Change the default password before upgrading
It is strongly recommended that the default password of the admin user is changed before performing an upgrade. The reason for this is that in some circumstances the administrator can become locked out if the default password is not changed and the new version requires an admin password that conforms to the strong password policy. The alternative is to disable strong password enforcement before upgrading, in which case the default admin password does not need to be changed.

Important: Change the default password before upgrading

It is strongly recommended that the default password of the admin user is changed before performing an upgrade. The reason for this is that in some circumstances the administrator can become locked out if the default password is not changed and the new version requires an admin password that conforms to the strong password policy.

The alternative is to disable strong password enforcement before upgrading, in which case the default admin password does not need to be changed.

Upgrading cOS Core Using InControl

When one or more firewalls are being managed using the Clavister InControl product, cOS Core upgrades can be done by creating an upgrade job in the InControl client. These jobs allow large numbers of firewalls to be upgraded at one time, with the upgrade optionally performed automatically at a nominated time. This can significantly simplify the upgrading of a small or large firewall population and is described further in the Upgrading Devices section of the separate InControl Administration Guide.

The Web Interface Change Log

When applying an upgrade in the Web Interface, if there are important upgrade issues that the administrator should be aware of then a special page will be displayed by cOS Core before it begins the update process. This page is called The Change Log.

The issues highlighted by the change log will also appear in the release notes for the new version but being reminded of them immediately before the upgrade begins can be useful. The change log might include such potential problems as incompatibility with previous cOS Core versions and it may not appear at all if an upgrade does not have any issues.

Taking A Full System Backup Before Upgrading is Recommended

It is recommended to make a full system backup before performing a system upgrade. If there is a requirement to later reverse the upgrade, this should be done by performing a full system restore to the original cOS Core version and the original configuration.

Causes of Upgrade Failure and Warning Messages

An upgrade might fail in which case cOS Core will show an explanatory message in the Web Interface. The reasons for failure can be one of the following:

Invalid .upg file.
Incorrect file checksum.
Incorrect file signature.
Incorrect .upg type.
Incorrect file for the platform.

In addition to error messages, the Web Interface can also generate warnings about the following potential problems:

Downgrading to an earlier version. This should not be done unless completely necessary.
The license subscription expiry date does not cover upgrading to the new version. This is important because cOS Core will go into lockdown mode after an update if the license does not allow it, where only management access will be possible.

Upgrades in a Virtual Environment

When running cOS Core in a virtual environment under VMware, KVM or Hyper-V, upgrades of cOS Core can be done just as they are performed on a single physical computer. It is not necessary to create a new virtual machine for a new version.

Note: Leave Dynamic High Buffers enabled

	Note: Leave Dynamic High Buffers enabled
It is recommended that the advanced setting Dynamic High Buffers is always left enabled (its default value) after an upgrade since the cOS Core memory allocations may have changed. A hardware restart is required for a change in the setting to take effect and this can be achieved using the CLI command: `Device:/>` shutdown In some scenarios support personnel might recommend disabling Dynamic High Buffers and setting High Buffers to a higher fixed value. Where cOS Core is handling tens of thousands of simultaneous connections then it may be necessary to manually set a value above the automatic value. However, if the HighBuffers value is set too high, this may adversely affect throughput. See Section 13.10, Miscellaneous Settings for a full explanation of these settings.

It is recommended that the advanced setting Dynamic High Buffers is always left enabled (its default value) after an upgrade since the cOS Core memory allocations may have changed. A hardware restart is required for a change in the setting to take effect and this can be achieved using the CLI command:

Device:/> shutdown

In some scenarios support personnel might recommend disabling Dynamic High Buffers and setting High Buffers to a higher fixed value. Where cOS Core is handling tens of thousands of simultaneous connections then it may be necessary to manually set a value above the automatic value. However, if the HighBuffers value is set too high, this may adversely affect throughput.

See Section 13.10, Miscellaneous Settings for a full explanation of these settings.

Checking Memory After an Upgrade

A cOS Core upgrade sometimes introduces new features which will increase the system memory requirements. This can lead to performance issues if the amount of free memory becomes constrained. It is therefore advisable to check the amount of available free memory after a major upgrade which introduces new features. Configuration adjustments may be advisable if the amount of free memory during operation with live traffic seems too low (for example, by reducing the total number of connections allowed or reducing the number of configured VPN tunnels).

The issue of memory and performance is also discussed in Section 3.4.2.3, Changing RX and TX Ring Sizes. The description of the Dynamic High Buffers setting found in Section 13.10, Miscellaneous Settings also provides further information about memory allocation.

Downgrading cOS Core

As mentioned previously in this section, downgrading to an earlier cOS Core should be done by restoring a full cOS Core system backup. For this reason, creating a full system backup is recommended before a cOS Core version upgrade.

Downgrading by installing an older cOS Core version in the form of a .upg file (leaving the original configuration unchanged) is not supported as there may be differences in the newer cOS Core version that make the configuration incompatible with an older version. The results of such a mismatch are unpredictable.

New Release Notifications

Notifications of new cOS Core releases are sent out to the email address associated with MyClavister accounts. Email preferences can be adjusted by choosing the Settings option after logging into the relevant account at the following URL:

https://my.clavister.com

2.7.2. Version Update Alerts

Overview

cOS Core can optionally receive information from Clavister servers related to available upgrades. This information is displayed to the administrator as alerts in the Web Interface. This informs the administrator that upgrades are available. This feature is enabled by default and must be disabled manually by the administrator, as shown in the example at the end of this section.

When enabled, alerts of available upgrades appear only in the Alerts portion of the cOS Core Web Interface toolbar.

Communication with Clavister Servers is Encrypted and Periodic

Communication between cOS Core and Clavister servers is encrypted and occurs at the following times:

Shortly after system startup.
Once per day following startup.

Required Prerequisites

This feature will only function if all of the following are true:

cOS Core is operating with a valid license.
cOS Core has access to the Internet.
The feature has not been disabled by the administrator.

Log Event Message Generation

A log message is generated when an alert is generated for the administrator that version upgrades are available. This message has a severity level of Notice and the log event name new_firmware_available.

Example 2.39. Disabling cOS Core Release Notifications

This example shows how to disable the diagnostics and quality improvements feature.

Command-Line Interface

Device:/> set UpdateCenter CheckForPatchReleases=No
			CheckForFeatureReleases=No

InControl

Follow similar steps to those used for the Web Interface below.

Web Interface

Go to: Status > Maintenance > Update Center
Disable the setting Maintenance Releases
Disable the setting Feature Releases
Click OK

2.7.3. Auto-Update Mechanism

A number of cOS Core security features rely on external servers for automatic updates and content filtering. The Intrusion Prevention and Detection subsystem and the Anti-Virus subsystem require access to updated signature databases in order to provide protection against the latest threats.

To facilitate the Auto-Update feature Clavister maintains a global infrastructure of servers providing update services for Clavister firewalls. To ensure availability and low response times, cOS Core employs a mechanism for automatically selecting the most appropriate server to supply updates.

For more details on these features see the following sections:

All these subsystems require that the customer has a current valid Clavister subscription agreement for the features to work and for the associated databases to be automatically updated. This is discussed further in Appendix A, Subscription Based Features

2.7.4. Backing Up Configurations

The administrator has the ability to take a snapshot of the cOS Core system at a given point in time and restore it later, if necessary. The snapshot can be of two types:

A Configuration Backup Copy

This is a copy of the current cOS Core configuration saved into a single file. It does not include the installed cOS Core version. This copy can then be used later for restoring only the configuration.

It is important to create, at a minimum, a configuration backup on a regular basis so that a configuration can be easily recreated in the event of replacement of the hardware platform. The alternative is to recreate a configuration by manually adding its contents, piece by piece.

Note that it also possible to create an anonymous configuration copy where sensitive data is automatically replaced by randomly generated data. This is discussed further in Section 2.6.11, Creating an Anonymous Configuration Copy.
A System Backup Copy

This a complete backup of both the configuration and the installed cOS Core software saved into a single file. This is useful if restoring both the configuration and the cOS Core version upgraded.

A complete system backup is a much larger file than a configuration backup and can be many megabytes, it is therefore not recommended to perform this on a regular basis because of the time involved. However, it is recommended to create this at least once when the Clavister firewall is initially configured and goes live. This is because having a full system backup will make it easier to restore the current configuration on replacement hardware.

	Note: Backups do not contain everything
	Backups include only static information from the cOS Core configuration. Dynamic information such as the DHCP server lease database or Anti-Virus/IDP databases will not be backed up.

Version Compatibility

Since a full system backup includes the full cOS Core software version, compatibility is not an issue with these types of backup.

With configuration only backups, the following should be noted:

A configuration backup created on a higher cOS Core version should never be uploaded to a lower cOS Core version. For example, a backup created from a system running version 11.20.00 of cOS Core should never be uploaded to a system running version 11.10.02.
A configuration backup created on a lower version can always be uploaded to a higher version, however there can be compatibility issues in certain cases which will be indicated by messages from cOS Core when the uploaded configuration is activated. Such issues can also result in a restart of cOS Core.

For this reason, a full system backup is useful when trying to transfer a saved configuration to new hardware where the new hardware comes preloaded with a higher cOS Core version. First, upload the full system backup to get a system with the right version and then upload the latest configuration backup. If there is a requirement to move to a higher cOS Core version, an upgrade can then be performed.

The Management Interfaces Used

Both types of backup, configuration and system, can be performed either by downloading the file directly from the firewall using SCP (Secure Copy) or alternatively using the Web Interface. Backup cannot be done using CLI commands.

Similarly, restoring a backup is done in the reverse fashion. Either by uploading the backup file using SCP or alternatively through the Web Interface. A restore cannot be done with CLI commands.

	Warning: Do not upload a system backup to dissimilar hardware
Do not try to upload a full system backup (configuration plus cOS Core) to a hardware platform that is not the same type as the original. This will cause the configuration to be automatically activated and cOS Core rebooted, with the possibility that cOS Core becomes unreachable. Upload of full system backups must be done to similar hardware since there is no opportunity to change the configuration before it is activated.

Warning: Do not upload a system backup to dissimilar hardware

Do not try to upload a full system backup (configuration plus cOS Core) to a hardware platform that is not the same type as the original.

This will cause the configuration to be automatically activated and cOS Core rebooted, with the possibility that cOS Core becomes unreachable. Upload of full system backups must be done to similar hardware since there is no opportunity to change the configuration before it is activated.

Operation Interruption

Backups can be created at any time without disturbing cOS Core operation. For restores, however, it is not recommended to have live traffic flowing since the restored configuration may significantly alter the security policies of the firewall.

After restoring a backup it is necessary to Activate the new configuration, as is done with a normal configuration change.

A complete system restore will require that cOS Core reinitializes, with the loss of all existing connections. Initialization may require some seconds to complete, depending on the hardware platform, and normal operation will not be possible during this time.

Backup and Restore using SCP

There are two files located in the cOS Core root directory which are created when a download operation commences. These files have the following names:

config.bak - This is the backup of the current configuration.
full.bak - This is the backup of the complete system.

SCP can be used to download either of these files. When the download is complete, the administrator should alter the filename manually to include the date of the download.

To restore a backup file using SCP, the administrator should upload the file to the Clavister firewall. The name of the file does not need to be changed in any way since cOS Core will read a header in the file to determine that it is a backup file. Any SCP upload of a backup file will cause that file's contents to be restored.

Backup and Restore using the Web Interface

As an alternative to using SCP, the administrator can initiate a backup or restore of the configuration or complete system through the Web Interface. When the Web Interface is used, cOS Core automatically names the two files listed above to include the current date. For example, the full.bak might become full-20210906.bak to show it is a snapshot of the state on September 9th, 2021. The example below illustrates using the Web Interface.

Example 2.40. Downloading a Complete System Backup

In this example we will back up the entire system.

InControl

Backing up the system with InControl is done in a different way to the Web Interface

Web Interface

Go to: Status > Maintenance > Backup and Restore > Backup System
A file dialog is shown - choose a directory for the created file and change the filename from the default if desired.
Download of the backup file will then start

	Tip: Examining configuration backup files
It is possible to open a configuration only .bak file in a normal text editor and examine its contents although the file will contain some binary information which means it cannot be modified and saved. Furthermore, a configuration .bak file only shows object property values that are different from the default values. Therefore, the best way to examine the configuration in a .bak file is to load it into cOS Core and use the Web Interface to view it without saving and activating it.

Tip: Examining configuration backup files

It is possible to open a configuration only .bak file in a normal text editor and examine its contents although the file will contain some binary information which means it cannot be modified and saved.

Furthermore, a configuration .bak file only shows object property values that are different from the default values. Therefore, the best way to examine the configuration in a .bak file is to load it into cOS Core and use the Web Interface to view it without saving and activating it.

2.7.5. Restore to Factory Defaults

A restore to factory defaults can be applied so that it is possible to return to the original system state when cOS Core was first installed. When a restore is applied, data such as the IDP and Anti-Virus databases as well as pcapdump files are not deleted. These must be explicitly removed by using the appropriate command. The license file is also not deleted.

Example 2.41. Complete Reset to Factory Defaults

Command-Line Interface

Device:/> reset -unit

InControl

This operation cannot be performed with InControl.

Web Interface

Go to: Status > Maintenance > Reset & Restore > Reset
Select Restore the entire unit to factory defaults then confirm and wait for the restore to complete.

	Important: Any upgrades will be lost after a factory reset
	It should be understood that a reset to factory defaults is exactly that. Any cOS Core upgrades performed since the unit left the factory will be lost.

Resetting to the Default cOS Core Configuration

An alternative to a complete factory reset is only resetting the firewall to its default configuration. This means that only the cOS Core configuration is reset to its factory defaults state, not the underlying hardware.

Resetting to the default configuration can be done through the CLI or Web Interface or InControl. When using the CLI, the command to do this would be:

Device:/> reset -configuration

Resetting only the default software configuration is important when preparing a firewall for the zero touch feature in the InControl management software. Zero touch is explained further in the separate InControl Administration Guide.

Reset via the Console Boot Menu

Connect via the local console port on the firewall using a console emulator on a computer or other device. Restart the firewall and press a key when the "Press any key to abort and load boot menu" message appears on the console. When the console boot menu appears, select the hardware reset option, then confirm and wait for the process to complete. Using the boot menu is described in Section 2.1.9, The Local Console Boot Menu. Using the boot menu is described in Section 2.1.10, Boot Menu for the NetWall 100/300/500/6000 Series.

	Warning: Do NOT abort a reset to defaults
	If the process of resetting to factory defaults is aborted before it finishes, the firewall can then cease to function properly with the complete loss of all stored user data.

Interface IP Addresses are Reset

Resetting to defaults will reset the management interface and IP address for management access to the default.

Apart from the management interface, the operation will also reset the IPv4 address of all other Ethernet interfaces to be 127.0.x.1 in the network 127.0.x.0/24. The value "x" is a number that starts with "1" and increases by one for each consecutive interface, skipping the management interface.

The IPv4 address 127.0.0.1 is not used since this is reserved for the loopback interface.

End of Life Procedures for Hardware Products

The restore to factory defaults option should also be used as part of the end of life procedure when Clavister firewall hardware is taken out of operation and will no longer be used. Returning a hardware device to its default state will overwrite all sensitive information such as VPN settings.

As a further precaution at the end of a hardware product's life, it also recommended that the physical memory media in a hardware device is destroyed and certified as destroyed by a suitable provider of computer disposal services.

2.7.6. Listing and Adding Ethernet Interfaces

The CLI command pciscan is a tool that is used in listing and upgrading the Ethernet ports for the Clavister firewall. It is usually relevant only for non-Clavister hardware although it can be used for examining the PCI hardware in Clavister devices.

Checking Ethernet Interfaces

If the pciscan command is used without parameters then a list of all the supported Ethernet interfaces in the firewall is obtained:

Device:/> pciscan
		
Idx VendorID DeviceID Bus  Slot IRQ Driver Info
012 0x8086   0x1010    2    4   0 "E1000"  Intel NETWORK - ETHERNET
013 0x8086   0x1010    2    4   1 "E1000"  Intel NETWORK - ETHERNET
020 0x8086   0x1012    5    4   0 "E1000"  Intel NETWORK - ETHERNET
021 0x8086   0x1012    5    4   1 "E1000"  Intel NETWORK - ETHERNET
022 0x8086   0x1012    6    4   0 "E1000"  Intel NETWORK - ETHERNET
"
"

Each line in the output shows the current configuration for each Ethernet interface.

To see all Ethernet interfaces on the firewall, regardless if they are supported or not, the relevant command is:

Device:/> pciscan -ethernet

To see all hardware on the PCI bus, both Ethernet and non-Ethernet, the command is:

Device:/> pciscan -all

Automatic Recognition of New Interfaces in Virtual Environments

If a new Ethernet interface is added to the underlying platform in a virtual environment such as KVM, then this can be detected by cOS Core and the configuration updated automatically using the following command:

Device:/> pciscan -cfgupdate

However, this command is only relevant to cOS Core running in a virtual environment. It is not relevant to Clavister hardware products and will result in an error message if used.

Forcing the Choice of Driver

When using non-Clavister hardware, there may be PCI cards installed that require a driver that cannot be automatically selected using the -cfgupdate option. In such a case the administrator can explicitly choose the driver from a list using the -force_driver option.

The index number of the PCI card is first identified from the output of the pciscan -all command. If the card number index is found to be 7 and the driver to choose is E1000 then the command is:

Device:/> pciscan -force_driver 7 E1000

The command offers tab completion for the available driver names so it is not necessary to know them in advance. It may be the case that a process of trial and error is required to identify the correct driver for the card.