InCenter 2.2.5 On-Premises Administration Guide


Table of Contents

1. Overview
2. Installation
2.1. Virtual Machine Resource Requirements
2.2. Installation Steps
3. InCenter Access
3.1. SSH Access to the CLI
3.2. REST API Access
3.3. WebUI Access
3.4. Access Certificates
3.5. File Transfer
3.6. SSH Server Management
4. Using the CLI
4.1. Tab Completion
4.2. Specifying String Values
4.3. Displaying Unsaved Changes
4.4. Activating Changes
4.5. Selective Activate
4.6. Viewing Node Status
4.7. CLI Relay
5. Using the WebUI
5.1. Starting the WebUI
5.2. Activating InCenter Changes
6. Managing NetWall Nodes
6.1. Overview
6.2. Setting Up Centralized Management
6.2.1. NetWall Node Centralized Management Setup
6.2.2. InCenter Centralized Management Setup
6.2.3. Adding NetWall Nodes with the CLI
6.3. Setting Up Monitoring
6.3.1. Adding NetWall Nodes with the WebUI
6.3.2. Configuring a Log Receiver in NetWall Nodes
6.4. Node Dashboards
6.4.1. Overview Dashboard
6.4.2. Network Allowed
6.4.3. Network Denied
6.4.4. Threats Unhandled
6.4.5. Threats Denied
6.4.6. VPN
6.4.7. Communication
6.4.8. OneConnect
6.4.9. Log Analyzer
6.5. The Health Display
6.6. CyberSecurity Score
6.7. Reporting
6.8. Alerting
6.9. SMTP Server Setup
6.10. Internet Proxy Setup
6.11. Upgrading Nodes
6.12. Device Monitoring
6.13. NetEye Monitoring
6.14. EasyAccess Monitoring
7. Managing NetShield Nodes
7.1. Adding Nodes with the CLI
7.2. Adding Nodes with the WebUI
7.3. Editing Node Properties with the WebUI
7.4. Deleting a Node
7.5. Using Public Key Authentication
7.6. Changing a Node's Management IP
7.7. Editing Node Configurations
7.7.1. Editing Configurations with the CLI
7.7.2. Editing Configurations with the WebUI
7.8. Deleting Configuration Objects
7.9. Host Key Management
7.10. Replacing a Standalone Node
8. Managing HA Pairs
8.1. Adding an HA Pair
8.1.1. Adding an HA Pair with the CLI
8.1.2. Adding an HA Pair with the WebUI
8.2. Editing an HA Pair
8.3. Deleting an HA Pair
8.4. Replacing an HA Pair Node
8.5. Upgrading an HA Pair
9. Managing Users
9.1. Managing Users with the CLI
9.2. Managing Users with the WebUI
9.3. User Password Policy
9.4. Setting Up MFA
10. Centralized Management Control
10.1. Overview
10.2. Disabling Centralized Management Control
10.3. Re-enabling Centralized Management Control
10.4. Node/Firewall Synchronization
11. Version Management
12. Revision History
13. License Management
13.1. License Handling
13.2. License Monitoring
13.3. License Replacement
14. OpenStack Deployment
15. Node Groups
15.1. Managing Groups with the CLI
15.2. Managing Groups with the WebUI
16. Configuration Templates
17. InCenter Administration
17.1. Host OS CLI Access
17.2. Changing the Host SSH Keys
17.3. Restarting InCenter
17.4. System Log Files
17.5. Downloading Log Files
17.6. Expanding Log Storage
17.7. Displaying Uptime
17.8. System Backup and Restore
17.9. Scripts
17.10. Upgrading the System
17.11. The techsupport Command
18. Log Servers
18.1. The Internal Log Server
18.2. Using External Log Servers
18.3. Using Syslog Servers
18.4. Log Migration to a New Instance
19. Redundancy Setup
20. IPS
20.1. Overview
20.2. Setting Up IPS
20.3. IPS Signature Management
20.4. SNORT File Usage
20.5. Best Practice Deployment
21. IDP
21.1. Overview
21.2. Setting Up IDP
21.3. Best Practice Deployment
A. Third Party Software

Chapter 1: Overview

[Note] Note: This document is also available in other formats

A PDF version of this document along with all current and older documentation in PDF format can be found at https://my.clavister.com.

It is also available in a framed HTML version.

The InCenter product is an administrative tool for the centralized monitoring and management of one or many Clavister Next Generation Firewalls. It also has the ability to provide monitoring of a number of other separate Clavister products.

InCenter is a Client/Server System

The InCenter software runs under the Linux operating system and comes with a 64-bit Ubuntu Linux distribution (referred to as just Linux in this publication). This is packaged as a single hypervisor image which can be loaded directly into an on-premises host server's virtual environment.

InCenter runs as a client/server system, as illustrated below.

Firewalls Under InCenter Control

Figure 1.1. Firewalls Under InCenter Control

The diagram above shows a single InCenter management client, the InCenter server and three firewalls communicating over a network. The client and InCenter server could be situated on the same computer and the network could be the public Internet.

The administrator communicates with the InCenter server through a network connection from a management computer. Administration tasks can be performed using the command line interface (CLI) via an SSH client console. Alternatively, a web user interface (WebUI) via a standard web browser can be used.

The InCenter server acts as a central management point, allowing the administrator to deploy configuration changes to multiple Clavister Next Generation Firewalls at the same time and keeping synchronized copies of their current configurations in a database on the server.

The Term "Node"

Note that in the InCenter context, each firewall under InCenter control will often be referred to as a "node". The term "device" is reserved for external user equipment such as smartphones, laptops and tablets.

The REST API

As an alternative method of management, custom software running on an external computer can interact with the InCenter server using the InCenter REST API. This is described further in Section 3.2, REST API Access.

InCenter Monitoring

The InCenter WebUI provides an extensive ability to display graphical monitoring of firewalls, regardless if they are under centralized control by InCenter or not.

In addition, this monitoring feature can be extended to non-firewalling Clavister products. These include:

  • Clavister NetEye.

  • Clavister EasyAccess.

Chapter 2: Installation

This section describes the installation of the InCenter software distribution on a server running under a suitable hypervisor.

2.1. Virtual Machine Resource Requirements

The following are the resource requirements of the virtual machine which runs InCenter.

A. Minimum installation resource requirements:

  • A minimum of 8 Gbytes of RAM.
  • A minimum of 40 Gbytes of disk space.
  • A minimum of 2 processor cores.

The above should be sufficient for:

  • 1-50 nodes.
  • 1-1,000 IP rule set entries per node.
  • Log receive rates of 100-1,000 logs/second.

B. Medium sized installation resource requirements:

  • A minimum of 8 Gbytes of RAM.
  • A minimum of 60 Gbytes of disk space.
  • A minimum of 2 processor cores.

The above should be sufficient for:

  • 50-1,000 nodes.
  • 1-1,000 IP rule set entries per node.
  • Log receive rates of 1,000-2,000 logs/second.

C. Large size installation resource requirements:

  • A minimum of 16 Gbytes of RAM.
  • A minimum of 120 Gbytes of disk space.
  • A minimum of 8 processor cores.

The above should be sufficient for:

  • 1,000-10,000 nodes.
  • 1,000-10,000 IP rule set entries per node.
  • Log receive rates of 10,000 logs/second.

Other Disk Space Considerations

The disk space requirement can be larger depending on how the following features are configured and used:

  • Revision history retention - The revision history may allow large number of entries to be stored. This topic is described further in: Chapter 12, Revision History.

  • Site license monitoring - License violations are stored persistently. These are described further in Chapter 13, License Management.

  • Log Server - The default enabled log server has retention enabled and is configured to use at most 10 Gbytes of disk size. Changing the log server settings so that the server stores more log data may also require extra disk space.

2.2. Installation Steps

The installation steps are as follows:

  • Load the appropriate InCenter virtual machine image into a virtual environment to create a new virtual machine.

  • The virtual machine will have one virtual Ethernet interface. Make sure that DHCP is enabled in the virtual environment for that interface so that it can automatically be assigned an IP address.

  • Start the virtual machine.

  • The virtual machine console will show the initialization of the base host operating system followed by the initialization of the InCenter server. When this is complete, the console will show the host operation system command prompt. Note that the InCenter system CLI is not accessible through the host operating console. It can only be accessed using SSH.

    Should access to the host operating system be required through the virtual machine console, the default username administrator and password administrator should be used.

  • The IPv4 address assigned to the virtual machine's interface will appear in the virtual machine's console, usually in the login prompt. Make a note of this IP address. This will be used for both administrator login to InCenter as well as the underlying host operating system. It is not necessary to log into the virtual machine in order to use InCenter.

  • Connect the Ethernet interface of a separate management computer to the virtual Ethernet interface of the virtual machine. This can be done across a local network.

  • The administrator can now connect to the InCenter CLI using a Secure Shell (SSH) client. This requires the management IP address noted earlier from the virtual machine console (described above). The port used will be the standard SSH port number of 22 (this number cannot be changed).

    Alternatively, connection can be made through a standard web browser over HTTPS to the management IP address using port 443.

  • The following default credentials are used for logging into InCenter:

    • Username: admin

    • Password: admin

    The default password should be changed by the administrator at the earliest opportunity to increase security.

Setting System Time

InCenter system time is taken from the host operating system. The host operating system has an NTP client which is enabled by default and the NTP client cannot be controlled via the InCenter system.

DHCP client IDs are Unique

All InCenter installations will generate their own unique DHCP client ID following installation. This means that there should be no confusion by DHCP servers when providing DHCP leases to InCenter.

Chapter 3: InCenter Access

This chapter describes how the InCenter server can be accessed for InCenter system management (for host OS access see Section 17.1, Host OS CLI Access). The access methods available for the InCenter system are the following:

  • SSH access to the InCenter CLI.

  • InCenter WebUI access using a web browser.

  • REST API access by external software.

  • SCP file transfer.

  • SFTP file transfer.

The above methods are discussed in detail in the sections that follow.

As mentioned previously in Chapter 2, Installation , it should be noted that the InCenter system CLI cannot be accessed via the virtual machine console. Only the host operating system CLI can be accessed in this way. InCenter CLI access is only possible using SSH.

Changing the Management IP Address and Using IPv6

The IP address used for connection to InCenter can be changed using a host operating script called staticIP.sh and this script is provided in the InCenter software installation package. Using the script is described in Section 17.9, Scripts.

Wherever IPv4 addresses are used in InCenter, IPv6 addresses can be used instead. This is true of management access. However, IPv6 addresses can only be used if the related Ethernet interface is allocated an IPv6 address using the staticIP.sh script.

3.1. SSH Access to the CLI

To access the InCenter CLI, use an SSH client (several third party clients are available, both open source and commercial) and connect to the InCenter server.

[Note] Note: Supported SSH and SFTP clients

InCenter supports the following client versions for SSH and SFTP access:

  • PuTTY versions 0.61 to 0.64.

  • OpenSSH versions 6.6 and 7.3.

Initial SSH Access Using Default Credentials

The initial default username and password for the InCenter CLI are always admin and admin (note that these are different with host OS access which is described in Section 17.1, Host OS CLI Access). It is recommended that the password for the admin user account is changed as soon as possible by the administrator.

The IP address for SSH access is always displayed in the console of the virtual machine when InCenter starts and the port number for access is always the standard SSH port number of 22 (this number cannot be changed).

Assume that the management IP address is 192.168.98.14. A typical login sequence using an SSH client console would be the following:

login as: admin
admin@192.168.98.14's password: admin
------------------------------------------------------------
Welcome to the InCenter CLI
Logged in as: Default admin user
------------------------------------------------------------
admin@InCenter:/>

The final prompt indicates that the InCenter CLI is now available. The first thing to do is add a Clavister Next Generation Firewall so that it becomes controlled by InCenter. This is explained in the next section.

[Note] Note: The admin password must be changed

After logging in for the first time and making any configuration change, the default system behavior is that the system will require that the admin password is changed to a stronger password before allowing a second login. This behavior is explained further in Section 9.3, User Password Policy.

Customizing SSH Parameters

It is possible to customize the parameters for SSH connection through the CLI. For example, to change the SSH timeout, the commands would be the following:
admin@InCenter:/> cc Settings  
admin@InCenter:/Settings> set SSH SessionTimeout=60

The following SSH properties can be changed in the same way:

  • Enabled - If SSH based management access is allowed. Default value: True

  • SessionTimeout - Number of minutes of inactivity before a timeout. Default value: 20

  • MaxSimultaneousSessions - Simultaneous sessions allowed. Default value: 20

  • AllowCLI - Allow SSH CLI access. Default value: Yes

  • AllowSCP - Allow SCP transfers. Default value: Yes

  • AllowSFTP - Allow SFTP transfers. Default value: Yes

Customizing other aspects of SSH access, such as changing the host key, is discussed in Section 3.6, SSH Server Management.

SSH Access Using Public Key Authentication

Instead of using username and password credentials, it is possible to set up SSH access to InCenter using public key authentication. This means that a correctly configured SSH client could log into the InCenter CLI automatically and also perform SFTP and SCP operations to InCenter without needing credentials to be entered.

To set this up in InCenter, the following steps are required:

  1. Upload the public key for authentication to InCenter using SCP or SFTP.

  2. Log in to the InCenter CLI using username and password credentials.

  3. The key file for SSH authentication should be in openssh format. Open this file and copy the public key to the system clipboard ready for pasting into the InCenter CLI.
  4. Now, use the user command to set the key. The -id parameter specifies the name of the user and -publicKeys is set to a comma separated list of one or more ssh keys (copied from the openssh public key file in the previous setep) in one set of quotes. For example:

    admin@InCenter:/> user -update -id=admin -publicKeys="<public-key>"
  5. Activate the change.

  6. Set up the SSH client for public key authentication using the private key file that matches the public key uploaded in the first step.

3.2. REST API Access

Overview

InCenter provides the ability for custom software running on an external computer to communicate with the InCenter server using a predefined REST API. The REST API supports full system configuration and administrative operations.

The API itself is documented in a separate REST API Guide which is provided as part of each InCenter release.

Like normal SSH access, InCenter access using the REST API requires authentication with the credentials of a User object. Setting up users is described in Chapter 9, Managing Users.

Changing REST API Properties

It is possible to customize the parameters for REST access using the CLI. By default, access to the InCenter server using the REST API is enabled. It can be disabled using the following command sequence:

admin@InCenter:/> cc Settings  
admin@InCenter:/Settings> set REST Enabled=No

This CLI set command above could have included any of the following command parameters to change REST API access properties:

  • Enabled - Enable REST API access. Default value: Yes

  • SessionTimeout - Timeout for REST access. Default value: 20

  • AllowTLS1_2 - Allow communication using TLS version 1.2. Default value: True

  • AllowTLS1_3 - Allow communication using TLS version 1.3. Default value: False

  • CertificateData - The certificate public key file used for access.

  • CertificateKey - The certificate private key file used for access.

Note that the port number used for REST API access is always 8443 and this cannot be changed.

The IP address used for connection by the API (either IPv4 or IPv6) is the same as the standard InCenter management IP and this is displayed on the console after InCenter startup.

The SessionTimeout is how long the session token that the client receives from InCenter will be valid. The default time is 20 minutes. The minimum value possible is 1 minute.

Replacing the Default REST API Certificate

After initial startup of the InCenter server, a default REST API certificate called rest_server is automatically used when serving HTTPS requests.

The certificate used for the REST API can be changed by first uploading new certificate files to InCenter's root directory using an SCP or SFTP client on an external computer. For example:

$ scp my_cert.cer admin@192.0.2.10: 
$ scp my_cert.key admin@192.0.2.10:

The CLI can then be used to change the default files:

admin@InCenter:/> cc Settings  
admin@InCenter:/Settings> set REST CertificateData=my_cert.cer 
admin@InCenter:/Settings> set REST CertificateKey=my_cert.key

Missing Activation Response After a Certificate Change

Regardless of how the REST API certificate is changed (it can be achieved using the CLI as above or using the REST API itself), if the user activates this configuration change with the REST API, there will be no response from the InCenter server. It is only when the subsequent commit action is performed that the activate operation can be confirmed.

If the REST activate was successful then the subsequent commit operation will also be successful. If the activate was not successful then the commit will return a message saying that an activate must be performed first.

3.3. WebUI Access

Access to InCenter using the WebUI is enabled by default and can be done using any modern web browser running on a separate computer with network access to the InCenter server.

Controlling WebUI Access

It is possible to customize the parameters for WebUI connections through the CLI. For example, to disable WebUI access, the command would be the following:
admin@InCenter:/> cc Settings  
admin@InCenter:/Settings> set WebUI Enabled=No

The following WebUI properties can be changed in this way:

  • Enabled - If WebUI based management access is allowed. Default value: True

  • AllowTLS1_0 - Allow communication using TLS version 1.0. Default value: False

  • AllowTLS1_1 - Allow communication using TLS version 1.1. Default value: False

  • AllowTLS1_2 - Allow communication using TLS version 1.2. Default value: True

  • CertificateData - The certificate file for the public key.

  • CertificateKey - The certificate file for the private key.

Note that the port number used for WebUI connection is always 443 (the standard HTTPS port) and this cannot be changed.

HTTPS Certificate Files Used with WebUI Access

By default, the InCenter server will use a preinstalled self-signed certificate for the HTTPS communication with a client browser. The certificate files used can be changed by setting the CertificateData and Certificatekey properties of the WebUI object.

Uploading certificate files and using them with the WebUI is discussed further in Section 3.4, Access Certificates.

3.4. Access Certificates

Both the REST API and WebUI access methods rely on HTTPS communication and therefore must send a server certificate to the client when it connects. By default, the InCenter server sends a preinstalled self-signed certificate which the client must accept to continue.

However, for both the REST API and WebUI it is possible to change the certificates used to any certificate uploaded to the InCenter server. The steps for doing this are the following:

  1. Upload the new certificate files in base64 encoding to the InCenter server. The files will consist of a public key .cer file and a private key .key file. File uploading methods are discussed in Section 3.5, File Transfer.

  2. Use the appropriate CLI command to set the CertificateData and CertificateKey properties of the REST or WebUI objects. For example:

    admin@InCenter:/Settings> set WebUI
    			CertificateData=mycert.cer
    			CertificateKey=mycert.key

3.5. File Transfer

File transfers between a management computer and the file system of the host operating system can be performed using the following methods:

  • Via the host operating system using SCP or SFTP

    Access to the file system of the host operating system is possible using the standard method, outside of InCenter control. This involves connecting to the IP of the management interface using an SSH port number of 2222.

    The credentials for this type of access are username administrator and password administrator.

  • Via InCenter using SCP or SFTP

    If port 22 is used instead of port 2222, SCP and SFTP access is mediated by InCenter. This method is enabled by default after InCenter installation. However, only the storage folder can be accessed using this type of access.

    The credentials for this type of access are, by default, username admin and password admin, but could be the credentials for any administrator User object in the InCenter database.

  • Using the REST API

    File transfer can also be done to InCenter from an external computer running a custom program that makes use of the InCenter REST API. This is described further in the separate REST API guide.

Downloading Files via InCenter

Files can be downloaded from InCenter using either an SCP or SFTP client. By using the default SSH port of 22, transfers will be mediated by InCenter so the only directory accessible will be /mgmtsystem/storage. This section will only show examples using SCP.

Assuming that the InCenter IP address is 192.168.23.206, the following will download all log files:

$ scp -r admin@192.168.23.206:./logs .

To download just the system.log file:

$ scp admin@192.168.23.206:./logs/system.log .

All the error files the can be downloaded with the following SCP command:

$ scp -r admin@192.168.23.206:./errors .

However, downloading the error files like this is usually not necessary because these will be included in the file generated by the techsupport command (described further in Section 17.11, The techsupport Command).

Cleaning Up Files

To manage files that have been uploaded to the InCenter server, the best method is to use SFTP. This allows files to be listed, filenames to be changed and unwanted files to be deleted.

3.6. SSH Server Management

When using SSH to communicate with InCenter, an SSH server internal to InCenter is being used. The server host key used is a 2048 bit RSA key and is automatically generated during initial system startup. This SSH server can be managed using the sshserver command.

Viewing SSH Server Status and the Current Key

Using the -showkey option with the sshserver command will display both the current status of the SSH server and the current key:

admin@InCenter:/> sshserver -showkey
SSH server status: running
Public Key:
AAAAB3NzaC1yc2EAAAADAQABAAABAQC5MhkCyKhqOABUcwFqVJIICZPlpJtRuP4
onxOMw/PmkrImJBbJhtZqmm4u81IybUs9lowa3dsE0v4BCBWAiAPV9CCgr7FwGM
zsUUFGy1+b0EiU/n2k9GCCQBCUVkCPNKz/IFdUBiMpxZbGCVvbeSd9pyFOc/lHc
9IBEn624eObmi1RCp16U+17P7C7xI6z1hVYMIjrEcWf7e7XGGhS/4NqgZPwFAhS
pmXdEjMislMpHXrGCet7PiqnBpOZ6UFpqxsaAxmcjDxxUvK67jVH
Fingerprint: SHA256:fyXp7IIhqLqZukeJ9oXXAlaJD1oRFVLtzA16Qhd+RPI

Changing the SSH Server Key

Using the -keygen option with the sshserver command will generate a new key for the server. The system will prompt the administrator to check that the action should be completed:

admin@InCenter:/> sshserver -keygen
This action will generate a new host key to the ssh server.
For this to take effect the ssh server needs to be restarted
and your current ssh session will end
Do you want to continue? [yes/no]: y

Chapter 4: Using the CLI

This section is an introduction to using the InCenter CLI. By default, CLI access to InCenter via SSH is enabled. The IP address and credentials to use for SSH access are discussed in Section 3.1, SSH Access to the CLI.

SSH access requires authentication with the credentials of a User object. The predefined admin user can be used (default password admin) for access. Alternatively, new users can be defined and doing this is described in Chapter 9, Managing Users.

Changing the SSH CLI Prompt and Welcome Message

The default CLI prompt takes the form:

admin@InCenter:/>

The default domain in the CLI prompt ("InCenter") can be changed, but only for the SSH CLI. For example, to change the SSH prompt domain to the string "myhostname.com", the command sequence would be:

admin@InCenter:/> cc Settings 
admin@InCenter:/Settings> set SSH PromptName=myhostname.com
Updated SSH
admin@myhostname.com:/Settings> cc
admin@myhostname.com:/>

In addition to changing the prompt, it is also possible to set the SSH CLI welcome message. For example:

admin@InCenter:/> cc Settings 
admin@InCenter:/Settings> set SSH WelcomeMessage=HelloThere

These changes need to be followed by an activate/commit command sequence to make them permanent.

The Basic CLI Command Set

The following make up the basic command set of the InCenter CLI for manipulating objects either within node configurations or InCenter itself:

  • add - Used for adding an object to InCenter.

  • set - Used for changing the properties of an existing object.

  • show - Used to display the current values of InCenter objects.

  • delete - Used for deleting InCenter objects.

  • activate - Used for deploying InCenter changes.

  • commit - Used to make deployed changes permanent following an activate command.

  • help - Used to provide help information about a command.

Other commands exist for specific administration purposes. For example, the cfglog command is used to display errors/warnings after activating a change. The user command allows management of the InCenter user database.

4.1. Tab Completion

The InCenter CLI provides a useful feature known as tab completion. When entering CLI commands, the Tab key can be pressed at any time. This can have the following functions:

  • The property name is automatically completed where possible.

  • The property value is completed where possible.

  • Where no property name or property value has been entered, a list of possible options is presented.

Consider following example, where the command entered so far is add Standalone:

admin@InCenter:/> add Standalone

If the Tab key is pressed, InCenter will autocomplete the object type:

admin@InCenter:/> add StandaloneNode

If the Tab key is pressed again, InCenter will autocomplete the first mandatory property followed by an equals sign:

admin@InCenter:/> add StandaloneNode Name=

If a Tab key is pressed again, InCenter cannot autocomplete the value but will provide help about the type of value that should be entered:

admin@InCenter:/> add StandaloneNode Name=
			Name: Name
			Type: string

Special Characters Used with Tab Completion

When entering the value of an object property, the tab character can be preceded by any of the following special characters:

  • Entering the period "." (period) character before entering tab will insert the current value of the object property.

  • Entering the " * " (asterisk/star) character before entering tab will cause the default property value to be automatically filled in. However, note that only mandatory properties have a default value. Optional properties do not have a default value.

4.2. Specifying String Values

When specifying string values in the CLI, the following rules apply:

  • Strings containing spaces must be delimited by either single or double quotes. For example:

    		"This is my string with spaces"

    Or alternatively:

    		'This is my string with spaces'
  • The backslash "\" character can be used to escape single or double quotes within a quoted string. For example:

    	"This is my quoted \'string\' with backslashes"

The above rules apply to all string values specified within the CLI, including passwords.

4.3. Displaying Unsaved Changes

The CLI show command can indicate that there are node related modifications which have not yet been activated. For example, the following command will show all the standalone nodes in InCenter along with information about the activation status:

admin@InCenter:/> show StandaloneNode
   Name      Username  IP              Port
-  --------  --------  --------------  ----
+  my-node1  admin     192.168.23.120  22
-  my-node2  admin     192.168.23.189  22
*  my-node3  admin     192.168.23.118  22

The example output above shows three nodes. The characters in the first column indicate if a node has been modified in some way but the modification has not yet been activated. The meanings of the characters are as follows:

* (asterisk) - The object has changes that have not been activated.

+ (plus) - The object has been added but the addition has not been activated.

- (minus) - The object has been deleted but the deletion has not been activated.

The following statements are therefore true for the output in the example above:

  • Node my-node1 has been added.
  • Node my-node2 has been deleted.
  • Node my-node3 has a modified configuration.

Once these changes have been activated and then committed, the initial character in the output will disappear from a subsequent show command.

4.4. Activating Changes

Changes made through the CLI are stored in the InCenter database but not deployed to nodes until an activate command is issued. This must then be followed by a commit command to make these changes permanent:

admin@InCenter:/> activate
Operation in progress... / 
Activate successful
Nodes activated: my-node1
Run the "commit" command to keep the new configuration
admin@InCenter:/> commit
Committed
admin@InCenter:/>

This command sequence is exactly the same as the sequence used when changing a node configuration directly. It makes sure that the change will not break communication with the server by requiring that a second command is entered within a certain period of time to commit the changes. The period of time is known as the RevertTimeout and is set by default to 30 seconds. The following is an example of changing this timeout to a value of 60 seconds:

admin@InCenter:/> cc Settings 
admin@InCenter:/Settings> set Activation RevertTimeout=60

User Database Changes are an Exception

It should be noted that changes made to the InCenter user database are an exception. Such changes are applied directly and do not require an activation sequence to be entered. This also means that such changes are not part of the InCenter configuration history and cannot be undone using the history.

Commenting a Change

When changes are deployed, it can be useful to add a brief comment which is saved in the history and is available for display later. This is done by using the -comment option to the commit command. The comment itself need to be surrounded by single or double quotes if it contains spaces. For example:

admin@InCenter:/> commit -comment="This is my comment"

Activated Changes are All-or-Nothing By Default

One or many changes might be pending before the activate command is entered to deploy them. By default, InCenter will try to activate all changes to all nodes in a single operation. The activations are done on an all-or-nothing basis. If any single activation fails for any reason on any node, the entire deployment is aborted.

With this default behavior, the commit command can be entered to make the changes permanent only after activation has been successful on all nodes.

However, the option also exists to perform a selective activate when pending changes are to be deployed only to certain nodes. This is discussed in Section 4.5, Selective Activate.

The Changes Activated Depend on the CLI Context

When the CLI context is the top level default context, an activate operation will apply all pending changes. However, if the CLI context is changed to be a particular standalone node will be applied when an activate command is entered. The same applies if the CLI context is an HA pair.

Ignoring Unreachable Nodes

The default all-or-nothing behavior described above, ensures that the configuration copy kept by InCenter is always synchronized with the copy on the node itself. However, it is possible to change this deployment behavior when there are unreachable nodes by setting the global activation option AcceptUnreachable to a value of Yes:

admin@InCenter:/> set Activation AcceptUnreachable=Yes

When this is done, unreachable nodes will not cause activation to be aborted for all nodes, only a validation failure will do this.

When allowing activation with unreachable nodes, it should be noted that InCenter will automatically push its current configuration copy onto the node when it becomes reachable again.

 

Configuration Changes Cannot Be Made Between activate and commit

After entering the activate command, InCenter will not allow a configuration change for any node until either of the following occurs:

  • A commit command is entered.

  • Or the commit command is not entered in time and a bi-directional time-out condition occurs. In this case, all the affected configurations reverted to their original state before the activate command was issued.

    In addition, none of the pending changes are lost. The sequence of the activate command followed by the commit command could therefore be entered again to try and deploy the same changes.

Deployment Errors/Warnings and the cfglog Command

As stated above, changes are all or nothing during deployment. Any errors or warnings will usually occur following the activate command and such messages will be displayed in the CLI console.

To investigate the cause of a deployment error further, configuration log messages can be examined using the cfglog command. The following is an example of cfglog output after trying to commit an erroneous change:

admin@InCenter:/> cfglog
Last Activation Output:
Session validation failed
Errors:
  my-node1:
    - IPAddress my_addr_obj1.Address: Value is not a valid reference
  my-node2:
    - IPAddress my_addr_obj2.Address: Value is not a valid reference

The output of the above command will provide only the first few errors/warnings. The complete list can be seen by adding the -verbose option.

The above output can be restricted to a single node after the cfglog command or changing the CLI context to be the node and using the cfglog command with no options. The following is an example of the first method:

admin@InCenter:/> cfglog my-node1
Last Activation Output:
Session validation failed
Errors:
  my-node1:
    - IPAddress my_addr_obj1.Address: Value is not a valid reference

When troubleshooting, it is also possible to drill down further by looking at the configuration messages generated by a particular node itself. This is done by using the -fromNode option with the cfglog command. In the following example, the activation log messages taken directly from the node named my-node1 are listed:

admin@InCenter:/> cfglog my-node1 -fromNode
Configuration log for my-node1:
Notices:
  Config notice  vsinit:3155  2018-11-24 15:46:51
  - Beginning system reconfigure. Activating new configuration.
  Config notice  vsinit:3663  2018-11-24 15:46:51
  - Reconfigure completed successfully.
  Config notice  vsinit:1337  2018-11-24 15:46:52
  - Configuration changes committed.

Fixing Configuration Mismatches with the export Option

If the configuration on a node becomes out of synchronization with InCenter's copy , it is possible to manually force the node by using the export option of the node command. For example:

admin@InCenter:/> node my-node1 export

This option does not require an activate and commit sequence to complete.

Note that an export operation can fail for one of the following reasons:

  • If the software version on the firewall is not supported by InCenter. The solution is to upgrade the firewall software version to one that is supported.

  • The firewall has a higher software version than the configuration copy in InCenter. The solution for this is to make a small change in the firewall's configuration and then deploy that change. This will cause the version of the firewall's software in the InCenter database to be updated to the correct version.

  • The firewall has a lower software version than the configuration copy in firewall version was downgraded. The solution for this is to upgrade the firewall version so it matches the version in InCenter.

The relationship between the version of InCenter and the version of software running on a managed node is explained further in Chapter 11, Version Management.

A way of just pushing the current InCenter copy of the configuration to the firewall is to make configuration changes in InCenter and to then activate the changes. As long the new configuration is valid then it will overwrite the configuration on the node. However, this will not ensure that both sides have the same software version.

To determine if there is a configuration mismatch, the status option of the node command can be used. For example:

admin@InCenter:/> node my-node1 status

Switching Off Deployment for a Node

It is possible to switch off the deployment of configuration changes for a particular node using the IncludeInDeploy property. This means that even though changes are activated, only the InCenter copy of the configuration is updated. This is explained further in Section 10.4, Node/Firewall Synchronization.

[Note] Note: Activating Changes for Large Node Numbers

Caution is recommended when activating changes for large numbers of nodes at the same time. If the number of nodes with pending changes exceeds 2,0000, activation should only be performed with remote node validation enabled. This feature is explained next.

Node Validation

By default, when configuration changes are activated, the validation processing required is performed locally by InCenter before the configuration is sent to nodes. However, deployment times when activating changes on large numbers of nodes can be improved by having configuration validation performed remotely on the node itself. This feature can be enabled globally for all deployments with the following command:
admin@InCenter:/> set Activation Validation=Node
To turn off all node validation and return to the original default, the command would be the following:
admin@InCenter:/> set Activation Validation=Local
As described earlier in this section, it is possible to change the default all-or-nothing activation behavior so that unreachable nodes are ignored. Both global behavior options can be combined:
admin@InCenter:/> set Activation Validation=Node
			AcceptUnreachable=Yes

4.5. Selective Activate

As discussed in Section 4.4, Activating Changes, the default action when configuration changes are activated is to deploy all unactivated changes. However, it is possible to activate changes selectively for a given standalone node or HA pair.

To do this, the activate command is followed by the name of the node or HA pair. For example, only the pending changes for the standalone node called my-node1 would be activated with the following command:

admin@InCenter:/> activate my-node1

In the same way, only the changes made on the HA pair called my-hapair might be activated:

admin@InCenter:/> activate my-hapair

An alternative method of activating only the pending changes for a particular standalone node or pair and enter an activate command with no parameters. For example, the following commands could be used to execute all the pending changes for the node called my-node1:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1> activate

Note that changing the CLI context to a node within an HA pair and issuing an activate command is not permitted and will give an error message.

4.6. Viewing Node Status

The status command can be used at any time to view the current status of either a StandaloneNode or an HAPair. It also provides a way to verify that InCenter is able to communicate with a node or HA pair.

Usage with NetShield Nodes

The following is an example for a standalone NetShield node:

admin@InCenter:/> status my-node1
This node is a standalone system
Uptime: 121 days 06:02:48
HostKey: SHA256:YmAMcRuqn2YcxEmU0Kr+VuFAldELmgY8+lKCfis
Config revision: In sync - 20 (2018-07-03 10:53:20)

The command can also be used on an HAPair or HAMemberNode:

admin@InCenter:/> status my-hapair-member1
This node is an HA MASTER
Active: yes
HA cluster peer is alive
Uptime: 25 days 01:08:02
HostKey: SHA256:MHOMtr77nLXymxZJOAMmi+7bR6QZ8UjREJNung
Config revision: In sync - 5 (2018-07-03 10:01:36)

The meaning of the "In Sync" message in the last line of output is explained next.

Usage with NetWall Nodes

The following is an example for a standalone NetWall node:

admin@InCenter:/> status my-node1
This is a NetWall node
Node is connected from ::ffff:192.168.102.99
This node is a standalone system
Uptime: 0 days 18:34:57

The command can also be used on an HAPair or HAMemberNode:

admin@InCenter:/> status my-hapair-member1
This is a NetWall node
Node is connected from ::ffff:192.168.102.99
This node is an HA MASTER
Active: yes
HA cluster peer is dead
Uptime: 0 days 18:36:30

4.7. CLI Relay

From the InCenter CLI, it is possible to start a normal CLI session to a NetShield node so InCenter relays commands to the node. However, this will not affect the InCenter configuration database or InCenter history in any way, and is the equivalent of a direct SSH console session to the firewall. Note that this is a feature that is only available for NetShield nodes and it is not available with NetWall nodes.

In addition, while under the centralized management control of InCenter, the only commands that can be relayed are those which do not modify the node configuration. This is explained further in Chapter 10, Centralized Management Control

For example, to open a direct CLI session to the node my-node1, the InCenter CLI command would be:

admin@InCenter:/> node my-node1 cli
Starting relayed node CLI connection towards node my-node1
at 192.168.98.14 with user admin
Welcome.
Logged in as administrator - admin
System:/>

The System:/> prompt is a standard firewall CLI prompt. Now, all firewall commands can be entered as though there is a direct connection to the node.

To terminate relaying and return to the default InCenter CLI context, the exit command is used:

System:/> exit
Ended relayed node CLI connection towards node my-node1
at 192.168.98.14 with user admin
admin@InCenter:/>

With this relay feature, InCenter is simply relaying CLI commands to the firewall outside of InCenter monitoring. It is equivalent to a direct SSH connection to the firewall. It is intended to be used for troubleshooting purposes only.

Note that the firewall will not allow the changing of its configuration using the relay feature while it is under centralized management control. Changes can only be made directly once the node is removed from centralized control. This is explained further in Chapter 10, Centralized Management Control.

Chapter 5: Using the WebUI

The InCenter WebUI is a graphical interface that allows interaction with the InCenter server through a standard web browser. It can be used for performing a range of administration and monitoring tasks.

The WebUI cannot fully replace all the InCenter CLI functions. However, it can perform a subset of CLI capabilities and provides node monitoring capabilities that are not possible with the CLI.

Controlling WebUI Access

The InCenter server has WebUI access enabled by default. Controlling WebUI access and changing the WebUI access settings is done using the InCenter CLI and is described in Section 3.3, WebUI Access

The WebUI Language

The interface language used in the InCenter WebUI comes from the default language used in the web browser, but will only change from English if the browser's language is available in InCenter. Currently, the languages available in InCenter are:

  • English (abbreviation: "en").

  • Japanese (abbreviation: "ja").

The WebUI language cannot be changed once the interface has opened. To switch language, the WebUI should be closed, the browser's default language changed and then the WebUI reopened.

Alternatively, the language can be selected by adding the parameter:

 lng=<language-abbreviation>

to the URL that opens the initial login page (it will not change the language mid-session, after login). The possible <language-abbreviation> values are given in the preceding language list.

For example, to open the login dialog in English:

 https://<InCenter-URI>/overview?lng=en

To open the login dialog in Japanese:

 https://<InCenter-URI>/overview?lng=ja

Note that once the WebUI is opened with a particular language, it will open for the next session with that same language if no language parameter is specified in the URL. The language can only be changed by explicitly specifying the language parameter at the end of the URL.

5.1. Starting the WebUI

The WebUI is available through any modern web browser. The only prerequisite is that the browser can access the InCenter server using HTTPS.

Connecting to the WebUI

After opening a browser, the InCenter WebUI is started by entering the IP address of the InCenter server in the browser's navigation field.

The IP address used for browser access is the same as the IP address used for InCenter SSH access and is displayed on the InCenter server console when it is started. Access is always using HTTPS and, like SSH access, the login credentials must match a previously defined user. The only user that is predefined in InCenter is the principle administration user that has the username admin.

An Exception Must Be Made for the Server Certificate

On initial connection, InCenter will send a self-signed certificate which the browser will not recognize. A security exception for this certificate must therefore be confirmed in the browser before the WebUI interface will be displayed. This only needs to be done once, unless the exception is removed from the browser later.

The self-signed certificate is predefined in the InCenter server. However, it can be replaced by the administrator with any other suitable certificate. It is recommended to use a certificate that matches the server's FQDN.

Logging In

After successful connection to the InCenter server, a login dialog will be displayed in the browser, as shown below.

WebUI Login Dialog

Figure 5.1. WebUI Login Dialog

The predefined administrator user on a freshly installed system has the following default credentials:

  • Username: admin

  • Password: admin

Note that the second time the administrator logs in with these credentials, they will be prompted to change the password to one that conforms to the system's password policy settings. The password policy is explained further in Section 9.3, User Password Policy.

If Multi Factor Authentication (MFA) is enabled in InCenter, the login dialog shown above will have an extra option to login using MFA. This is discussed further in Section 9.4, Setting Up MFA.

The Overview Page

After successfully logging into the WebUI, the Overview dashboard is displayed which provides a graphical summary of activity for all the nodes currently communicating with InCenter. An example is shown below.

Initial WebUI Overview Dashboard

Figure 5.2. Initial WebUI Overview Dashboard

If no nodes have been added previously, the first step a new administrator will probably want to take is to add one or more InCenter. Doing this is described in Section 6.3.1, Adding NetWall Nodes with the WebUI

5.2. Activating InCenter Changes

None of the changes that are made using the WebUI, either to the InCenter or to nodes, are made permanent until they are committed. If any pending uncommitted changes exist, this is indicated by an orange disk appearing over the changes icon in the top right of the display with a number indicating the number of pending changes.

The example below shows the changes icon with single change is waiting for activation by the administrator.

The Activation Pending Icon

Figure 5.3. The Activation Pending Icon

By pressing the changes icon, the Changes display will open with a list of pending changes. This can also be opened by selected the Changes option from the System menu in the navigation pane.

The Changes Menu Option

Figure 5.4. The Changes Menu Option

An example of the Changes display is shown below, with a single pending change which is the addition of a new standalone node called my-node1.

Pending InCenter Changes

Figure 5.5. Pending InCenter Changes

The administrator now has one of the following choices:

  • Press the Commit Changes button to commit all changes. It should be noted that a commit operation is all or nothing. A selective commit is not possible.

  • Press the Reject all button to reject all the changes. This will discard any pending changes made since the last commit operation.

  • Select a single change to see its details and optionally reject just that change.

Also note that filters can be applied to the pending list to show specific types of changes.

If the administrator chooses to commit changes, a final dialog appears for confirmation and provides the opportunity to enter a comment into the history for the commit.

Confirming InCenter Changes

Figure 5.6. Confirming InCenter Changes

Finally, the WebUI will display a message indicating completion of activation.

The Activation Complete Message

Figure 5.7. The Activation Complete Message

Chapter 6: Managing NetWall Nodes

6.1. Overview

InCenter can be used to centrally manage and/or monitor one or more NetWall nodes.

The following setup options are available:

  • Centralized Management with Optional Monitoring Setup

    For centralized management to be enabled, communication between the InCenter server and NetWall node needs to be set up and then the node can be added to InCenter so its configuration can be imported. Doing this is described in Section 6.2, Setting Up Centralized Management.

    InCenter can also be used for node monitoring by receiving, storing and analyzing log event messages sent by NetWall nodes. Monitoring can optionally be added to centralized management by configuring nodes to send log messages back to InCenter. This is discussed further in Section 6.3, Setting Up Monitoring.

  • Monitoring Only Setup

    Monitoring can be set up without also enabling centralized management. Doing this is described in Section 6.3, Setting Up Monitoring.

    Note that it is possible to have some NetWall nodes under centralized management (with optional monitoring) while others only have monitoring enabled.

Group Membership is Supported

InCenter Group membership can be specified for NetWall nodes when they are added. Group membership can also be specified later, after adding a node. A single group can contain multiple nodes and multiple groups can be created.

However, a group cannot contain both NetShield and NetWall nodes mixed together. When a Group object is created, its NodeType property must be set according to the type of nodes it will contain. The default NodeType value is NetWall.

Using groups is described further in Chapter 15, Node Groups.

6.2. Setting Up Centralized Management

When InCenter, they can become subject to centralized management by InCenter. The functions provided by centralized management are discussed in Chapter 10, Centralized Management Control.

[Note] Note: Skip this section if setting up monitoring only

It is possible to enable monitoring only for a node without also enabling centralized management. To enable monitoring only, follow the steps described in Section 6.3, Setting Up Monitoring.

To bring a node under centralized management control, setup steps are required on both the InCenter so that the two can communicate using the relevant keys. After setting up this communication, the node is added to InCenter and its configuration imported.

The following list summarizes the process of centralized management setup:

  1. Set up the NetWall node to communicate with InCenter. Doing this is described in Section 6.2.1, NetWall Node Centralized Management Setup.

  2. Set up InCenter to communicate with the node. Doing this is described in Section 6.2.2, InCenter Centralized Management Setup.

  3. Add the NetWall node in InCenter so its configuration is imported and centralized management becomes enabled. Doing this is described in Section 6.2.3, Adding NetWall Nodes with the CLI.

The steps listed above are covered sequentially in more detail in the sections that follow.

[Note] Note: InCenter and InControl management are incompatible

Do not try to set up InCenter centralized management for a NetWall node that is already being centrally managed by the Clavister InControl product. Centralized management by InControl should be disabled first.

However, monitoring by InCenter is still possible even though a node is under InControl management.

6.2.1. NetWall Node Centralized Management Setup

This section describes the detailed steps for setting up NetWall nodes for centralized management. These operations are performed as a local administrator of the firewall and not through InCenter.

1. Set the name on the node

If it has not been done already, set the name of the node's CLI with the following command:

Device:/> set Device Name=<device-name>

2. Copy the InCenter SSH Key

The first step is to copy the InCenter public SSH key into a file. This is done by first listing the key in the InCenter CLI with the following command:

admin@InCenter:/> nodeserver -showkey

Create a new text file containing this key using copy and paste from the console output.

Note that the sshserver command is not used since that is for a different SSH server from the one used by InCenter to communicate with NetWall nodes and the two servers use different public keys. The nodeserver key is always automatically the same on a second redundant InCenter node and can be regenerated with the command nodeserver -keygen.

3. Add an SSH Public Key object

In the NetWall node WebUI, go to Object > Key Ring and select Add > SSH Public Key. Enter a suitable name and select Upload SSH public key file so the file created in the previous step can be uploaded.

This step could alternatively be done by using SCP to upload the file to the sshpublickeys folder of the node.

Another way to do it is using the NetWall node's CLI. An example CLI command would be the following:

Device:/> add SSHPublicKey my-pub-key PublicKey="ssh-rsa AADAB3NzaC1y23"

The key value in the above command has been truncated and a real key would be much longer.

4. Add a Remote Management object

In the NetWall node's WebUI, go to System > Remote Management and select Add > InCenter Management. Set the Primary IP property to the IP address of the InCenter server (this IP could be first defined as an address book object) and set the SSH server public key property to the SSH Public Key object created in the previous step.

Note that the Secondary IP property is only used when a second InCenter node is available to provide redundancy.

If required, an alternate routing table can also be selected for management traffic that is outgoing to the InCenter server. By default, the main routing table will be used.

Using the NetWall node's CLI, the command to set up a remote management object would be:

Device:/> add RemoteManagement RemoteMgmtInCenter
			InCenterPrimaryIP=<InCenter-IP>
			InCenterKey=<key-object>

The next section describes how to set up InCenter.

6.2.2. InCenter Centralized Management Setup

The section described the detailed steps for setting up InCenter for centralized management of a particular NetWall node.

1. Copy the SSH key from the NetWall Node

In the WebUI of the NetWall node, go to Objects > Key Ring and open the RSA 2048 bit object in the list. Select the OpenSSH format option and then copy and paste the key to a local text file with a filetype of .pub.

In the cOS Core CLI, the key can be displayed with the following command:

Device:/> show SSHHostKey RSA

2. Create an SSHPublicKey object in InCenter

Upload the key file from the previous step to InCenter using SCP and then create a new SSHPublicKey with the following CLI command:
admin@InCenter:/> add SSHPublicKey Name=my-sshkey
			PublicKey=my-keyfile.pub

The next section describes how to add the node to InCenter. When added, the node's PublicKey property will be set to the SSHPublicKey object value created in the last step above.

6.2.3. Adding NetWall Nodes with the CLI

This section describes how to add a new NetWall node to InCenter so that centralized control is enabled for the node. It is assumed that the setup has already be done so InCenter and the node can communicate.

This section does not apply if InCenter is being used for monitoring only (without centralized management). For monitoring only, nodes must be first configured to send log messages to the InCenter server and then added using the WebUI, which is described in Section 6.3, Setting Up Monitoring.

To add any NetWall node so it comes under centralized management control for the first time, the steps are as follows:

  • If it has not been done already, perform the centralized management setup described previously in Section 6.2, Setting Up Centralized Management.

  • Access the InCenter CLI using an SSH client.

  • Add the Clavister Next Generation Firewall as a standalone node.

  • Activate and commit the change.

  • Import the node's configuration to the InCenter database.

  • Make any required changes to the node configuration.

  • Activate all changes and commit them.

Adding any node and importing its configuration into InCenter usually only has to be done once. However, there are exceptions. For example, after upgrading the firewall software version, a node must be re-imported.

[Note] Note: Add nodes in batches of no more than 500

When adding a large number of nodes, no more than 500 nodes should be added in any one batch before the changes are activated. This is true both for the add operation and the import operation.

Adding a Node

When a node is added, the PublicKey property of the StandaloneNode must be specified for authentication. This property is assigned a SSHPublicKey object that will have been already created by the administrator during centralized management setup. An example CLI command to do this would be the following:

admin@InCenter:/> add StandaloneNode
			Name=my-node1
			NodeType=NetWall
			PublicKey=my-node-key

Saving Changes

The addition of a node, and any subsequent configuration change made with the InCenter, will not be saved or deployed to a node until an activate command is issued. This must be followed by a commit command to make changes permanent:

admin@InCenter:/> activate
Activate successful
admin@InCenter:/> commit
Committed
admin@InCenter:/>

Note that the activate command will apply all pending changes to affected nodes at the same time. If an error occurs, all changes are rolled back. If all changes are successful, the commit command will make the changes permanent.

Verifying the Connection to the Node

Once the node addition is activated, the node will be able to connect to InCenter. There will be a brief delay before the node establishes a connection with InCenter. This connection can be verified in InCenter by using the following command:

admin@InCenter:/> node <node-name> status

Importing the Node Configuration

After the communication with InCenter is established, the next step is to import the firewall configuration into InCenter. This is done with the following command sequence:

admin@InCenter:/> node my-node1 import
Importing configuration from: my-node1
Import done
admin@InCenter:/> activate
Activate successful
admin@InCenter:/> commit
Committed
admin@InCenter:/>

This sequence results in the node configuration being read and saved in the InCenter configuration database on the server. Subsequent configuration changes made through InCenter will now be reflected in both the node configuration and InCenter's database copy.

In addition, the node is brought under centralized management control by InCenter and it will no longer be possible to change the configuration of the node by direct connection to it. This is explained further in Chapter 10, Centralized Management Control.

[Note] Note: Import requires an Activate/Commit

An import command must be followed by an activate then a commit command for the import into the InCenter to be completed.

If an import operation is not performed, it will not be possible to make node configuration changes under the control of InCenter.

A node Can Also Be Exported

InCenter provides an export operation for a standalone node which is the opposite to import. This will push the current configuration held in the InCenter database out to the node so that it synchronizes with the database. For example:
admin@InCenter:/> node my-node1 export
The export command is discussed further in Section 4.4, Activating Changes

Editing Node Properties with the CLI

After adding to InCenter, a node's properties can be changed using the set command. For example, to change the name of the node called my-node1 to my-new-name, the following command could be used:

admin@InCenter:/> set StandaloneNode my-node1 Name=my-new-name

6.3. Setting Up Monitoring

This section describes how to set up a NetWall node for monitoring by InCenter. Monitoring means that the node will send log event messages to InCenter so that its activity can be analyzed and presented graphically in the InCenter WebUI. It is possible to enable this feature without the node also being under centralized management by InCenter.

For monitoring to function, the following two steps are required:

  1. If centralized management AND monitoring is required, centralized management must first be set up using the steps described in Section 6.2, Setting Up Centralized Management.

    If ONLY monitoring is required (without centralized management) then the NetWall nodes being monitored must be added to InCenter using the WebUI and this is described in Section 6.3.1, Adding NetWall Nodes with the WebUI. None of the centralized management setup steps are required.

  2. The NetWall node must be set up to send log messages back to InCenter. Doing this is described in Section 6.3.2, Configuring a Log Receiver in NetWall Nodes.

6.3.1. Adding NetWall Nodes with the WebUI

[Note] Note: Skip this section if centralized management is used

This section should be skipped if the node is subject to centralized management by InCenter. Adding the node with the CLI is part of centralized management setup and only the sending of logs by the node needs to be configured.

A NetWall node can be added to InCenter using the WebUI. However, addition in this way should only be done when setting up the monitoring function without centralized management.

Any NetWall node with cOS Core software version 13.00.00 or later installed can be monitored using InCenter.

To add a NetWall node to InCenter for monitoring, select the Nodes option from the Manage section in the navigation pane, as shown below.

s

Figure 6.1. s

Press the Add button and select Node.

Option

Figure 6.2.  Option

This starts the new node wizard which will go through the following steps:

  1. Properties

    Select the NetWall option and specify a logical name for the node with an optional comment.

    Node Wizard - Properties

    Figure 6.3.  Node Wizard - Properties

    [Important] Important: The InCenter name must match the firewall name

    The node name specified in InCenter must match the local node name on the firewall itself. In addition, the firewall name should not be duplicated within InCenter. Therefore, the name may need to be changed locally to a new value in cOS Core before performing the addition in InCenter.

  2. Done

    In the last step, a summary is displayed to confirm the details of the addition.

    Wizard - Done

    Figure 6.4.  Wizard - Done

  3. Pressing the Done button will now close the wizard and the added node will appear in the node list.

This change to InCenter now needs to be activated and doing this is described in Section 5.2, Activating InCenter Changes.

6.3.2. Configuring a Log Receiver in NetWall Nodes

For a NetWall node to send log messages to InCenter, a log receiver object needs to be locally configured on the node. Doing this using the node's local WebUI or CLI is described below.

This setup procedure is the same both for nodes that are being centrally managed by InCenter and for those where only monitoring is being done by InCenter.

Note that if a node is under centralized management, an alternative way to set up a Log Receiver in the NetWall node is through InCenter, instead of doing it locally. If InCenter is only monitoring the node then this will not be possible.

Configuring a Log Receiver with the WebUI

To configure a Syslog log receiver in the node's WebUI, open the WebUI in a browser and go to: System > Device > Log Receivers. Then press the Add button and select the option Syslog Receiver.

Add Syslog Receiver

Figure 6.5. Add Syslog Receiver

The dialog for this new object can then be filled in, as shown in the example below.

Add Syslog Receiver Dialog

Figure 6.6. Add Syslog Receiver Dialog

Note that the IP address specified is the same IP address that is used for SSH management access for InCenter.

The option to make log messages InCenter compatible must also be enabled and this is found in the Advanced tab. Note that this setting can only be found in NetWall nodes running the software version 12.00.16 or later.

Enabling Log Message Compatibility

Figure 6.7.  Enabling Log Message Compatibility

Configuring a Log Receiver with the CLI

The node CLI could be used instead to configure the log receiver. The following is an example of a command to do this where the destination IP for log messages is 203.0.113.10:

Device:/> add LogReceiver LogReceiverSyslog my-syslog-receiver
			IPAddress=203.0.113.10
			InCenterCompatibility=Yes

6.4. Node Dashboards

For each node that sends logs to InCenter, a number of dashboards are available in the WebUI which provide both details and a summary of node activity. These dashboards provide information in both a graphical and numerical way.

nodes should first be correctly configured to send log messages to InCenter. The node configuration steps are described in Section 6.3, Setting Up Monitoring. In addition, a node should also have been added to InCenter and doing this is described in Section 6.3.1, Adding NetWall Nodes with the WebUI. It is not necessary for a node to be under full centralized control by InCenter for monitoring to function.

The No-Logs Message

When trying to display dashboards for the first time, the message shown below might appear to indicate that no log messages are available for dashboard display.

The No-Logs Message

Figure 6.8. The No-Logs Message

The above message appears for one of the following reasons:

  • InCenter aggregates log messages every five minutes and so the first log messages received may not have been aggregated yet. Waiting a few minutes before refreshing the dashboard should resolve this issue.

  • InCenter has not received any logs from any nodes. The actions given in the no-logs message should be followed to troubleshoot this problem.

Dashboard Types

A dashboard can display information for a single node or a summary for multiple nodes. All dashboards are accessed by selecting the Analyze option in the navigation pane and can then be selected by type, as shown below.

Dashboard Type Menu

Figure 6.9. Dashboard Type Menu

Each dashboard provides numeric and graphical summaries of the historical activity for the nodes that have been added to InCenter. Every dashboard has common controls for filtering the information displayed. This filtering can be by:

  • Node

    A particular node can be selected from a drop-down menu.

    Menu

    Figure 6.10.  Menu

    Multiple nodes can be selected in this way to view the aggregated events from those nodes. The Clear All Filters button can be used to clear all the dashboard filters and return to the default settings.

  • Time Period

    The most recent period of time can be selected from a drop-down list. This period can range from the last 5 minutes to the last 30 days. Note, however, that the default time of keeping log messages is 3 days and only aggregated data is kept for up to 30 days.

    Dashboard Period Selection

    Figure 6.11. Dashboard Period Selection

  • Refresh Interval

    The refresh interval determines how often the dashboard will be automatically refreshed by InCenter. This can be switched off or set at different refresh intervals up to a maximum of one hour.

    Dashboard Refresh Interval

    Figure 6.12. Dashboard Refresh Interval

  • View Mode

    The administrator has a choice of viewing dashboards in a view mode of Fast or Rich. The Fast option makes use of aggregated log message data. The slower Rich option makes use of the raw log message data but also allows drill-down into that data.

    Dashboard View Mode

    Figure 6.13. Dashboard View Mode

    The following should be noted about using these options:

    1. Since the Fast mode relies on aggregated data and because data aggregation by InCenter takes time to complete, using this mode with a short time window of five minutes or less may result in the dashboard display being empty and not showing any node activity.

    2. Log message data is kept for a maximum of one year.

Adding a Filter

It is possible to narrow the log messages being analyzed by adding a filter to a dashboard. This is done by selecting the Add a Filter option. This opens a dialog which allows a field in log messages to be specified along with a string value to match against plus a logical operator that links the two. Below is an example of a filter that specifies only log messages where the source IP address matches 203.0.113.5 are to be included.

Add Dashboard Filter

Figure 6.14. Add Dashboard Filter

Alternatively, it is possible to build complex queries using the query DSL search language by clicking the Edit Query DSL option.

Note that this filter type is not removed if the Remove All Filters button is pressed. Instead, the delete "cross" button on the filter must be pressed to remove it.

6.4.1. Overview Dashboard

When the InCenter interface is first opened, the default Overview dashboard is presented. This is an overall summary of recent activity for all monitored firewalls. This summary can be displayed at any later time by selecting the Overview option.

Overview Option Button

Figure 6.15. Overview Option Button

Below is an example of the information presented by the first part of the summary.

Overview Dashboard Example

Figure 6.16. Overview Dashboard Example

The second part of the summary provides further graphical represenations of the overall node activity plus a tabular summary of the 10 most active active nodes in terms of data volumes.

Continued Overview with Top Nodes by Data Volume

Figure 6.17. Continued Overview with Top Nodes by Data Volume

6.4.2. Network Allowed

The Network Allowed dashboards present an overview of traffic sources that have been allowed by the IP policies of monitored firewalls. An example of the initial dashboard display is shown below.

Network Allowed Dashboard - Overview Example

Figure 6.18. Network Allowed Dashboard - Overview Example

The various tabs in this dashboard present the following information:

  • Applications

    This summarizes the allowed connections that have been processed by application control.

Network Allowed - Applications Dashboard Example

Figure 6.19. Network Allowed - Applications Dashboard Example

  • Network & Policies

    This presents the source IP addresses that were allowed.

Network Allowed - Network Dashboard Example

Figure 6.20. Network Allowed - Network Dashboard Example

  • Web

    This summarizes the types of URLs that were allowed by web content filtering.

Network Allowed - Web Dashboard Example

Figure 6.21. Network Allowed - Web Dashboard Example

  • Log

    This is a list of the log messages received.

Network Allowed - Log Dashboard Example

Figure 6.22. Network Allowed - Log Dashboard Example

6.4.3. Network Denied

The Network Denied dashboards present the sources of traffic that has not been allowed by IP policies. It can be viewed as the opposite of the Networks Allowed dashboards. The tabs also present the denied networks by type, corresponding to the same tabs in the Network Allowed dashboard.

Network Denied Dashboard Example

Figure 6.23. Network Denied Dashboard Example

6.4.4. Threats Unhandled

The Threats Unhandled dashboards are similar to the Threats Denied dashboards but provide an overview of threats that were detected in the "high risk" or "very high risk" categories but which were not blocked.

It may be often desirable to introduce security policies which first audit threats instead of blocking them straight away. This dashboard can give the administrator a way to gauge the effect of changing policies from auditing to blocking.

Threats Unhandled Dashboard Example

Figure 6.24. Threats Unhandled Dashboard Example

6.4.5. Threats Denied

The Threats Denied dashboards present an overview of threats that have been blocked by monitored firewalls. An example is shown below where a time range of 24 hours has been selected.

Threats Denied Dashboard Example

Figure 6.25. Threats Denied Dashboard Example

The following tabs allow a further breakdown of the denied data:

  • Anti-Malware

    This summarizes connections dropped by anti-virus scanning.

  • Botnet

    This summarizes connections dropped by botnet protection.

  • DoS

    This summarizes connections dropped by DoS (denial of service protection).

  • IDP

    This summarizes connections dropped by the intrusion detection prevention (IDP) subsystem.

  • Scanner

    This summarizes connections dropped by scanner protection.

6.4.6. VPN

The VPN dashboards present a summary of activity related to IPsec, L2TP and PPTP tunnels. Selecting one of the protocol specific tabs provides more detailed information about each protocol.

VPN Dashboard Example

Figure 6.26. VPN Dashboard Example

Note that SSL VPN traffic monitoring is covered by Section 6.4.8, OneConnect.

6.4.7. Communication

The Communication dashboard presents a summary of the traffic that is passing through nodes between external hosts and devices. An example of the first part of the communication summary page is shown below.

Communication Dashboard Overview - Summary

Figure 6.27. Communication Dashboard Overview - Summary

The second part of the communication summary page provides a tabular summary of the IP rule set entries that are triggering within nodes in order to allow traffic. Columns within this table provide detailed information on how many connections are being allowed for each listed rule set entry and how much traffic is flowing.

Communication Dashboard Overview - Policies

Figure 6.28. Communication Dashboard Overview - Policies

6.4.8. OneConnect

The OneConnect dashboard presents the activity of users who are connecting to a NetWall node over SSL VPN using the OneConnect client. Below is an example of the first part of the summary for OneConnect.

OneConnect Overall Summary

Figure 6.29. OneConnect Overall Summary

The second part of the summary page presents the top users in terms of connections, failed connections, traffic and session time.

OneConnect Users Summary

Figure 6.30. OneConnect Users Summary

6.4.9. Log Analyzer

The Log Analyzer dashboard presents a summary of the log event messages that InCenter is receiving from configured firewalls. The graph at the top provides a graphical summary of the different categories of logs received.

Log Analyzer Dashboard Example

Figure 6.31. Log Analyzer Dashboard Example

6.5. The Health Display

The Health display for each firewall summarizes the overall health of the node by displaying the recent levels for CPU load, memory usage, data volume and concurrent connections.

The health display is different from dashboards in that the source data is node generated telemetry data that is always sent every 5 minutes, regardless of what configuration objects exist in the firewall.

To display the health of any firewall, select the node in the node list and then press the Health button.

Select Health Button

Figure 6.32. Select Health Button

The health display will open up. The screenshots below show examples of the graphs displayed.

Example Health Display - CPU

Figure 6.33. Example Health Display - CPU

Example Health Display - Connections

Figure 6.34. Example Health Display - Connections

Example Health Display - Memory

Figure 6.35. Example Health Display - Memory

Example Health Display - Data

Figure 6.36. Example Health Display - Data

The health display uses machine learning software to track the values of the health telemetry and indicate any anomalies with a red line on the graphs and add these to an anomaly list at the bottom of the page.

An Example Health Anomalies List

Figure 6.37. An Example Health Anomalies List

6.6. CyberSecurity Score

InCenter provides the Clavister CyberSecurity Score option with the monitoring feature of NetWall nodes. This option generates a set of "score" displays which are easily understood, snapshot summaries of the current security status for individual or groups of NetWall nodes, or an overall summary for all monitored nodes.

Scores can be displayed by selecting one of the menu options under the CyberSecurity Score heading in the navigation pane of the InCenter WebUI.

CyberSecurity Score Menu Options

Figure 6.38. CyberSecurity Score Menu Options

Log Data Collection and Score Calculation Frequency

The CyberSecurity Score feature works by calculating score parameters each day at midnight using the previous 24 hours of log messages received from monitored NetWall nodes. In other words, the score information presented by InCenter is a summary of the security status during the previous day. Note that the "midnight" time when the calculations are performed is determined by the time zone used by the InCenter server and not using the timezone in which individual nodes are located.

If InCenter has not yet received sufficient data, it will display the following message when a score display is requested.

The Insufficient Data Message

Figure 6.39. The Insufficient Data Message

The above message may also be displayed when the score display is requested for a particular node that has insufficient data but there is sufficient data for other nodes.

The Overview Option

The score Overview option will provide a summary for all monitored NetWall nodes. Below is an example of a typical summary display for all nodes.

CyberSecurity Score Summary View

Figure 6.40. CyberSecurity Score Summary View

By using the drop-down box on the upper-right, this display can be recalculated for individual nodes or node groups.

On the left side of this display is an alphabetical score between A (the highest level) and F (the lowest). This provides a quick indicator of overall security status.

On the right side is a threat indicator that takes a percentage value between 0 (the lowest threat level) and 100 (the highest threat level). Unlike the other measures which are averages, the threat indicator value is the highest value found among all the nodes or among those that are currently selected.

In between are colored bar meters that provide a score level between A and F for the following individual security categories:

  • Protection

    This score highlights problems due to node configurations. For example, features are disabled leaving the network open for potential attacks. It will also highlight if nodes are not using the latest software version.

  • Health

    This score highlights the health of nodes in terms of performance and connectivity.

  • Behavior

    This highlights threats from internal hosts. If threats are seen from specific nodes then it could be wise to review how these nodes are allowed to communicate on the network.

  • Users

    This highlights identity and authentication issues that may impact the availability. It will also highlight high ratios of non-authenticated traffic, which make it harder to track down the original source of threats.

  • Connections

    This score highlights the level of protection and security on VPN connections

  • Node

    This score highlights the level of protection and security status of nodes.

Under the score display is a list of Top 3 Suggested Improvements which indicates suggested ways that the overall score could be improved.

The Details Option

The Details menu option presents a more detailed view of the score for each node.

The CyberSecurity Score Details View

Figure 6.41. The CyberSecurity Score Details View

An individual node can now be selected to provide a drill-down into the individual indicators that went into how the scores were calculated.

Drill-down of the CyberSecurity Score Details View

Figure 6.42. Drill-down of the CyberSecurity Score Details View

The colored bar on the right side of each indicator gives a measurement for the contribution of that factor to the overall score for that node. A red bar shows that the indicator made a significantly negative contribution and green shows a marginally negative contribution. The indicators are initially ordered with the most negative contributors first. The recommendation is for the administrator to address the most negative indicator first in order to improve the overall score for that node.

Using the Date Picker to See Earlier Scores

When a cybersecurity score is displayed, an earlier average score on a particular day can be displayed by using the date picker. The date picker is a drop-down box above all score displays.

The CyberSecurity Score Date Picker

Figure 6.43. The CyberSecurity Score Date Picker

Clicking the picker presents a day by day calendar which shows historial summary score values. By clicking on a day in the calendar, the complete score details for that day will be displayed.

The CyberSecurity Score Date Picker Calendar

Figure 6.44. The CyberSecurity Score Date Picker Calendar

6.7. Reporting

The Reporting section of the InCenter system allows dashboards to be turned into PDF reports that can be automatically sent to a list of email recipients. The creation of reports can be done at a chosen point in time, either at the beginning of the day, week or month.

To create a new report, select the Configure option from the Reporting menu in the navigation pane. (The Preview option duplicates the information from the dashboard displays.)

Reporting Menu

Figure 6.45. Reporting Menu

Next, press the Add button to define a new report.

Add Report Button

Figure 6.46. Add Report Button

The new report dialog will be displayed. The first information to enter is a name for the report along with selecting the firewalls which will contribute the data for the report. The firewalls must have been previously added to InCenter for them to appear in the drop down list. The list also includes any groups that have been previously defined.

Add Report Dialog - Name

Figure 6.47. Add Report Dialog - Name

In the next part of the dialog, select the dashboards that will be included in this report.

Add Report Dialog - Dashboards

Figure 6.48. Add Report Dialog - Dashboards

Define the frequency that the report will be generated. This can be either at the beginning of each day or each week or each month.

Add Report Dialog - Schedule

Figure 6.49. Add Report Dialog - Schedule

Next, the recipient(s) of the report can be entered as a comma separated list of email addresses along with text for the subject line and text for the email body (the report PDF will be an attachment to emails).

Add Report Dialog - Recipients

Figure 6.50. Add Report Dialog - Recipients

Note that SMTP server setup will be required in order for InCenter to send emails. Doing this is described in Section 6.9, SMTP Server Setup.

After clicking the Save button at the bottom of the dialog, this report will now appear as one line in the report list and can be edited or deleted at any time.

Reporting List

Figure 6.51. Reporting List

6.8. Alerting

InCenter provides the ability for the administrator to be made aware when certain events occur, including the option to receive alarms for specified events via email. In describing the alerting subsystem. the following terms will be used:

  • Ticket Type

    A ticket type corresponds to a particular log event message that can be generated by a node. All ticket types are predefined in InCenter. Ticket types can be set to a state of either being enabled or disabled. When they are disabled, InCenter will ignore them if they occur. By default, all types are enabled.

  • Ticket

    This collects together all the occurrences of a given ticket type for a given node. When a ticket is created it is initially in an unacknowledged state and must be explicitly acknowledged by the administrator before it moves to an acknowledged state.

  • Alarm

    An alarm consists of an email that is sent when selected ticket types occur.

The alerting feature is managed through the Alerting menu options in the navigation pane, which are shown below.

Alerting Menu Options

Figure 6.52. Alerting Menu Options

When the Ticket Types option is selected, a list of all available ticket types is presented. An example of the first few lines of this list is shown below.

Alerting - Ticket Types List

Figure 6.53. Alerting - Ticket Types List

Each line in the list corresponds to a particular event or ticket type that InCenter will create a ticket for from incoming log messages. The slider on the right allows the administrator to choose if that ticket type should be monitored or not. By default, all ticket types are enabled. The pencil icon on the left indicates an On/Off change that has not yet been activated.

When a monitored event occurs, it is added to the Open Tickets list and the number of unacknowledged tickets in the list is indicated by a number next to the alerts icon in the toolbar.

Alerts Icon with Unacknowledged Tickets

Figure 6.54. Alerts Icon with Unacknowledged Tickets

Displaying the Ticket List

The current ticket list is displayed either by clicking the alters icon in the toolbar or by selecting the Tickets option in the navigation menu. As shown in the example below, the list displays how many times that ticket type's log event has occurred for a given node in the Events column. The Created column shows when the ticket first occurred for that node and the Updated column shows when it last occurred for the node.

Alerting - Open Tickets List

Figure 6.55. Alerting - Open Tickets List

The ticket list can be extensive so filtering is possible by type, severity or node. Below, only the tickets with Medium severity are being selected.

Alerting - Ticket Filters

Figure 6.56. Alerting - Ticket Filters

Each open ticket line corresponds to a given ticket type for a given node. Tickets remain in the Open list until they are acknowledged. This is done by first selecting a line in the Open list to open the Ticket Details dialog. An example of this dialog is shown below.

Alerting - Ticket Details Dialog

Figure 6.57. Alerting - Ticket Details Dialog

By pressing the Acknowledge button, the entire ticket is moved to the Acknowledged list (accessed through the Acknowledged tab). The acknowledged tickets will persist as an historical record for 30 days before being automatically deleted by InCenter.

Unacknowledged Tickets Limit

There is a limit of 1000 unacknowledged tickets across all ticket types and nodes. When this is reached, the oldest unacknowledged ticket will be auto-acknowledged when a new ticket occurs. Similarly, the maximum number of unacknowledged tickets across all ticket types and nodes is 1000. When the maximum is reached, the oldest acknowledged ticket is deleted when a new ticket acknowledgement occurs.

Viewing Log Messages

The Events link in the dialog shown above allows the administrator to view the individual received log messages that caused the event count for this ticket type and this node to be incremented. However, not all logs may be saved by InCenter. Some may be discarded when they duplicate previous log events or do not add extra information to them.

Alerting - Ticket Details Events

Figure 6.58. Alerting - Ticket Details Events

The Information link will provide a brief summary description of the ticket type, as shown in the example below.

Alerting - Ticket Details Information

Figure 6.59. Alerting - Ticket Details Information

Setting Up Alarms

If an email is to be sent when a specific event occurs, an alarm can be created by selecting Alarms from the Alerting menu and then pressing the Add button.

Alerting - Adding an Alarm

Figure 6.60. Alerting - Adding an Alarm

The new alarm dialog is then presented. An example of this is shown below.

Alerting - Adding an Alarm Dialog

Figure 6.61. Alerting - Adding an Alarm Dialog

In the above dialog, the administrator can select the ticket types for which alarms are to be generated and the email addresses of the recipients. An email is generated whenever any of the selected tickets occurs and sent to all the recipients.

SMTP server setup will be required in order for InCenter to send emails. Doing this is described in Section 6.9, SMTP Server Setup.

Note that an alarm is only sent the first time a ticket is generated for a given ticket type and node. In other words, only once per ticket type per node. If that ticket is then acknowledged by the administrator, it becomes possible for the same alarm to be sent again if the same ticket types occurs again.

Managing Alerting Using the CLI

Alerting can be managed using the CLI. First, change the CLI context to be Alerting and use the set command to disable the ticket types that are not of interest. For example:

admin@InCenter:/> cc Alerting 
admin@InCenter:/Settings> set TicketType virus-detected Enabled=No

The command alerting can be used to display and acknowledge tickets. For example, the following command will display all unacknowledged tickets with a high severity by using the -list option:

admin@InCenter:/> alerting -list -severity=high

The following command will list all unacknowledged tickets that have a high severity for the node called my-node1:

admin@InCenter:/> alerting -list -severity=high -node=my-node1

To show the details for a ticket, the ticket number must be specified using the -show option:

admin@InCenter:/> alerting -show=5

The -acknowledge option is used to acknowledge a specific ticket number:

admin@InCenter:/> alerting -acknowledge=106

6.9. SMTP Server Setup

Configuring an external SMTP server can be done either in the WebUI or using the CLI. Note that the InCenter SMTP client will use STARTTLS when communicating with an SMTP server.

Configuring an SMTP Server Using the WebUI

When report or alarms are configured so they are sent to an email address, an SMTP server must also be configured. This is done by pressing the SMTP Settings button in the initial WebUI page for alarms or reports. The button on the alarms page is shown below.

SMTP Settings

Figure 6.62. SMTP Settings

An external SMTP server can now be defined.

Add an SMTP Server Dialog

Figure 6.63. Add an SMTP Server Dialog

Configuring an SMTP Server Using the CLI

The following is an example of a CLI command to configure an SMTP server:

admin@InCenter:/> cc Settings 
admin@InCenter:/Settings> set SMTPServer Enabled=Yes
			Host=smtp.example.com
			FromAddress=reports_alerts@mycompany.com
			Username=mySMTPUsername
			Password=mySMTPPassword

6.10. Internet Proxy Setup

The Service Provisioning Network (SPN) is a set of servers distributed around the world to which firewalls will need to connect from time to time in order to perform functions such as updating the signature database used by the IDP subsystem. Sometimes, firewalls may not have direct Internet access and therefore cannot connect directly to the SPN. The InCenter server can solve this problem by acting as an HTTP proxy server for SPN access.

Note that this InCenter feature only provides an HTTP proxy for traffic flowing between firewalls and the SPN. It does not relay other types of HTTP traffic.

Setting up the proxy server feature requires the following steps

  1. Enable the proxy server feature in InCenter

    The CSPNProxy setting must be enabled:

    admin@InCenter:/> cc Settings 
    admin@InCenter:/Settings> set CSPNProxy Enabled=Yes
  2. Route InCenter Internet traffic

    Make sure that InCenter's underlying Linux system is configured so that there is an all-nets route for Internet traffic that is outgoing from InCenter's virtual interface.

  3. Enable the proxy server feature in on each node

    The UpdateCenter object in the configuration on each node must be changed so that the requests for database updates are sent to InCenter has the IPv4 address 10.6.101.179 the cOS Core CLI command would be the following:

    Device:/> set UpdateCenter 
    			EnableProxy=Yes
    			HTTPProxyIP=10.6.101.179
    			HTTPProxyPort=8080

    This can also be done in the cOS Core WebUI by going to Status > Update Center and selecting the Proxy tab.

    The destination IP address used is the same IP that is used for SSH management access to InCenter and the same that log messages are sent to.

6.11. Upgrading Nodes

It is possible to upgrade the version of the software running on a node under InCenter control by performing the upgrade through InCenter.

The following steps are required to perform upgrades:

  1. Upload the software upgrade package to InCenter

    Upload the relevant NetWall node upgrade package (.upg) file to InCenter using SCP. The following is an example SCP command line for doing this:

    > scp <filename>.upg admin@<InCenter_ip>:/firmware/
  2. Apply the uploaded file with the upgrade command

    The upgrade command is then used to apply the .upg file to a single node (groups cannot be used). If the upgrade package file called clavister-coscore-13.00.01.12-vmware-en.upg is to be applied to the node called my-node1, the CLI command would be the following

    admin@InCenter:/> upgrade my-node1
    			clavister-coscore-13.00.01.12-vmware-en.upg
    Upgrading to firmware:
    Version:     13.00.01
    Date:        Thu Sep 19 11:02:50 2019
    Platform:    X86
    Description: Clavister cOS Core 13.00.01.12 upgrade package.
    Warning: You are about to upgrade firmware on "my-node1" node
    Are you sure? Yes/[No]: yes

6.12. Device Monitoring

The Devices menu option presents the available data about the types of user devices which are connecting to all the NetWall nodes being monitored by InCenter. This is achieved by cOS Core performing device fingerprinting on the traffic from from devices as they connect.

Note that the term "device" in the context of InCenter refers to client equipment such as smartphones, tablets and laptop computers. It should be contrasted with the firewalls through which they are connecting which are referred to as "nodes" in InCenter.

This display is accessed via the Devices menu option and then choosing the Details option.

Devices Menu Option

Figure 6.64. Devices Menu Option

The devices that are fingerprinted are only those where the node is acting as a DHCP server to the device (or it is relaying DHCP server traffic to the device). The complete details of how cOS Core fingerprinting functions is discussed further in the Device Intelligence section of the separate cOS Core Administration Guide.

Below is an example of how the detailed device list is displayed.

Devices Details List

Figure 6.65. Devices Details List

Note that each line gets an arbitrary logical name from InCenter, which appears in the Name column. For example, device7. However, this name is used only as a unique identification by InCenter and is not used by the device itself or the node.

A textual description of the Type column icon will appear in a tooltip when hovering the cursor over the icon. An example of this is shown below.

Device Type Icon Tooltip Explanation

Figure 6.66. Device Type Icon Tooltip Explanation

The type classification comes from the Fing™ device database, which cOS Core uses to identify a device.

6.13. NetEye Monitoring

Clavister NetEye is a standalone product that performs high performance virus scanning, which includes SSL inspection. NetEye is available either as a cloud service or running on dedicated local hardware.

InCenter can be used to collect and analyze log messages generated by NetEye but, at this time, only a NetEye cloud instance is capable of sending log messages to an InCenter cloud instance. Both the NetEye cloud instance and associated InCenter cloud instance must be registered to the same MyClavister account.

When NetEye log messages are received by the InCenter server, they are stored in the same way as logs from a Clavister firewall and can be displayed in graphical form using the InCenter WebUI. This display is accessed via the NetEye menu option in the Analyze menu. Below is an example of the WebUI summary display forNetEye.

NetEye Monitoring Overview

Figure 6.67. NetEye Monitoring Overview

Setting Up Log Sending in NetEye

No configuration of InCenter is required in order to process incoming NetEye log messages. However, NetEye itself must be configured to send log messages to InCenter. This is done in the NetEye configuration page of the relevant MyClavister account and is described further in the Neteye Setup chapter of the separate InCenter NetEye Cloud Getting Started Guide.

6.14. EasyAccess Monitoring

The Clavister EasyAccess product provides a scalable and highly customizable identity and access management solution. It functions as a standalone product, separate from Clavister firewall products.

The core component of EasyAccess is a centralized server which can run on different computing platforms. The server can send log event messages about its activity to external log servers. When these log messages are received by the InCenter server, they are stored in the same way as logs from a Clavister firewall and can be displayed in graphical form using the InCenter WebUI. This display is accessed via the EasyAccess menu option in the Analyze menu. Below is an example of the WebUI summary display for EasyAccess.

EasyAccess Monitoring Overview

Figure 6.68. EasyAccess Monitoring Overview

Setting Up Log Sending in EasyAccess

No configuration of InCenter is required in order to process incoming EasyAccess log messages. However, EasyAccess itself must be configured to send log messages to the InCenter server. The details for doing this can be found in an article in the Clavister Knowledge Base at the following link:

https://kb.clavister.com//317174827

In summary, the following changes are required in the EasyAccess server configuration:

  1. Go the folder EasyAccess/Server/Config and open the EasyAccess configuration file log4j2.xml in a suitable text editor.

  2. Between the lines </RollingFile> and </Appenders>, insert the following XML:

    <Syslog
      name="CEF"
      host="<InCenter-server-IP-address>" 
      port="514"
      protocol="UDP">
      <PatternLayout>
        <Pattern>%m%n</Pattern>
      </PatternLayout>
    </Syslog>

    Note that if sending logs to a standard Syslog server instead of InCenter, the inserted lines would be:

    <Syslog
      name="CEF"
      host="<InCenter-server-IP-address>" 
      port="514"
      protocol="UDP">
     facility="LOCAL7"
    </Syslog>
  3. Change the <Logger> entry in the <Loggers> section of the file so it has an AppenderRef to the Syslog definition above:

    <Logger name="EVENT" level="INFO" additivity="false">
      <AppenderRef ref="EVENT"/>
      <AppenderRef ref="CEF"/>
    </Logger>
  4. For Windows, go to the folder EasyAccess/Server/bin and open the file EasyAccess.vmotion in a suitable text editor. Add the following:

    -Dcom.phenixidentity.globals.datetimepattern=yyyy-MM-dd'T'HH:mm:ssXXX"

    For Linux, open the file binstart-PhenixID.sh in the folder EasyAccess/Server. Add the following to the file as a single line:

    JAVA_OPTS="${JAVA_OPTS}
    -Dcom.phenixidentity.globals.datetimepattern=yyyy-MM-dd'T'HH:mm:ssXXX"
  5. After saving all changes, restart the EasyAccess server so they take effect.

Chapter 7: Managing NetShield Nodes

When any firewall is added to InCenter it means the following:

  • InCenter has the ability to communicate with that particular firewall.

  • There is a copy of that node's configuration in the InCenter database.

  • For NetShield nodes only, the local configuration in the firewall has a flag set to indicate that it is now under centralized management control. The meaning of this type of control is discussed further in Chapter 10, Centralized Management Control

  • For NetShield nodes, changes to the node configuration can be performed via the InCenter user interface and then be deployed in a single operation. The node's configuration as well as the InCenter database copy are both updated with the changes. With NetWall nodes, examining node statistics becomes possible through dashboards in the WebUI.

A RemoteMgmtSSH Object is Required on a Node

Before a NetShield firewall can be added, its configuration must be changed locally to allow InCenter access using SSH. To do this, a RemoteMgmtSSH object must be added that allows SSH access by InCenter so that the firewall acts as an administrator SSH server. Doing this is described in the separate cOS Core Administration Guide.

Along with creating a RemoteMgmtSSH object on a NetShield node , it is recommended to also make the following local changes to the node:

  • It is recommended to create a new User object on the firewall which will be the user that is used by InCenter for connection.

  • The AuthenticationProfile object on the firewall used for InCenter access should have its MaxMultipleSessions property set to at least a value of 5 or higher.

  • For direct local access to InCenter, it is recommended to disable the default admin user and instead set up a new User object in the local database for this purpose.

Note that configuring a NetWall node for InCenter access is done differently to NetShield nodes and is described in Section 6.3, Setting Up Monitoring.

Node Naming

When a node is added to InCenter, a Name must be specified. The following should be noted about this name:

  • The name must be unique within InCenter.

  • The default node name is a valid name but it can be used for only one node in InCenter because of the requirement for uniqueness.

  • The current name on the firewall will be changed to the node name specified in InCenter. In other words, the two are always synchronized.

7.1. Adding Nodes with the CLI

When managing a Clavister Next Generation Firewall for the first time using the CLI, the steps are as follows:

  • Create an RemoteMgmtSSH object on the node to allow InCenter management.

  • Access the InCenter CLI using an SSH client.

  • Add the Clavister Next Generation Firewall as a node.
  • Import the node's configuration to the InCenter database.

  • Make any required changes to the node configuration.

  • Activate all changes and then commit them.

Adding any node and importing its configuration into InCenter only has to be done once.

Adding a Node with Username/Password Authentication

Suppose the Ethernet interface of a Clavister Next Generation Firewall for management access has the IPv4 address 203.0.113.5 and that the username for administrator access is admin and the password is mysecret.

To add this node with the logical name my-node1, the CLI would be:

admin@InCenter:/> add StandaloneNode
			Name=my-node1
			IP=203.0.113.5
			Username=admin
			NodeType=NetWall
			AuthenticationMethod=Password
			Password=mysecret

Note that the NodeType property must be specified. The NodeType would be set to NetShield for NetShield nodes.

The following should be noted about the add command:

  • The NodeType property must be specified since this does not have a default value.

  • The optional parameter Group= could have been included in the above command to specify membership of some InCenter node group. Group membership can also be specified later. Using groups is described further in Chapter 15, Node Groups.

  • A license could also be optionally specified for the node using the Licensed=Yes parameter along with the License= parameter to specify the License object to use. Licenses are discussed further in Chapter 13, License Management.

Adding a Node with Public Key Authentication

Using public key authentication is the recommended method of adding a node to InCenter.

Assuming that a SSHPrivateKey object has already been defined in InCenter and the node itself has been configured for public key authentication, the modified CLI to add a node would be the following:

admin@InCenter:/> add StandaloneNode
			Name=my-node1
			IP=203.0.113.5
			Username=admin
			NodeType=NetWall	
			AuthenticationMethod=PublicKey
			PrivateKey=my-key

Note that if the node had already been added using username and password authentication, the authentication method could be changed to use public key authentication with a set command. For example:

admin@InCenter:/> set StandaloneNode my-node1
			AuthenticationMethod=PublicKey
			PrivateKey=my-key

Creating a SSHPrivateKey object is described in Section 7.5, Using Public Key Authentication.

Deploying Changes to a Node

Even though the node is added in the CLI, no subsequent configuration made with the InCenter CLI changes will be stored in the InCenter database or deployed to the node until an activate command is issued. This must be followed by a commit command to make these changes permanent:

admin@InCenter:/> activate
Activate successful
admin@InCenter:/> commit
Committed
admin@InCenter:/>

This command sequence is exactly the same as the sequence used when making the firewall configuration change directly. It makes sure that the change will not break communication with the server by demanding that a second command is entered to commit the changes.

The activate command will apply all pending changes to affected nodes at the same time. If an error occurs, all changes are rolled back. If all changes are successful, the commit command will make the changes permanent.

Importing the Node Configuration

Once a node is defined to InCenter, the next step is to import the firewall configuration into InCenter. This is done with the following command sequence:

admin@InCenter:/> node my-node1 import
Importing configuration from: my-node1
Import done
admin@InCenter:/> activate
Activate successful
admin@InCenter:/> commit
Committed
admin@InCenter:/>

This sequence results in the node configuration being read and saved in the InCenter configuration database on the server. Subsequent configuration changes made through InCenter will now be reflected in both the node configuration and InCenter's database copy.

In addition, the node is brought under centralized management control by InCenter and it will no longer be possible to change the configuration of the node by direct connection to it. This is explained further in Chapter 10, Centralized Management Control.

[Note] Note: Import requires an Activate/Commit

An import command must be followed by an activate then a commit command for the import into InCenter to be completed.

If an import operation is not performed, it will not be possible to make node configuration changes under the control of InCenter.

Nodes Can Also Be Exported

InCenter provides an export operation for a standalone node which is the opposite to import. This will push the current configuration held in the InCenter database out to the node so that it synchronizes with the database. For example:

admin@InCenter:/> node my-node1 export

The export command is discussed further in Section 4.4, Activating Changes.

Editing Node Properties with the CLI

Any of a node's properties except the name can be changed using the set command. For example, to change the IP address of the node called my-node1 to 203.0.113.9, the following command could be used:

admin@InCenter:/> set StandaloneNode my-node1 IP=203.0.113.9

7.2. Adding Nodes with the WebUI

To add a node using the WebUI, select the Nodes option from the Manage section in the navigation pane.

s

Figure 7.1. s

Press the Add button and select Node.

Option

Figure 7.2.  Option

This starts the new node wizard which will go through the following steps:

  1. Connect

    The initial step is to enter the name of the node and the details for communicating with it.

    Wizard - Connect

    Figure 7.3.  Wizard - Connect

  2. Properties

    The logical name and any node group membership are specified.

    Wizard - Properties

    Figure 7.4.  Wizard - Properties

    The IncludeInDeploy option would only be disabled if there is a requirement to not deploy configuration changes to the node. This option is explained further in Section 10.4, Node/Firewall Synchronization.

    Any license that is to be used can also be specified.

    Wizard - License

    Figure 7.5.  Wizard - License

    Licenses are discussed further in Chapter 13, License Management.

  3. Done

    In the last step, a summary of the node is displayed to confirm the details.

    Wizard - Done

    Figure 7.6.  Wizard - Done

7.3. Editing Node Properties with the WebUI

To edit the node properties in the WebUI, select the line in the main WebUI for the node, then select the Edit Properties option from the options menu on the right.

Edit Menu

Figure 7.7.  Edit Menu

This opens a dialog to allow editing of the node properties.

Edit Dialog

Figure 7.8.  Edit Dialog

Changes are not deployed after the dialog is closed. Instead, editing can be performed on several different nodes and the changes are stored until the Commit button on the right is pushed. The commit operation will then deploy all the pending changes to all nodes.

WebUI Commit Button

Figure 7.9. WebUI Commit Button

Instead of deploying changes, the administrator can choose to instead press the Reject button. This will cause all pending changes to be discarded.

7.4. Deleting a Node

Deletion of a node or HA pair from InCenter is done by selecting the node line in the main WebUI display and selecting the Delete option from the right-hand menu.

It should be noted that although a node is deleted from InCenter, it will remain under centralized management control until this control is disabled locally on the node.

Disabling centralized management control in a deleted node is discussed further in Chapter 10, Centralized Management Control.

7.5. Using Public Key Authentication

Instead of using a username and password credential pair for authentication when connecting, a node can be set up to use SSH Public Key Authentication. Doing this in the node is described on the section entitled SSH and SCP with Public Key Authentication in the separate cOS Core Administration Guide for the node. Once this is set up, the following steps can be used to add the node to InCenter:

  1. Upload the private key file to InCenter using SCP or SFTP.

  2. Create a SSHPrivateKey object for the uploaded file. This can be done using the CLI:

    admin@InCenter:/> add SSHPrivateKey
    			Name=my-key
    			KeyData=my-private.key

    Note that the KeyData value for the SSHPrivateKey object could be entered directly as a base64 encoded string, without uploading a key file. However, using a file is more convenient. If a file is used, its filetype must be either .key or .pem.

  3. Add the node to InCenter, specifying the authentication method as PublicKey.

Note that if the node had already been added with a username and password, authentication could be changed to use public key authentication by changing its properties.

7.6. Changing a Node's Management IP

Once a node is under the control of InCenter, the management IP address can be changed through the InCenter CLI. This is done by changing both of the following two IP address values:

  • The IP property of the StandaloneNode object.

  • The IP of the node's address book object assigned to the management interface.

For example, assume the node called my-node1 has the address book object if1_ip assigned to the management interface. To change the management IP, first change the IP property of my-node1:

admin@InCenter:/> set StandaloneNode my-node1 IP=192.0.2.1

Next, change the IP of the if1_ip address book object on the node:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1> set IPAddress if1_ip=192.0.2.1

Additionally, the node configuration's RemoteMgmtSSH object may also need to be changed if access from a different IP address is required.

To save these changes, they should be followed by the activate and commit commands.

7.7. Editing Node Configurations

Once a node is defined to InCenter, it is possible to change that node's configuration.

7.7.1. Editing Configurations with the CLI

To edit a node configuration in the CLI, the CLI context must be changed to be that of the target node by using the cc command. For example, to change the CLI context to be the node called my-node1, the following command would be entered:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1>

The CLI prompt changes to indicate the context has become my-node1 and a valid InCenter command can now be entered to change the configuration of my-node1.

Assume that a new IPv4 address book object called my-address1 is to be added to the node's configuration. The following InCenter CLI command would be used to do this:

admin@InCenter:/my-node1> add IPAddress
			Name=my-address1
			Address=192.0.2.1
Added IPAddress my-address1
admin@InCenter:/my-node1>

New and Old Values Are Visible Before Activating

When using the show command, the old and new values for most objects will be displayed before the changes are activated. For example, suppose that the IP address of the object my-address1 is changed with the following command:

admin@InCenter:/my-node1> set IPAddress my-address1
			Address=203.0.113.3

A show command before activating the change will now display the old and new values:

admin@InCenter:/my-node1> show IPAddress my-address1
Property                     Value           Original
---------------------------  --------------  -----------
                      Name:  my-address1
             ActiveAddress:  0.0.0.0
                   Address:  192.0.2.1       203.0.113.3
      NoDefinedCredentials:  No
            UserAuthGroups:  <empty>
                  Comments:  <empty>

Once this change is activated, the original value is lost and only the Value column would be displayed.

Configuration Changes are Stored Temporarily Until Activated

After making changes, there is a choice as to when an activate with its associated commit command is issued. Further changes on other nodes can be performed before deployment. All undeployed changes will be kept in non-volatile memory and will be retained across any type of system restart. Only the reject command will delete all pending changes.

If an activate/commit command sequence is entered at the top level CLI context, all undeployed changes for all nodes will be saved in the InCenter database and deployed. However, selective activation is also possible. If the active command sequence is entered in the CLI context for a particular node then only the changes pending for that node will be deployed. This is explained further in Section 4.5, Selective Activate.

If the activate and commit operations are performed in the current node context, the command sequence would look like the following:

admin@InCenter:/my-node1> activate
Activate successful
admin@InCenter:/my-node1> commit
Committed

The address object my-address1 has now been added to both the firewall configuration of node my-node1 as well as the copy of that configuration held by the InCenter database.

Returning to the Default CLI Context

To return to the original, default CLI context, the command is cc on its own.

admin@InCenter:/my-node1> cc 
admin@InCenter:/>
[Note] Note: The default or root CLI context

The default CLI context is also sometimes referred to as the root context.

Knowing What Has Been Modified and Discarding Changes with Reject

The following command will show a summary of undeployed configuration changes:

admin@InCenter:/> show -changes

To discard all undeployed changes, use the following command in the default CLI context:

admin@InCenter:/> reject

When the reject command is used in the CLI context for a StandaloneNode, HAPair or HAMemberNode, all pending changes for that context are discarded. For example, to reject all pending changes for the StandaloneNode called my-node1 the command would be the following:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1> reject

When performing a reject operation on an HAPair, the changes are rejected for both the contained HAMemberNode objects.

Note that if the CLI context is an HAMember then the reject command will discard any changes made to its parent HAPair object as well as the other HAMember object in the pair.

Using Reject Command Options

As stated above, the reject command on its own will reject all pending changes for the context in which it is used. However, it is possible to use the -all option to reject all pending changes for the entire system, regardless of the CLI context in which the command is used:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1> reject -all

The reject operation can also be applied to a specific configuration object. For example, if all the changes made to the IPAddress object called my-ip1 on node my-node1 are to be rejected then the command would be the following:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1> reject IPAddress my-ip1

If the reject operation is applied to an object with child objects, by default, the child objects will be unaffected. To change this default behavior so that changes to all child objects (and any children below them) are rejected, the -recursive option must be used.

Note that the reject command cannot have the object types StandaloneNode or HAPair or NodeGroup as an argument.

A Summary of CLI Configuration Editing

The following is a summary of points that should be noted when editing configurations through the CLI:

  • A node has to be added and imported before its configuration can be edited.

  • An import or series of imports requires an activate and commit before any configuration editing can be done.

  • The InCenter CLI context must be changed to be that of the node before its configuration can be edited.

  • All changes are kept by InCenter but only applied to the InCenter database and deployed to the node when an activate command is entered. A commit command make the changes permanent.

  • Before the activate command is issued, all changes are kept in non-volatile memory and will be retained through system restarts. They can be discarded with the reject command.

  • The activate and commit command sequence will make all changes permanent, regardless of the InCenter CLI context.

7.7.2. Editing Configurations with the WebUI

To edit the configuration of a node, the node must be first selected in the main WebUI node list and the Configure button pressed.

Button

Figure 7.10.  Button

Depending on the type of node, the way the configuration is edited is different:

  • For NetWall nodes, this will open a new tab in the browser and the node's WebUI will be displayed. This WebUI represents direct communication between the browser and the NetWall node and InCenter does not mediate this communication. NetWall configuration steps are therefore described in the separate cOS Core Administration Guide for the node and will not be described further in this publication.

  • For NetShield nodes, node configuration is mediated by InCenter and an InCenter WebUI page will be displayed for configuration of the chosen node.

    The rest of this section will further describe the steps for editing a NetShield configuration.

Adding a Configuration Object

As an example of configuration editing, the following steps will add a new IP address to a node configuration:

  • Select the type of object to be added or edited from the dropdown box. This displays a list of the existing configuration objects of that type.

Object Selection

Figure 7.11.  Object Selection

  • Press the Add button to open a dialog to input the property values of the new object.

New Address Object Dialog

Figure 7.12. New Address Object Dialog

  • Press the Save button to put the new object into the list of pending changes that are yet to be committed.
New Object Dialog Save

Figure 7.13. New Object Dialog Save

  • In the object list, an object that has been added but still has the commit pending will have an a plus icon next to it, as shown below.

Pending Object Addition Icon

Figure 7.14. Pending Object Addition Icon

  • The new object will appear in the address list but the change still needs to be committed by selecting first selecting the Changes menu option.

Changes Option

Figure 7.15. Changes Option

  • The pending configuration changes are now displayed. It is possible to have many pending changes in a single node. It is also possible to have pending changes from more than one node. All pending changes can now be committed to all nodes by pressing the Commit Changes button. As explained in Section 5.2, Activating InCenter Changes, it is also possible in this step to discard all pending changes or any single pending change.

Commit Changes

Figure 7.16. Commit Changes

  • A message is then displayed to confirm that the changes have been committed and will indicate if there were any configuration warnings.

Commit Confirmation

Figure 7.17. Commit Confirmation

  • Clicking on the warnings link will display the warnings. Below is an example of some warnings from adding an address object.

Commit Warnings

Figure 7.18. Commit Warnings

7.8. Deleting Configuration Objects

This section describes deleting configuration objects with the WebUI.

Deleting Configuration Objects Using the WebUI

  • Assume there is a routing table called my_rt1 in the routing table list that must be deleted. The routing table list must first be displayed and the routing table my_rt1 selected.

Initial Routing Table List

Figure 7.19. Initial Routing Table List

  • Open the function menu on the right-hand side and select the delete option from the menu.

Object Delete Option

Figure 7.20. Object Delete Option

  • The object is now marked for deletion during the next commit operation and a negative sign appears next to it in the object list.

Object Deletion Pending

Figure 7.21. Object Deletion Pending

To complete the deletion, the configuration changes must be committed in the same way that the add operation was committed earlier in this section. The deletion can also be discarded instead of being committed.

Deleting Configuration Objects Using the CLI

When using the CLI, object deletion is performed using the delete command. For example, the following will delete the address object my-ip1 on the node called my-node1:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1> delete IPAddress my-ip1

7.9. Host Key Management

Host Keys Provide SSH Authentication

The host key is a unique cryptographic key that is held by each node and which is used for SSH authentication. When a node is added to InCenter, a copy of the key is automatically added to the InCenter database.

When InCenter then communicates with the node over SSH, it checks that the same key is being used. Doing this prevents a man-in-the-middle attack where a third party might pretend to be the node.

Changing the Host Key

The administrator can change the host key used by a node by issuing the sshserver -keygen command directly on the node. For example, to change the key used by my-node1, the CLI command would be:

admin@InCenter:/> node my-node1 cli 
my-node1> sshserver -keygen

At this point, the key for the node held in the InCenter database will not match the key. Any user initiated communication attempt between InCenter and the node will fail and give an error message. For example:

admin@InCenter:/> activate 
Activation failed
Node my-node1: Host key (173a5578777da3c4f31f98e) on node
does not match stored key (173a5578777da3c4f31fasd).
Consider that your node may have been compromised.
The host key can be updated in node properties.

To update the InCenter database, the HostKey property for the node now needs to be updated by the administrator using the node key shown on the message above:

admin@InCenter:/> set StandaloneNode my-node1
			HostKey=173a5578777da3c4f31f98e

The stored key now matches the node key.

7.10. Replacing a Standalone Node

When replacing a standalone node that is under InCenter control, the following steps should be used:

  1. Remove the old node and replace it with an identical node so its network connections are the same.

  2. Connect to the new node locally and set its management IP address to be the same as the old unit. This will allow InCenter to communicate with it.

  3. Use the export command in the InCenter CLI to force the current configuration into the new node. For example:

    admin@InCenter:/> node my-node1 export

    This command does not require an activate and commit sequence.

  4. The first time this command is used, an error message will be returned indicating that the host key on the node is different to the host key held by InCenter. This situation is described in the previous section and is solved by changing the stored host key to the correct value shown in the error message. The InCenter CLI command to do this is:

    admin@InCenter:/> set StandaloneNode <node-name>
    			HostKey=<key-value>
  5. Finally, activate and commit the change.

Chapter 8: Managing HA Pairs

High Availability (HA) is a feature in firewalls that allows two nodes to form an HA pair. If one node in the pair should fail then the other node will automatically take over the failed node's functions with minimal disruption to traffic flows. InCenter can be used to manage an HA pair as a single HA Pair object.

Control

Figure 8.1.  Control

The above diagram illustrates an HA pair consisting of two nodes connected together by a sync interface connection and with the pair under InCenter control.

Methods of Adding HA Pairs

One of the following two methods can be used to get an HA pair that is under InCenter control:

  • An existing HA pair can be imported into InCenter.

  • Two standalone nodes which are not part of an HA pair can be imported into InCenter and an HAPair object can be created with them within InCenter.

These two methods are described in the sections that follow.

8.1. Adding an HA Pair

This section describes how to create an HAPair object in InCenter from two nodes. There are two possible scenarios which will be covered:

  • The two nodes are standalone and not already part of an HA pair.

  • The two nodes are already part of an HA pair outside of InCenter.

In either case, the adding process begins by adding both nodes into InCenter as standalone nodes in the normal way. Assuming that this has been done, the procedures for creating an HAPair object for these nodes are described in the subsections that follow.

The ID Defaults to 1

The ID of an HA pair created in this way always defaults to a value of 1. However, the ID can be set to as specific value by specifying it as a property of the HAPair object at the beginning of the procedure.

An Existing HA Pair Can Function During the Import Process

Like importing a standalone node into InCenter, an existing HA pair can continue to function with live traffic during the import process into InCenter.

Viewing HA Cluster Status

Once an HAPair is created in InCenter, the status of cluster can be found using the status command followed by the HAPair name:

admin@InCenter:/> status my-hapair

Similarly, the status of a single member of the cluster can be shown by using the name of the individual node instead of the hapair name. The status command is discussed further with example output in Section 4.6, Viewing Node Status.

8.1.1. Adding an HA Pair with the CLI

Creating an HA Pair from Stand Alone Nodes

This section describes how to create an HAPair object from two standalone nodes using the CLI. Assume that two standalone nodes called my-node1 and my-node2 have already been imported into InCenter. The following steps are used to create a functioning HA pair using these two nodes as the primary and secondary:

  1. Create a new HAPair object that includes the two nodes:

    admin@InCenter:/> add HAPair
    			Name=my-hapair
    			NodeType=NetWall
    			PrimaryNode=my-node1
    			SecondaryNode=my-node2

    Note that the NodeType property must be specified. The NodeType would be set to NetShield for NetShield nodes.

  2. For NetWall nodes:

    Set the Sync interface on each HA member.

    admin@InCenter:/> cc HAMemberNode my-ha-mem1 
    admin@InCenter:/my-ha-mem1> set HighAvailability SyncIface=if1 
    admin@InCenter:/my-ha-mem1> cc 
    admin@InCenter:/> cc HAMemberNode my-ha-mem2 
    admin@InCenter:/my-ha-mem2> set HighAvailability SyncIface=if1 
  3. For NetShield nodes:

    Set the Sync interface for the pair.

    admin@InCenter:/> cc HAPair my-hapair 
    admin@InCenter:/my-hapair> set EthernetInterface if2 HAType=Sync
    Updated EthernetInterface if2
  4. The shared and private IP addresses of the management interface must be entered. Assume that the management interface is If1 for NetWall and if1 for NetShield, the shared IP address is 10.6.15.88 and the private IPs for each unit are to be 10.6.15.87 and 10.6.15.89. The following are example commands to set these addresses.

    For NetWall nodes:

    admin@InCenter:/> cc HAPair my-hapair 
    admin@InCenter:/my-hapair> set IP4Address InterfaceAddresses/If1_ip
    			Address=10.6.15.88
    Updated IP4Address InterfaceAddresses/If1_ip
    admin@InCenter:/my-hapair> add IP4HAAddress
    			Name=If1_private_ip
    			Address:0=10.6.15.87
    			Address:1=10.6.15.89
    Added IP4HAAddress If1_private_ip
    admin@InCenter:/my-hapair> set Ethernet If1
    			IP=InterfaceAddresses/If1_ip
    			PrivateIP=If1_privateip
    Updated Ethernet If1

    For NetShield nodes:

    admin@InCenter:/> cc HAPair my-hapair 
    admin@InCenter:/my-hapair> set IPAddress if1_ip
    			Address=10.6.15.88
    Updated IPAddress if1_ip
    admin@InCenter:/my-hapair> add IPAddress
    			Name=private_0
    			Address=10.6.15.87
    Added IPAddress private_0
    admin@InCenter:/my-hapair> add IPAddress
    			Name=private_1
    			Address=10.6.15.89
    Added IPAddress private_1
    admin@InCenter:/my-hapair> set EthernetInterface if1
    			PrivateIP:0=private_0
    			PrivateIP:1=private_1
    Updated EthernetAddress if1
  5. An activate and commit command sequence should now be entered to save the changes.

  6. The shared and private IP addresses of the other interfaces (including the Sync interface) can now be set in the same way as in the previous step.

Importing an Existing HA Pair

The following steps should be followed to bring an HA pair under InCenter control using the CLI:

  1. Add the firewalls as individual nodes in InCenter. Doing this is described in Section 7.1, Adding Nodes with the CLI for cOS Stream and Section 6.2.3, Adding NetWall Nodes with the CLI for cOS Core. The nodes in the HA pair will be treated as being standalone, even though they are part of a functioning HA pair.

  2. Add the HA pair to InCenter by creating a new HAPair object using the add HAPair CLI command. Assuming that the names of the two nodes already imported are my-node1 (the currently active node in the pair) and my-node2 (the currently passive node in the pair). Then the CLI command would be:

    admin@InCenter:/> add HAPair
    			Name=my-hapair
    			NodeType=NetWall
    			PrimaryNode=my-node1
    			SecondaryNode=my-node2
    			ClusterID=2
  3. Enter activate and commit commands to save the changes.

Viewing HA Pair Status in the CLI

Once an HAPair is created in InCenter, the status of pair can be shown using the status command followed by the HAPair name:
admin@InCenter:/> status my-hapair
Similarly, the status of a single member of the pair can be shown by using the name of the individual node instead of the hapair name. The status command is discussed further with example output in Section 4.6, Viewing Node Status.

8.1.2. Adding an HA Pair with the WebUI

This section describes how to create an HAPair using the WebUI. The same procedure is used, regardless if the nodes are already part of an HA pair or not. The WebUI detects if there is an existing pair and alters the setup steps accordingly.

The steps to add an HAPair object with the WebUI are as follows:

  1. Add the two nodes as standalone nodes into InCenter. Doing this is described in Section 7.2, Adding Nodes with the WebUI. This must be done even if the nodes are already in an existing HA pair.

  2. Start the new HA pair wizard by selecting HAPair from the Add menu.

    Add HAPair Option

    Figure 8.2. Add HAPair Option

  3. Properties

    Specify the nodes that are to be part of the HA pair.

    Add HA Pair Wizard - Properties

    Figure 8.3. Add HA Pair Wizard - Properties

    The ID specified will replace the ID of an existing HA pair.

  4. Network

    If the nodes are not part of an existing pair then additional information will be required to set up the pair.

    Add HA Pair Wizard - Network (new pair)

    Figure 8.4. Add HA Pair Wizard - Network (new pair)

    If the nodes are already part of an existing pair then this will be indicated by the wizard and the existing network values will be used.

    Add HA Pair Wizard - Network (existing pair)

    Figure 8.5. Add HA Pair Wizard - Network (existing pair)

  5. The final step displays a summary of the HA pair that will be created.

    Add HA Pair Wizard - Summary

    Figure 8.6. Add HA Pair Wizard - Summary

    When the Done button is pressed, an HAPair object is created with any required import and activation also being performed. Previously standalone nodes will be combined into an HA pair while an existing HA pair will be imported.

8.2. Editing an HA Pair

Once the HA pair is under InCenter, there are two kinds of changes that can be performed using the CLI:

  • Changes to HAPair Context Objects

    Changes are made to the HAPair object in the same way as changes are made to a StandaloneNode object. However, when these changes are deployed, they are automatically applied to both nodes in the pair.

  • Changes to HAMemberNode Context Objects

    Each HAPair created will have two HAMemberNode objects associated with it, each corresponding to a node in the pair. Certain configuration objects are not duplicated between nodes in the pair and these must be changed on each node individually. This is done by changing the CLI context to the relevant HAMemberNode object and then entering the required commands.

Editing the Configuration of an HAPair Object

As an example of changing an HAPair configuration, to add a new IP address object to both nodes in the HA pair, the command sequence could be:

  1. Change the CLI context to be the pair:

    admin@InCenter:/> cc HAPair my-hapair 
    admin@InCenter:/my-hapair>
  2. Add the IP address:

    For NetWall nodes:

    admin@InCenter:/my-hapair> add IP4Address
    			Name=my-address1
    			Address=192.0.2.1
    Added IPAddress my-address1

    For NetShield nodes:

    admin@InCenter:/my-hapair> add IPAddress
    			Name=my-address1
    			Address=192.0.2.1
    Added IPAddress my-address1
  3. Deploy the change to the pair:

    admin@InCenter:/my-hapair> activate
    Activate successful
    admin@InCenter:/my-hapair> commit
    Committed
  4. If no other changes need to be made to the HA pair, change the CLI context back to the default:

    admin@InCenter:/my-hapair> cc 
    admin@InCenter:/>

Editing the Configuration of an HAMemberNode Object

It is possible to edit the configuration of just one of the members in an HA pair. This is done by changing the CLI context to be the individual HAMemberNode and then applying the change.

Take the following example for a NetWall node:

If a new EthernetDevice object has to be added to just one of the nodes in an HA pair. In this case, the change must be performed on the relevant HAMemberNode object since EthernetDevice is one of the configuration object types that is not available in the HAPair object.

This configuration change can be done using the following steps:

  1. Change the CLI context to be the HAMemberNode object:

    admin@InCenter:/> cc HAMemberNode my-node1 
    admin@InCenter:/my-node1>
  2. Add the EthernetDevice object with an identifier:

    admin@InCenter:/my-node1> add EthernetDevice
    			Name=my-ethdevice
    			HWIdent=pci=77:77.7
    Added EthernetDevice my-ethdevice
    admin@InCenter:/my-node1>
  3. Deploy the change to the node:

    admin@InCenter:/my-node1> activate
    Activate successful
    admin@InCenter:/my-node1> commit
    Committed

HA Pair Synchronization

When an HA pair is under InCenter control, the following should be noted:

  • Auto-synchronization of the HA pair in the firewalls is disabled.

  • InCenter deploys the configuration to each member of the pair in sequence.

  • The order of deployment can be set in the HAPair object.

8.3. Deleting an HA Pair

There are two ways in which an HA pair can be deleted from InCenter:

  • Both the HAPair object and the associated StandaloneNode objects can be deleted from InCenter. This will leave a functioning HA pair that is no longer under InCenter control.

  • The HAPair object can be deleted but not the StandaloneNode objects. This leaves two nodes which are no longer part of an HA pair but are still under InCenter control.

Performing these two operations is described next.

Deleting the HA Pair and Standalone Node

To delete the HA pair and nodes from InCenter control, the sequence of commands would be the following:

admin@InCenter:/> delete HAPair my-hapair
Deleted HAPair my-hapair
admin@InCenter:/> delete StandaloneNode my-node1
Deleted StandaloneNode my-node1
admin@InCenter:/> delete StandaloneNode my-node2
Deleted StandaloneNode my-node2
admin@InCenter:/> commit

This will remove the HA pair from InCenter control but still leave it fully functioning as an external HA pair. However, the AutoSyncCfg property of the HighAvailability object in both firewall configurations will still be disabled. It is recommended to enable this property.

Deleting the HA Pair and Leaving Standalone Nodes

Instead of removing the HA pair and its component nodes, the requirement may be to keep the nodes in the pair under InCenter control as standalone nodes.

This is done with the following steps:

  1. Delete the HAPair object:

    admin@InCenter:/> delete HAPair my-hapair
  2. The nodes are now standalone in InCenter but their management interface IP address is a shared address and must be changed before activating. If the management interface is if1, this is done with the following commands.

    For NetWall nodes:

    admin@InCenter:/> cc StandaloneNode my-node1
    admin@InCenter:/my-node1> set IP4Address if1_ip
    			Address=10.6.15.87
    Updated IP4Address if1_ip 
    admin@InCenter:/my-node1> cc
    admin@InCenter:/> cc StandaloneNode my-node2
    admin@InCenter:/my-node2> set IP4Address if1_ip
    			Address=10.6.15.89
    Updated IP4Address if1_ip
    admin@InCenter:/my-node2> cc
    admin@InCenter:/> activate
    admin@InCenter:/> commit

    For NetShield nodes:

    admin@InCenter:/> cc StandaloneNode my-node1
    admin@InCenter:/my-node1> set IPAddress if1_ip
    			Address=10.6.15.87
    Updated IPAddress if1_ip 
    admin@InCenter:/my-node1> cc
    admin@InCenter:/> cc StandaloneNode my-node2
    admin@InCenter:/my-node2> set IPAddress if1_ip
    			Address=10.6.15.89
    Updated IPAddress if1_ip 
    admin@InCenter:/my-node2> cc
    admin@InCenter:/> activate
    admin@InCenter:/> commit

This leaves two standalone nodes still under InCenter control.

[Important] Important: Initial management IP must not be the shared IP

Avoid using the shared management IP as the initial standalone management IP for either node in an HA pair, or as setting the shared IP as the management IP when removing them from the HAPair object to become standalone nodes. This could result in both nodes using the same management IP, which can be a valid scenario if the deletion from InCenter is reverted at this stage.

If this needs to be done then first assign some temporary management IP in step 2 above. Then change this to the shared IP later, after the activate and commit.

8.4. Replacing an HA Pair Node

When replacing a node in an HA pair, the following steps should be used:

  1. If configuration changes need to be made before the node is replaced (so the pair has only one working node) then the HAMemberNode object property IncludeInDeploy should be set to a value of No for the node that is not working. This will mean that InCenter will not try to deploy configuration changes to this node. Deployment will fail if this setting is not changed and the node is not operational.

  2. Remove the old node and replace it with an identical node so that its network connections are the same.

  3. Connect to the new node locally and set its management IP address to be the same as the old unit. This will allow InCenter to communicate with it.

  4. Use the export command in the InCenter CLI to force the current configuration into the new node. For example:

    admin@InCenter:/> node my-hapair export

    This command does not require an activate and commit sequence.

  5. If IncludeInDeploy was disabled in the first step, this must be enabled again before the new configuration can be deployed to the new node.

8.5. Upgrading an HA Pair

When upgrading the firewall software version on the nodes in an HA pair, the procedure is similar to upgrading a standalone node:

  1. Upgrade the versions on each HA pair node in the normal standalone way outside of InCenter so that each has the same software version. Upgrading is described in the separate Administration Guide for the firewall software.

  2. The next time an activate and commit operation is performed on the HAPair object, InCenter will automatically update its stored version number of the software.

Chapter 9: Managing Users

The InCenter server maintains a database of InCenter users. That is, the users that are allowed to login and make use of InCenter. When the InCenter server is started for the first time, there is a single predefined user account already set up with the following credentials:

  • Username: admin

  • Password: admin

Any number of new InCenter user accounts can be created for administration and/or auditing purposes. Having a number of different users can be useful for logging purposes, where the username responsible for an action is included in that action's log message and history entry.

[Note] Note: User changes are not part of the history

It should be noted that modifications to the user database and are not themselves kept in InCenter's revision history and therefore user changes cannot be reverted in the same way that other changes can.

Authentication Methods

The following user authentication methods are available:

  • Authentication with Username/Password Credentials

    This default type of authentication involves the user entering credentials consisting of a username and password combination.

  • SSH Public Key Authentication

    It is possible for a user to have CLI access to InCenter using SSH with public key authentication instead of using username/password credentials. Setting up public key authentication for a user is described in Section 3.1, SSH Access to the CLI.

  • Multi Factor Authentication in the WebUI

    InCenter can be configured to use Multi Factor Authentication (MFA) where WebUI authentication is done using an external Clavister EasyAccess authentication server. This is discussed further in Section 9.4, Setting Up MFA.

User Access Levels

Every InCenter user has one of the following two values for its AccessLevel property:

  • Administrator

    The user has full system access and can read or change any part of the configuration. InCenter will not allow the number of this type of user to fall below one.

  • Auditor

    The user has no ability to change the configuration or affect the operation of InCenter but has read-only access to the complete configuration. The user also has the ability to execute informational CLI commands such as the techsupport and statistics commands.  

    Also note that an auditor cannot read or download files in the InCenter system using SFTP or SCP. However, they can read or download such files using the REST API.

The access level does not have a default value so it must be specified for every user that is added to the system. Users that already existed prior to upgrading to a version with the access level feature will automatically get the Administrator level.

Conforming to the User Password Policy

When the InCenter PasswordPolicy object is enabled (it is by default) then the password which is set for a new user must conform to the length and content specified by the policy. By default, the password length must be not less than 8 characters and must contain at least one upper case, one lower case, one numeric plus one non-alphabetic/non-numeric character. This is explained further in Section 9.3, User Password Policy.

Simultaneous Access by Multiple Users

It is possible for more than one InCenter user to be logged in at the same time to the same InCenter server instance. It is also possible for the same user to be logged in at the same time from multiple client computers.

In the case of such multiple login sessions by an administrator user, all sessions will see and update the same configuration set. In other words, the changes made in one session will affect all other concurrent sessions.

In the case of an auditor user, they can see the changes being made by an administrator user but cannot activate or otherwise affect such changes.

Disabling Users

By default, a user is enabled. However, it is possible to disable a user account. This means that the user remains in the user database but their credentials will not be recognized at login.

9.1. Managing Users with the CLI

In the CLI, all InCenter user account management is performed exclusively using the user command. For example, adding a new user account to InCenter using the CLI is done with the following command:

admin@InCenter:/> user -add
			-username=user2
			-password=mypassword
			-role=Administrator

Unlike other configuration changes, modifying the user database does not require that a activate and commit command sequence is entered to apply changes. User changes are applied immediately. In addition, user changes are not part of the InCenter configuration history.

Modifying an Existing User

The -update option is used when modifying an existing user. For example, to change a password:

admin@InCenter:/> user -update -id=user2 -password=mypassword2

Note that once a user exists in the user database, it is identified in the user command with the id=<username> parameter.

Displaying All Users

The CLI command for displaying all users is the following:

admin@InCenter:/> user -show
   Username Role          Comments	
-- -------- ------------- ----------
   admin    Administrator
   user1    Auditor
   user2    Administrator

Here, the output shows three user accounts: admin which is the default user account plus two others called user1 (an auditor) and user2 (a second administrator).

Deleting a User

The CLI command for deleting a user, say the user called user3, is the following:

admin@InCenter:/> user -delete -id=user3

Disabling a User

The following command can be used to disable a user:

admin@InCenter:/> user -disable -id=user1

When a user is disabled, a small "o" symbol will appear next to them when all users are listed:

admin@InCenter:/> user -show
   Username AccessLevel   Comments	
-- -------- ------------- ----------
   admin    Administrator
 o user1    Auditor

To re-enable the same user, the command would be:

admin@InCenter:/> user -enable -id=user1

9.2. Managing Users with the WebUI

To display all current system users names in the WebUI, first press the Admin button at the top right of the WebUI display and select Manage Users.

Selecting Manage Users

Figure 9.1. Selecting Manage Users

A list of current users will be presented. To add a new user, press the Add button.

Add User Selection

Figure 9.2. Add User Selection

The new user dialog will now be presented and the new user credentials can be entered.

New User Dialog

Figure 9.3. New User Dialog

In addition, the access level for the user can be set as either Administrator or Auditor.

When the user details have been entered, press the Save button to add the new user.

Unlike other configuration changes, modifying the user database does not require that the change is activated before it is applied. User database changes are applied immediately.

9.3. User Password Policy

The user password policy for InCenter is determined globally by a single predefined PasswordPolicy object.

The following are the PasswordPolicy object's properties:

  • Enabled

    When set to a value of No, the password policy is not applied. The default value is Yes which means that all user passwords must conform to the policy parameters specified by the object.

  • MinimumPasswordLength

    The minimum number of characters in passwords. The minimum allowed length is 4. The default value is 10.

  • PasswordExpiryInDays

    The number of days before a user is prompted to change the current password. A value of zero (the default) means there is no expiry time and the current passwords can be used indefinitely.

  • HistoryCount

    The number of password changes required after a password is changed before that same password can be reused again. The maximum value allowed is 30 changes before reuse. A value of zero (the default) means there is no limit on password reuse.

  • PasswordComplexity

    If disabled, passwords can consist of any keyboard characters. If enabled (the default), a new password must contain all of the following:

    1. At least one lowercase character.

    2. At least one uppercase character.

    3. At least one numeric character.

    4. At least one character which is non-alphanumeric. For example, characters such as "?" or "!" or "*".

Changing the Password Policy

The following command is an example of changing the current password policy:

admin@InCenter:/> set PasswordPolicy
			MinimumPasswordLength=15
			PasswordExpiryInDays=365
			HistoryCount=24

Disabling the Password Policy

Disabling the password policy (it is enabled by default) is done with the following command:

admin@InCenter:/> set PasswordPolicy Enabled=No

Behavior of the Default Administrator Password

Even though the PasswordPolicy object is enabled by default, the predefined management user called admin still has the predefined weak password of admin and this will allow the administrator to log in.

However, as soon as any change to the InCenter system is deployed, the admin user will be forced to change the password to one that conforms to the password policy on the next occasion they try to log in. This forced change can only be avoided by disabling the PasswordPolicy object.

Passwords Can Contain Spaces and Quotes

Passwords in InCenter follow the same rules as any other string value specified in the CLI. This means they contain spaces and quotes. For the rules of how strings can be specified, see Section 4.2, Specifying String Values.

Switching the Password Policy to On from Off

If the PasswordPolicy object is disabled, then enabled later, any user that was created in-between with a non-conforming password will be prompted to change to a conforming password when that user next tries to log into the system.

Upgrading From Older InCenter Versions

After upgrading from any InCenter version that lacks the password policy feature to a version that has it, the PasswordPolicy object is always disabled by default. The administrator must explicitly enable the object if its password requirements are to be applied.

Password Display in the User Interfaces

For security, InCenter will not display passwords in the user interface. Instead, a number of asterisk characters will be displayed and the number will match the actual length of the password. This approach is also used with other types of sensitive information that is displayed by InCenter, for example with certificate data.

9.4. Setting Up MFA

As mentioned previously, Multi Factor Authentication (MFA) can be enabled for user logins. When the user tries to login using the WebUI, they can be redirected to an external Clavister EasyAccess authentication server. This server will use SAML to return the authentication status to InCenter. Depending on how the authentication server is configured, additional factors (such as a code sent to the user's phone) can be requested by the authentication server during the login process.

Enabling MFA

The following steps are needed to enable MFA:

  1. For each user in the InCenter user database that are to be authenticated using MFA, the samlUsername property must be set. For example:

    admin@InCenter:/> user -update -id=admin -samlUsername=myEAusername

    The samlUsername is the username that is expected by the authentication server when it authenticates the user. It can be the same or different from the standard username property.

  2. In EasyAccess, a SAML Metadata object should be created which specifies that the service provider metadata is downloaded from the following URL:

       https://<incenter-server-ip>:8443/saml/mgmtsystem/metadata.xml

    A suitable EasyAccess Federation Scenario is then created to authenticate users, possibly using secondary factors such as tokens (any of EasyAccess's MFA features could be used). This Federation Scenario references the SAML Metadata object created above and will also reference a data source of valid users.

    Further information about EasyAccess setup can be found in the separate EasyAccess Getting Started Guide.

  3. Using SCP or SFTP, upload the following files in PEM format to InCenter:

    • The public key file(s) of the identity provider (such as EasyAccess).

    • The public key file of the service provider (in this case, InCenter).

    • The private key file of the service provider (in this case, InCenter).

    The service provider certificate files must be generated externally using a suitable tool. Also note that any of these files can be alternatively specified in the InCenter CLI as a base64 encoded string.

  4. Using the InCenter CLI, enable the SAML object in Settings and assign values to its properties:

    admin@InCenter:/> cc Settings 
    admin@InCenter:/Settings> set SAML Enabled=Yes
       IdentityProviderCertificates=my_idp.crt
       LoginUrl=https://<auth-server-ip>:8443/saml/authenticate/samlauth
       LogoutUrl=https://<auth-server-ip>:8443/saml/authenticate/SLO
       ServiceProviderCertificate=my_sp_rsa.pem
       ServiceProviderPrivateKey=my_sp_rsa
       IdentityProviderType=EasyAccess

    Note that the IdentityProviderCertificates property could be assigned a comma separated list of more than one public key file.

    In the corresponding EasyAccess federation scenario setup, the LoginUrl corresponds to the scenario's POST SSO URL property and the LogoutUrl corresponds to the POST SLO URL property.

  5. After activate and committing the InCenter configuration changes, MFA is set up.

The MFA Login Sequence

The user login sequence sequence through the WebUI is the following:

  1. The user connects to the InCenter server as usual through a web browser.

  2. If MFA has been enabled for the system, a button for EasyAccess login appears at the top of the login dialog.

  3. The user presses the button and is redirected to web pages displayed by the authentication server. The pages displayed and the extra authentication factors required depend on how the authentication server is configured.

  4. After authentication is successful, the browser returns to the InCenter overview page with the user logged in.

Notes on the MFA Feature

The following should be noted about using MFA with InCenter:

  • The MFA option is only available with on-premises InCenter.

  • MFA can only be enabled and configured using the InCenter CLI, as described above.

  • Only a single authentication server can be specified for each InCenter server instance.

  • Logins using MFA can only be done through the WebUI.

  • The MFA option in the WebUI login dialog will only appear if MFA is enabled.

  • A login using MFA for a user will fail if their samlUsername property in InCenter is not set. Additionally, the value set must also exist as a username in the authentication server's data source.

  • Enabling or disabling MFA is applied to all users in the InCenter user database. It cannot be done on a per user basis. Disabling MFA is done with the command:

    admin@InCenter:/> cc Settings 
    admin@InCenter:/Settings> set SAML Enabled=No

    Note that disabling MFA will not affect any samlUsername values that have already been set.

  • The access privileges (administrator or auditor) for a user logging in using MFA is determined by the user's entry in the InCenter user database and not by the authentication server.

  • Clavister has fully tested the MFA feature with Clavister EasyAccess. Other, third-party authentication servers may function correctly but are not officially supported by Clavister.

Chapter 10: Centralized Management Control

10.1. Overview

When any Clavister Next Generation Firewall is imported as a node into InCenter, a software flag is set on that node to indicate that it is now under Centralized Management Control.

This means that it is no longer possible to change that node's configuration using a direct management connection and all configuration changes must be made via the InCenter interface.

[Note] Note: Management by InControl is incompatible

If a node is already under centralized management by the Clavister InControl product, this must be first disabled before bringing it under InCenter centralized control. However, using only InCenter monitoring functions is possible with such a node without disabling management by InControl.

Consequences of Centralized Control

While under the centralized control of InCenter, the following will be true for an administrator that connects directly to the firewall.

  • When the administrator connects via an SSH console or when the firewall restarts, the local firewall console will display a message to indicate that it is under centralized management control.

  • The administrator will only have read-only access to the configuration and is restricted to commands like show for viewing configuration data as well as other data such as log messages.

  • The following local operations are exceptions and can be performed while under centralized management control:

    1. Upgrading the InCenter version.

    2. Uploading a new InCenter license file.

    These operations can only be carried out directly on the firewall and none of them will change the centralized control status.

10.2. Disabling Centralized Management Control

If the firewall is under centralized management control, this can be disabled with the following steps:

  1. Delete the node in InCenter.

  2. Enable local control directly on the firewall.

The second step is necessary because even though a node is deleted from InCenter, its local firewall configuration will continue to have its centralized control flag set. This centralized control flag can be disabled by connecting to the firewall directly and entering the following firewall CLI command:

System:/> localconfiguration -enable

The disabling of centralized management will occur immediately after this command is entered but local activation of the configuration is required to make the change permanent.

The localconfiguration command without parameters will indicate if the firewall is currently under centralized management or not.

10.3. Re-enabling Centralized Management Control

If centralized management has been disabled using the firewall command localconfiguration -enable but the node has not been deleted from InCenter, changes to the configuration can still be entered using InCenter. It is only when those changes are activated that InCenter will give a message indicating that the node is no longer under centralized management control and the changes cannot be deployed.

After locally ending centralized management control but not deleting the node from InCenter, it is possible to bring back the node under centralized management control again. This is done in different ways, depending on whether it is for a standalone node or an HA pair:

  • For a Standalone Node

    Perform an export operation. This will push the current InCenter database version of the configuration back to the node, overwriting the node's current configuration and bringing it back under centralized management control.

  • For an HA Pair

    In InCenter, force the current HA pair configuration in the InCenter database to be pushed out to the pair by using the node <node-name> export command. For example:

    admin@InCenter:/> node my-hapair export

    Any configuration changes made to the HA pair while it was not under centralized management control will be lost and these will need to be re-applied in InCenter.

10.4. Node/Firewall Synchronization

The configuration of the StandaloneNode, HAMemberNode or HAPAir in InCenter should be synchronized with the configuration on the actual firewall or HA cluster firewalls.

After configuration changes are made in InCenter, this synchronization is automatically maintained following an activate and commit sequence, when the changes are both saved in InCenter and deployed to the firewall.

However, synchronization can be deliberately switched off by setting the IncludeInDeploy flag for a node to No. For example:

admin@InCenter:/> set HAMemberNode my-ha-member1
			IncludeInDeploy=No

In this case, the InCenter configuration is updated after activate/commit but the firewall is not and becomes out of sync. This can be seen in the last line of the message generated by the status command which contains the text "Out of sync", as seen in the example below:

admin@InCenter:/> status my-ha-member1
This node is an HA Master
Uptime: 63 days 10:04:56
HostKey: SHA256:Ruqn2YcxEmU0Kr+VuFAldELmgY8+lKCfisJ+RE
Config revision: Out of sync - local 21 (2018-07-03 10:56:12)
/ remote 20 (2018-06-27 14:51:10)

The principal use-case for switching off synchronization is if one of the members of an HA cluster has failed. In this case, InCenter needs to be directed to deploy changes only to the working firewall in the cluster, otherwise the deployment will fail.

If the IncludeInDeploy property is changed from No to Yes then the firewall can be synchronized in the following ways:

  • An activate/commit sequence following configuration changes.

  • By using the export command to push the InCenter configuration to the firewall.

Chapter 11: Version Management

Managing Different Firewall Software Versions

InCenter is able to manage nodes running different firewall software versions. This section describes the issues related to managing nodes running various software versions and also describes these node's relationship with the version of InCenter itself.

Note that for cOS Stream software, version management by InCenter is only available from node software version 3.00.00 and later.

Upgrading Both InCenter and Nodes

For InCenter versions after 1.67.00 there is no longer a tight coupling between InCenter and the software running on the nodes that it manages. This means that the node's software version can be upgraded without requiring a matching upgrade of the InCenter version.

However, it is still recommended that the InCenter version is upgraded when a new version becomes available. When upgrading both the InCenter version and the software version of the nodes it manages, InCenter should be upgraded first.

Upgrading of node software is done as normal, by connecting to each node directly, outside of InCenter, and uploading the relevant upgrade package. This is described in the separate Administration Guide for the node software. Upgrading the node will not, by itself, change the version information held by InCenter.

InCenter Upgrades Will Change the Highest Supported Version

In order for InCenter to be able to correctly manage a particular version of firewall software, its database must contain the full details of the features in that version. It is for this reason that InCenter may need to be upgraded together with the firewalls under its control.

The administrator may not be able to configure a particular feature in a node of the InCenter software version is unaware of this feature.

Updating the InCenter Node After Upgrading the Firewall

There are two software version numbers when InCenter manages firewalls: the software version on the firewall itself and the version in InCenter's local Node object. Ideally, they should be the same but could become out of sync after an upgrade of the firewall's software.

Resynchronization of the versions is performed automatically by InCenter when deploying configuration changes. Whenever a configuration change is deployed with the activate command, InCenter checks that its node version number matches the version number of software on the firewall itself. If the firewall has a higher version than the node in InCenter then the InCenter version is automatically upgraded before deploying the new changes and outputs a message to indicate this. A typical message is shown in the example CLI below:

admin@InCenter:/> activate
Operation in progress...-
Activate successful
Nodes activated: my-node1
Notices:		
-StandaloneNode my-node1:
Configuration upgraded from version 3.20.03 to 3.30.00
Run the "commit" command to keep the new configuration
admin@InCenter:/>

However, If the firewall software version is less than the node version held by InCenter, the activate will fail and a message is output to indicate this. The firewall software must then be upgraded before the activate can succeed.

If the activation process fails, perhaps because of a problem with a configuration change, then the node's version information in InCenter will be left unchanged.

The automatic upgrade of node version held by InCenter will work in the same way on both an HAPAir and a StandaloneNode.

Displaying the Node Software Version

A data item called Version is displayed by InCenter when it shows node information. This is the firewall software version number which is in the InCenter database for that node.

Below is an example of the version value being displayed for a node called dev1:

admin@InCenter:/> show StandaloneNode
   Name  Username  IP           Port  Version
-  ----  --------  -----------  ----  ------------
   dev1  admin     10.6.44.135  22    3.30.00.05

The version field might be blank if the node has been added but the configuration has not yet been imported or the import failed for some reason, such as having an old version which InCenter does not support.

If there is a mismatch with the software version in the InCenter database because the node has been upgraded, InCenter will detect this on the next activate operation and convert its database version so it matches.

Importing an Unsupported Version Will Not Be Allowed

If the administrator tries to add and then import a node whose software version is not supported by InCenter (because InCenter has not been upgraded), then the import will fail and InCenter will produce a message saying that the version is unsupported.

Chapter 12: Revision History

Overview

InCenter maintains a revision history which stores all the changes made to the nodes under its control as well as InCenter itself. This history allows the administrator to view the changes made with each deployment and to also undo changes by restoring the node configuration to a particular point in the history. The CLI command revision is used to interact with the revision history.

InCenter User Database Change History is not Kept

It should be noted that there is one exception to the statement that the revision history stores all InCenter configuration changes. Changes to the InCenter user database and to the User objects in the database are not kept in the revision history and therefore cannot be reverted using the feature. User changes are also applied immediately when they are made and do not require activation.

Revision Command Options

The following are the revision command options for different actions:

  • -list : List history entries.

  • -show : Show a history entry.

  • -revert : Revert to a given entry in the history.

  • -diff : Show the difference between two entries.

  • -delete : Delete an entry and all entries preceding it.

Note that changes become part of the revision history only when they are committed.

Listing the Revision History

The revision command with no parameters or with the -list option will list the entire revision history:

admin@InCenter:/> revision -list
ID Date                User   Comment                 Nodes
-- ------------------- ------ ----------------------- --------
1  2017-05-10 15:35:48 System System created revision

The above output shows the revision history before adding any nodes or deploying any configuration changes. Each revision gets a unique ID number to identify it and the first (ID 1) represents the initial InCenter configuration after installation.

Suppose a new node is added to InCenter and this change is committed. This history might now look like the following, with the latest revision appearing at the top of the list:

admin@InCenter:/> revision -list 
ID Date                User    Comment                 Nodes
-- ------------------- ------  ----------------------- --------
2  2017-05-11 09:16:03 admin   Adding a node           my-node1
1  2017-05-10 15:35:48 System  System created revision

To list the revisions for only a selection of nodes, the -nodes option can be used followed by a comma separated list:

admin@InCenter:/> revision -list -nodes=my-node1,my-node2

Restricting Command Output

To restrict the number of entries displayed when listing the history, the -num option can be used:

admin@InCenter:/> revision -list -num=10

To restrict the entries displayed to a particular user, the -user option can be used:

admin@InCenter:/> revision -list -user=admin

The -comment option can be used to list only items with matching comments. The string match can be a partial match anywhere within the comment for the history item to be listed. For example:

admin@InCenter:/> revision -list -comment=ing 
ID Date                User    Comment                 Nodes
-- ------------------- ------  ----------------------- --------
2  2017-05-11 09:16:03 admin   Adding a node           my-node1

Applying Commands to a Single Node

When using the revision command, the action specified can be applied to a single node by changing the CLI context to be that node. For example:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1> revision -list 
ID Date                User    Comment                 Nodes
-- ------------------- ------  ----------------------- --------
2  2017-05-11 09:16:03 admin   Adding a node           my-node1

Using the Revision ID Number

Each revision in the history has a unique ID number and this can be used to apply an action only to a specific revision. One example of usage is when listing the details of an individual revision in conjunction with the -show option. To list the details of the revision with an ID of 2, the command would be the following:

admin@InCenter:/> revision -show 2
Property          Value
----------------  ---------------------------
       Revision:  2
       Username:  admin
           Date:  2017-05-11 09:16:03 (+0000)
        Comment:  Adding a node
          Nodes:  my-node1
		

The following shortened form of the command will display the same output:

admin@InCenter:/> revision 2

Note that the ID of a history entry stays the same, regardless of the CLI context.

Reverting to a Previous Entry

It is possible to undo the last committed InCenter change by reverting to the previous entry in the history. For example, suppose the last action was to add a new node. To undo this action, the option -revert previous can be used. Like other configuration changes, this must be followed by an activate and commit sequence to save the change:

admin@InCenter:/> revision -revert previous
Revert action was successful
admin@InCenter:/> activate
Activate successful
admin@InCenter:/> commit
Commit successful
admin@InCenter:/> revision
ID Date                User   Comment                     Nodes
-- ------------------- ------ --------------------------- --------
4  2017-05-11 09:20:03 admin  Reverting node add          my-node2
3  2017-05-11 09:18:15 admin  Adding a node               my-node2
2  2017-05-11 09:16:03 admin  Adding a node               my-node1
1  2017-05-10 15:35:48 System System created revision

Note that the revert does not delete any history entries. Instead it adds a new entry to indicate that a revert was performed.

A revert action can be applied to only a specific node:

admin@InCenter:/> revision –revert previous
			StandaloneNode my-node1

This action could also be applied to an HAPair:

admin@InCenter:/> revision –revert previous
			HAPair my-hapair1

Reverting to a Given Entry

To revert to a specific point in the history, the ID number is used with the -revert option. For example, to revert to the initial configuration then the command would be:

admin@InCenter:/> revision -revert 1
Revert action was successful

Reverting a Node Deletion

If a node has been removed from InCenter using the delete StandaloneNode command, the deletion can be undone by using the revision -revert -node command. For example:

admin@InCenter:/> revision -revert -node=my-node1

When a deleted node is restored in this way, it is the original configuration held by InCenter that is pushed to the node. An import of the node's configuration does not take place.

Note that the -node= parameter cannot take a list of nodes. However, the -node= parameter can specify an HAPair object.

Applying Actions to a Specific Object

It is possible to apply the actions of the revision command to a specific object within a node configuration or within the InCenter system by changing the CLI context to be that object. Suppose that the administrator wants to revert to the previous version of the OSPFProcess object called my-proc in the configuration for the node called my-node1. The CLI commands would be the following:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1> cc OSPFProcess my-proc 
admin@InCenter:/my-node1/OSPFProcess/my-proc>
				revision -revert previous

Now consider reverting the last change to an object within the InCenter system. For example, it might be desirable to revert the last change made to the system settings:

admin@InCenter:/> revision -revert previous Settings

Note that this will revert all the children of Settings to the value they had before the last activate and commit that affected the settings.

It is important to understand that the revert action will affect not only the target object but all the children of the object (but any parent is left unaffected). In the OSPF example above, the OSPFProcess object will probably have one or more OSPFArea objects and these too will be reverted as well as their own child objects.

Comparing History Entries

The -diff option can be used to show the changes between two points in the revision history. For example, to show the difference between the current configuration and the history at ID number 5:

admin@InCenter:/> revision -diff 5
Identifier                   Old Value  New Value
---------------------------  ---------  ------------
|- StandaloneNode testDev
|  |- System
|  |  |- Comments                       My comment
|  |- IPAddress my_ip1                  <added>

The difference between two IDs in the history can be displayed. For example, to show the difference between the entries with IDs 5 and 6:

admin@InCenter:/> revision -diff 5 6
Identifier                   Old Value  New Value
---------------------------  ---------  ------------
|- StandaloneNode testDev
|  |- System
|  |  |- Comments                       My comment
|  |- IPAddress my_ip1                  <added>

Deleting History Entries

It is possible to use the -delete option to delete a specific entry in the history. Whenever this is done, all earlier entries before the targeted entry are also deleted. In other words, all entries with a lower ID number are also deleted. This deletion will also include the initial default system entry that exists before any other history entries are added.

Suppose the current history is the following:

admin@InCenter:/> revision
ID Date                User   Comment                     Nodes
-- ------------------- ------ --------------------------- --------
3  2017-05-11 09:20:10 admin  Adding a node               my-node2
2  2017-05-11 09:16:03 admin  Adding a node               my-node1
1  2017-05-10 15:35:48 System System created revision

Now apply a delete operation from ID 2:

admin@InCenter:/> revision -delete 2 
Warning: Revision 2 and all older revisions will be deleted
Are you sure? Yes/[No]: Yes 
Delete action was successful

Relisting shows what remains after the delete:

admin@InCenter:/> revision
ID Date                User   Comment                     Nodes
-- ------------------- ------ --------------------------- --------
3  2017-05-11 09:20:10 admin  Adding a node               my-node2

Note that the delete operation always requires confirmation from the administrator before continuing.

Automatic Cleanup

It is possible to enable an automatic cleanup feature for the revision history using the following command in the Settings context:

admin@InCenter:/> cc Settings 
admin@InCenter:/Settings> set RevisionHistory
			CleanupEnabled=Yes

Once enabled, the revision history will be automatically cleaned up after each activate and commit sequence. This cleanup consists of both deleting older entries before a given cutoff date (the default is 30 days) as well as making sure that the deletions do not reduce the history to less than a minimum number of history entries (the default is 30 revisions).

The default values for the cleanup can be changed. For example, if any entry older than 60 days is to be deleted but deletions cannot reduce the history to less than 10 entries, the command would be:

admin@InCenter:/Settings> set RevisionHistory
			MaximumDaysToKeep=60
			MinimumRevisionsToKeep=10

Automatic cleanups are performed after a commit operation completes so the history entry added by the commit will be included in the calculation to determine which entries should be deleted.

The lowest value that can be entered for both the property MaximumDaysToKeep and the property MinimumRevisionsToKeep is zero. If both are set to zero then no revision history will be retained.

Note that the cleanup process applies to the entire revision history and it is not possible to apply the cleanup to just one node.

The Software Version in History Entries

Each entry in the revision history has associated with it the software version number associated with the node to which the change was applied. Whenever a configuration change is committed, the version number is stored with the entry.

If a committed change causes the node version number to be changed (because the software version on the firewall was higher) then it is this updated version number that is stored with the history entry for the commit.

In some cases, the history entry may just be for such a node version change. Performing an import of a node where the software version was higher in the actual firewall is an example of when this happens.

Reverting to a Different Software Version

The software version associated with a history entry becomes important when a revert operation is performed to a previous history entry with a different (usually lower) software version.

In this case, the version number of the original history entry is ignored. The current version number will remain unchanged and an object or an entire configuration in the revision history will be upgraded to the current version number. For a single object, this upgrade is performed during the revert operation. For an entire configuration, the upgrade is done after the activate operation and a message is generated to indicate this. An example of this is shown below:

admin@InCenter:/> revision -revert 3 
Revert action was successful
admin@InCenter:/> activate 
Operation in progress...
Activate successful
Notices:
-StandaloneNode my-node1: Configuration upgraded
from version 3.20.02.06 to 3.30.00.02
Run the "commit" command to keep the new configuration

Note that it is not possible to revert to a lower version of the software directly using InCenter. However, one way to do this would be to install the lower version on the firewall directly and then re-import the node into VFM. Another way would be to use the following steps:

  1. Install the lower version on the firewall directly.

  2. In InCenter, perform a revert operation to a history entry for the node with a software revision that matches the revision on the firewall.

  3. Perform an activate and commit for the node.

Chapter 13: License Management

InCenter provides the ability to centrally manage the licenses for multiple nodes. A license is contained in a license file and one or more license files can be uploaded to InCenter. These licenses can then be deployed as needed to one or more nodes that are under InCenter control.

License management in InCenter can be used for both NetShield and NetWall nodes. However, site licensing applies only to NetShield nodes.

There are two kinds of licenses that can be uploaded to InCenter and then deployed to nodes:

  • Site Licenses

    A single site license can be deployed to one or more arbitrary nodes. The site license will allow a fixed level of aggregate throughput for all the nodes which have it.

  • Node Licenses

    A node license is the same as the normal license used for nodes that are not under InCenter control. A single node license is intended for only one specific node and cannot be used with any other nodes.

InCenter license management is divided into the following functions:

  • License Handling.

  • License Monitoring.

These topics are discussed separately in the sections that follow.

13.1. License Handling

The license handling function in InCenter deals with the import and storage of licenses, as well as the deployment of licenses to nodes under InCenter control.

In all cases, the commands used are the same for both site licenses and node licenses. The main difference is that a single site license can be deployed to more than one node. A node license can only be used with one specific node.

License Import, Node Association and License Deployment

The process of importing a license, associating it with a node and deployment to that node is done using the following steps:

  1. Upload the license to InCenter

    A license file must first be uploaded to the InCenter server using SCP or SFTP. A typical scp utility command to do this on an external management computer might be:

    > scp my_license.lic admin@196.51.100.10:
  2. Create a License object for the license file

    In the InCenter CLI, create a License object which points to the uploaded file:

    admin@InCenter:/> add License Name=myLicense1
    			LicenseData=my_license.lic

    A summary of all License objects can be displayed with the license -show command:

    admin@InCenter:/> license -show
       Name        Type    Comments
    -  ----------  ------  --------
       myLicense1  Node
  3. Associate a license with a node

    The License object must now be associated with a node. Assuming that the node my-node1 already exists, the command to create an association would be:

    admin@InCenter:/> set StandaloneNode my-node1
    			Licensed=Yes
    			License=myLicense1

    It should be noted that this does not actually deploy the license but only creates an association with the node.

  4. Activate changes

    At this point, an activate command followed by commit should be entered to activate the preceding changes.

  5. Deploy a license to associated nodes

    Finally, to deploy a license to all associated nodes, the license command is used:

    admin@InCenter:/> license -deploy myLicense1

If the license file has been uploaded and its License object created, a node could be added and have its license assigned at the same time in the add command.

Switching Between Site and Node Licenses

It is possible to replace one license on a node by simply deploying a new license so the old one is overwritten. The new license that is deployed does not have to be the same type as the old license so that it is possible to easily switch between a node license and a site license.

13.2. License Monitoring

The license monitoring function in InCenter deals with the display of deployed licenses and the monitoring of the node system resources that are controlled by the parameters in licenses.

Note that the site licensing related features described in this section apply only to NetShield nodes.

The importing of license files and associating InCenter License objects with a file is described previously in Section 13.1, License Handling.

Displaying License Objects

A summary of all License objects can be displayed with the license -show command and the type of each license is shown in the Type column:

admin@InCenter:/> license -show
   Name        Type    Comments
-  ----------  ------  --------
   myLicense1  Node
   myLicense2  Site

Using the license command with no parameters will display further details with the licenses also broken down by type:

admin@InCenter:/> license
Site Licenses:
Name        Registration Key     Limits
----------  -------------------  ---------------------------------
myLicense2  1856-1121-1331-1234  IPsecFwb: 10Mbps TotalFwb: 20Mbps 
Node Licenses:
Name        Registration Key     MAC Address
----------  -------------------  -----------------
myLicense1  2191-7212-1154-3298  ca-fe-ba-00-22-21

Note that site licenses are shown with a summary of the aggregate limits. Node licenses are tied to the Ethernet interface MAC address shown.

Displaying Individual License Objects

Once a License object has been created, the contents of its associated license file can be viewed with the license <license-name> command. The output will be of a different form depending on if a site license or a node license is being displayed.

The following is an example of a site license:

admin@InCenter:/> license myLicense1
Property                            Value
----------------------------------  ------------------------
                     LICENSE_TYPE:  FIREWALL
                    REGISTERED_TO:  Name-of-Customer
                 REGISTRATION_KEY:  0000-1111-1111-2222
                REGISTRATION_DATE:  2018-12-09
                    LAST_MODIFIED:  2018-12-09
                      ISSUED_DATE:  2018-12-09
                           OEM_ID:  4
             UPGRADES_VALID_UNTIL:  2019-12-09
          TECHNICAL_SUPPORT_UNTIL:  2019-12-09
                     SITE_LICENSE:  2
                    DISPLAY_BRAND:  Clavister 
                    DISPLAY_MODEL:  Clavister Next Generation Firewall 
                               OS:  1
                         PROP_BGP:  1
                        PROP_CONN:  30000000
                      PROP_DETNAT:  1
                    PROP_DIAMINSP:  1
                     PROP_GTPINSP:  1
                  PROP_IKETUNNELS:  2000
                     PROP_IPSECTP:  10
                        PROP_OSPF:  1
                    PROP_SCTPINSP:  1
                     PROP_SIPINSP:  1
                  PROP_THROUGHPUT:  20
                        PROP_VLAN:  4096
Nodes using license: my-node1
Monitored limits (2019-02-01T01:24:40+0100):
IPsecFWb 2Mbps out of 5Mbps
TotalFWb 5Mbps out of 7Mbps
Total license violations detected: 115

The contents of a site license will now be explained using the example above. Node licenses will not be discussed further in this section since they will follow the normal rules for a license.

A Site License Has Aggregate Throughput Limits

A site license has throughput limits associated with it which are shown at the bottom of the license output above along with how much of the limit is being used. The limits consist of the following:

  • IPsecFWb - Bit rate inside IPsec tunnels.

  • TotalFWb - Bit rate for both IPsec and other traffic.

However, it should be noted that these limits apply to the aggregate traffic throughput of all the nodes that have that site license assigned to them in InCenter. They would only apply to a single node if the site license was associated with just that node and no others.

Site License Aggregated Traffic Values are not Real-time

It should be noted that the aggregated throughput values calculated by InCenter are not real-time. They are calculated by summing the average throughput for each node over a 5 minute window. The time of the computation is given at the end of the license listing (such as the one above) in the line beginning Monitored limits. Both the monitored node throughput and the aggregated values are stored in InCenter for a period of time.

Examining site License Limits and Limit Violations

The -violations options of the license command can provide a list of license limits along with any limit violations. Adding the -num option limits the number of violations displayed. The example below displays the last 5 violations that occurred for all licenses along with the limits for those licenses:

admin@InCenter:/> license -violations -num=5
Registration Key     Limits
-------------------  ------------------------------------------
0000-1111-1111-2222  IPsecFWb: 0/10Mbps, TotalFWb: 24.73/20Mbps
0000-1111-1111-2222  IPsecFWb: 0/10Mbps, TotalFWb: 24.81/20Mbps
0000-1111-1111-2222  IPsecFWb: 0/10Mbps, TotalFWb: 24.81/20Mbps
0000-1111-1111-2222  IPsecFWb: 0/10Mbps, TotalFWb: 24.8/20Mbps
0000-1111-1111-2222  IPsecFWb: 0/10Mbps, TotalFWb: 24.67/20Mbps
Showing violations 1-5 out of 115.

Note that the output above has had the right-hand side truncated to fit on the page. The exact time of the violation is usually shown in an additional column on the right.

If the -num option is omitted then just the last 20 violations that occurred are displayed.

13.3. License Replacement

When a new license becomes available, the following steps should be used to replace an existing license:

  1. Download the new license file to the management computer.

  2. Upload the license to the InCenter server.

  3. Set the LicenseData property of the License object to be changed to the uploaded license file. For example:

    admin@InCenter:/> set License my-license
    			LicenseData=new_lic.lic
  4. Perform an activate and commit sequence.

  5. Enter the license -deploy command to deploy the license to the relevant nodes. For example:

    admin@InCenter:/> license -deploy my-license

Chapter 14: OpenStack Deployment

InCenter can be deployed in an OpenStack environment that uses KVM on Intel x86_64 as nova compute nodes. This section covers InCenter deployment and describes the steps required with the Horizon implementation of the OpenStack dashboard.

It is assumed that OpenStack has already been installed in a Linux environment and that the Horizon dashboard is also available.

OpenStack Prerequisites

To install InCenter with Horizon, the following is required:

  • A Clavister InCenter image should be imported into OpenStack.

  • An OpenStack flavor should exist that provides the same resources that are required for a standard InCenter installation. These resources are described in Chapter 2, Installation .

Setup Steps

The following steps are required for setup:

A. Create a Security Group.

B. Deploy an instance of InCenter.

These steps will now be described in detail.

A. Create a Security Group

A Security Group needs to be created. This can be done through Horizon but here it is done with the OpenStack Neutron utility using the following steps:

  1. Define the security group:

    root@controller:~# neutron security-group-create
    		-description 'my security group'
    		netguard -security-group
  2. Add the rule:

    root@controller:~# neutron security-group-rule-create
    		-direction ingress
    		-remote_ip_prefix 0.0.0.0/0
    		netguard-security-group
  3. Verify that the group exists:

    root@controller:~# neutron security-group-list

B. Deploy an instance of InCenter

Before launching a new InCenter instance, a disk volume should be created from the imported image so it has permanency. This is done in Horizon with the following steps:

  1. Select Volumes under the Compute tab.

  2. Select Create Volume.

  3. Select the imported InCenter image as the Volume Source and press Create Volume.

  4. Now, the instance of InCenter can be started by pressing Launch Instance under the Instances tab.

  5. Select a suitable Name and Flavor and choose Boot from volume as the Boot Source.

  6. Under the Access & Security tab, select the previously created security group called netguard-security-group.

  7. Select which networks to use under Networking.

    For access to Clavister Next Generation Firewall instances in Openstack, the InCenter instance should be assigned the same security group as that used for the management network for the firewall instances and the interface IP address should reside on the same network.

  8. Press Launch.

[Note] Note: SSH keys are copied from the host

When InCenter starts up in an OpenStack environment, any SSH keys added for the administrator user on the host machine will be added automatically to the admin user in InCenter as well. This means that logging in using SSH to both the host machine and InCenter can be done with the same keys immediately after startup.

Chapter 15: Node Groups

A node can be assigned membership in a NodeGroup. The group concept provides the ability to apply administrative actions to a group instead of an individual node, where the action will apply to all the nodes within the group

No predefined groups exist in InCenter so they must be defined by the administrator and then assigned to nodes.

The Rules of Using Groups

The following rules should be noted when using groups:

  • A single node can only be a member of a single group.

  • A group can contain other groups.

  • Individual HA members cannot be group members, instead it is the HAPair object that is added and as a consequence the HA members are indirectly included in the group.

  • If a group is deleted, its members become children of any parent group. If there is no parent to the original group then the children will have no group membership after deletion.

15.1. Managing Groups with the CLI

The way to create a new group called my-group is with the following command:

admin@InCenter:/> add NodeGroup
			Name=my-group1
			NodeType=NetWall

Note that the NodeType property must be specified. The NodeType would be set to NetShield for NetShield nodes.

Adding a Node to a Group

The node called my-node1 can then be added to this group:

admin@InCenter:/> set StandaloneNode my-node1
			Group=my-group1

Group Membership of Other Groups

A group can itself be a member of a group. Suppose there is a group called my-top-group. The group my-group1 could be added to it:

admin@InCenter:/> set NodeGroup my-group1 Group=my-top-group

Commands That Can Use Groups

Instead of specifying a StandaloneNode or HAPair object in a command, a NodeGroup can be specified instead. The command will then be applied to all members of the group, including subgroups.

The <all> Group

The special keyword <all> can be used if a command is to be applied to all the nodes under <all> control. This can be regarded as a predefined group that contains all nodes.

15.2. Managing Groups with the WebUI

To define a new Group object, go to Manager > Groups and then press the Add Group button.

Groups Display

Figure 15.1. Groups Display

Press the Add button to bring up the new group dialog. The upper part of this is shown below. The type of group must be chosen as either all NetShield or all NetWall. The two types cannot be mixed in a group. It is possible to specify an existing group which will have this new group is a member.

Add Group Dialog

Figure 15.2. Add Group Dialog

In the lower part of the dialog, the individual nodes can be added to the group. Other groups or HA pairs can also be added. With the same dialog, it is also possible to add members to the group from existing groups.

Add Group Members

Figure 15.3. Add Group Members

Chapter 16: Configuration Templates

[Note] Note: NetWall nodes are not supported

This feature is only supported by NetShield nodes. NetWall nodes are not supported.

A ConfigurationTemplate is an InCenter object that contains a set of configuration objects. The template can be applied to a particular node so the configuration additions and changes introduced in the template are transferred to the node.

The common use case for templates are as a way to simplify a common configuration task such as setting the same initial configuration for a newly added node.

When the template is applied to a node, there is no lasting relationship created between the configuration changes made and the template. The template can be reused with other nodes and also edited.

Creating a Template

Below is an example of template creation:

admin@InCenter:/> add ConfigurationTemplate
			Name=my-tp1
			Version=3.30.00.25

The show command can list all currently defined templates:

admin@InCenter:/> show ConfigurationTemplate
   Name        Version     Comments
-  ----------  ----------  --------
   my-tp1      3.30.00.25

Specifying the Template Version

The Version property must be specified when a template is created. The following should be noted about this property:

  • The version property must be set to any firewall software version that is supported by InCenter. Tab completion can be used to list all valid versions to help enter the number. The InCenter command show StandaloneNode will display a node's version number.

  • The template can be applied to a node running the same or a higher version.

  • The template cannot be applied to a node running a lower version number.

  • The template version property can be changed to a higher version number, in which case the template will be automatically upgraded to the new version when the change is activated.

  • The template version property cannot be changed to a lower version number.

Adding Template Objects

To add new configuration objects to a ConfigurationTemplate, the CLI context must be changed to be the template. In general, the configuration commands that apply for a node context are also valid for a template context. For example: add, set, delete, cc and show.

Add an IP address object to the template called my-ip with the IP 192.168.0.10:

admin@InCenter:/> cc ConfigurationTemplate my-tp1 
admin@InCenter:/my-tp1> add IPAddress
			Name=my-ip1
			Address=192.168.0.10

Add an empty interface object to point out the interface on the target:

admin@InCenter:/my-tp1> add EthernetInterface Name=if1

Add an IPRule that makes use of these objects:

admin@InCenter:/my-tp1> cc IPRuleSet main 
admin@InCenter:/my-tp1/IPRuleSet/main> add IPRule
			Action=Deny
			destinationInterface=if1
			DestinationNetwork=my-ip1
			SourceInterface=any
			SourceNetwork=all-nets
			Service=all_services
			Name=my-rule1

A change to the TCP settings could then be made:

admin@InCenter:/my-tp1/IPRuleSet/main> cc .. 
admin@InCenter:/my-tp1> set TCPSettings
			TCPAllowReopen=HigherSeq

Now, the activate and commit commands should be entered to save the template.

Applying Templates

A template apply operation will only transfer configuration changes that have been made to the template. Applying a template without changes will not introduce any changes on the target node(s).

The template my-tp1 can now be applied to the node my-node1 using the template command:

admin@InCenter:/> template -apply my-tp1 my-node1

Alternatively, the template could be applied to a list of nodes and groups:

admin@InCenter:/> template -apply my-tp1 my-node2,my-group1

Using the keyword <all> will apply the template to all nodes:

admin@InCenter:/> template -apply my-tp1 <all>

Applying the template my-tp1 will result in the following changes being made to the configuration of the target node(s):

  • Address object my-ip1 is added to the node's address book.

  • No change is made to the EthernetInterface if1.

  • IPRule object my-rule1 is added to the node's main rule set.

  • The TCPSettings property TCPAllowReopen has been set to HigherSeq.

After a template is applied, the changes introduced can be viewed using the command show –changes within the node's CLI context. The normal activate and commit sequence must also be entered to save the changes to any affected nodes.

Reapplying the same template will update changes made to settings and named objects such as IPAddress objects, but it will duplicate index objects such as IPRule objects. So if the exact same template (in the example) is reapplied, the only change to the target node(s) will be that a new IPRule is added.

Listing Template Contents with -show

A convenient way to view the effect that a template will have when it is applied is to list its contents using the template -show command. The following example shows this command being used with the template example that was created earlier in this section:

admin@InCenter:/> template -show my-tp1
Identifier                        Value
--------------------------------  ------------
|- ConfigurationTemplate my-tp1
   |- EthernetInterface if1
   |  |- Name                     if1
   |- IPAddress my-ip1
   |  |- Name                     my-ip1
   |  |- Address                  192.168.0.10
   |- TCPSettings
   |  |- TCPAllowReopen           HigherSeq
   |- IPRuleSet main
      |- IPRule 1(my-rule1)
      |- Action                Deny
      |- Name                  my-rule1
      |- SourceInterface       any
      |- DestinationInterface  if1
      |- DestinationNetwork    my-ip1
      |- Service               all_services
      |- SourceNetwork         all-nets

A Summary of Template Usage

The following points should be noted about template usage:

  • A template can only add or change node configuration objects. It cannot delete objects.

  • A template should be self-contained, meaning that an object must be added in a template before it is used later in that template. For example, if a line in a template refers to interface if1 then if1 must have been added by an earlier line in that template. This can be seen in the previous example.

  • If a template is run more than once on the same node then an add operation will not be performed twice where an object is only identified by name and not by index. However, list objects that have a list index for reference (for example, IPRule objects) will be added again each time the template is run.

Template Version Mismatch

The following is a list of the possible version mismatch scenarios and a description of how to deal with them:

  • If the software version on the firewall is not supported by InCenter.

    The solution is to change the firewall version.

  • The firewall has a higher software version than the node in InCenter.

    The template can still be applied to the node. When the changes are activated, the node will be upgraded to the software version automatically but the template version will remain the same.

    To change the template version, this must be done manually. A reason to change the template version is to be able to include features in the template that are only available in a higher version.

  • The firewall has a lower software version than the node in InCenter.

    The solution to this is to upgrade the firewall software version.

Chapter 17: InCenter Administration

This section describes the administration of the InCenter system itself. Instead of using the InCenter CLI, the shell of the host operating system will be used to perform most of the tasks in this section. However, as explained in the next section, file transfers can be performed under the control of InCenter or outside of it.

17.1. Host OS CLI Access

To perform administrative tasks via the host operating system, the system's CLI shell can be accessed using any of the following methods:

  • Via the hypervisor console for the virtual machine running the server.

  • By connecting a console to the console port of the virtual machine. A single console port is configured in the InCenter virtual machine image distribution.

  • Over a network connection using an SSH client. Connection is made to the same IP address used for InCenter access but using the port number 2222.

    By default, this SSH access is disabled. It can be enabled by entering the following Linux CLI commands, using either the local console or hypervisor console described above:

    $ sudo systemctl enable ssh 
    $ sudo systemctl start ssh

The Default Login Credentials

The default username and password credentials for host operating access are the following:

  • Username: administrator

  • Password: administrator

After logging in, the user will be in the default folder /home/administrator. All InCenter system files are contained in a subfolder to this called mgmtsystem (in other words, the path /home/administrator/mgmtsystem).

17.2. Changing the Host SSH Keys

The default host SSH keys used in a InCenter Linux system are always unique. However, there may be a need to set the keys to particular values. For example, this need can arise in a InCenter redundancy cluster where they are not synced automatically and where the keys on all nodes should be manually set to the same values to avoid warning messages from the management client.

Changing the keys is done by overwriting the SSH key files using SCP. These files are found in the Linux folder /home/administrator/mgmtsystem/ and they must be given the same names when they are replaced. These names are always host_rsa and host_rsa.pub.

For example, a typical SCP command to upload and overwrite the host_rsa file might be the following:

> scp -P 2222 host_rsa
	administrator@198.0.2.10:/home/administrator/mgmtsystem 

Once the host SSH files are replaced, the InCenter system must be restarted.

17.3. Restarting InCenter

The restart of InCenter is done using the reboot command. This command can have any of the following forms:

  • Without any parameters

    When used without any parameters, the reboot command will only reboot the InCenter system.

    admin@InCenter:/> reboot
  • With the -host parameter

    When used with the -host parameter, the reboot command will reboot both the underlying host system as well as the InCenter system.

    admin@InCenter:/> reboot -host
  • With the -debug parameter added

    For troubleshooting purposes, the -debug=<level> option can be added to either of the two preceding forms. The <level> can be an integer between 1 and 10. For example:

    admin@InCenter:/> reboot -host -debug=5

    Following the restart, InCenter will run in debug mode and will output additional diagnostic messages as it functions. The detail of these messages is controlled by the level number. Debug mode should only be used in conjunction with assistance provided by a qualified support technician.

17.4. System Log Files

InCenter system log files can be accessed directly by opening the folder called storage.

This folder is accessible for file transfer by using SCP or SFTP via port 22. Doing this is described further in Section 3.5, File Transfer.

Within the storage folder are the following subfolders:

  • errors

    The folder errors will contain any Exception Reports. An exception report is a text file generated by InCenter when a system error occurs. All these files have the filetype .err and can be read with a normal text editor. Normally, these files are only intended to be examined by qualified support personnel.

    When an exception report is generated, the InCenter system will restart automatically after the exception report is written. Any logged in users will have to log in again. However, any pending configuration changes that were not deployed before the restart will still be ready for deployment after logging back into InCenter.

    When an exception report file is created, a message is also written in the system.log file (described below) that has the text System exception occurred.

    There is a maximum number of 30 exception reports that can be stored at any one time. When this limit is reached, the oldest report will be deleted by the InCenter system when a new report created.

  • logs

    The folder logs contains log event messages generated by InCenter during normal operation. These files are described next.

Log Files

The logs folder contains the following files, which are plain text files that could be viewed using any text editor:

  • system.log

    This file contains a line for each significant system event. These events are mostly related to system startup/shutdown, activate, commit and deploy.

  • audit.log

    This file contains a line for each interaction that the administrator has with the InCenter system. This can include the following:

    1. SSH access.

    2. File transfers.

    3. REST API access.

  • debug.log

    This log file is used for system debugging and is not relevant to normal operation. In normal operation, no debug logs will be generated. In order to enable debug levels, the system must be restarted using the reboot command and including the -debug option. This is described further in Section 17.3, Restarting InCenter

[Note] Note

The time and timezone used by InCenter are taken from the host operating system. This can be changed with the host operating system CLI command:

$ sudo dpkg-reconfigure tzdata

Log Files are Automatically Compressed at a Given Size

When any of the log files described above reach a certain size (by default 10 megabytes), they are automatically compressed using the GZIP algorithm and the file is renamed. The new name of the compressed file takes the form <type>.nn.gz, where <type> is one of system, audit and debug and nn is a unique sequential number assigned to each compressed file.

For example, the last compressed audit.log will get the name audit.log.0.gz. The lowest numbered file is always the latest, with all existing compressed files having their number incremented when a new one is generated. After compression takes place, subsequent log messages are placed in a new file with the original uncompressed name (in the case of audit files, audit.log).

Any of the compressed files can be uncompressed using a suitable GZIP utility.

There is a Maximum Number of Log Files

To ensure that log files do not take up too much space, there is a maximum number of compressed log files which can exist. By default, this is set at 10. In other words, a maximum of 10 rotated GZIP files can exist at once with one of the digits from 0 to 9 in their filename. Those 10 will be in addition to the current uncompressed .log file.

When there are 10 compressed log files and the current file reaches the size limit, the oldest compressed file is deleted to make way for the new compressed file.

Changing Maximum Size and Maximum Number

The maximum uncompressed log file size can be set to an administrator defined value through the InCenter CLI. For example, the following command will set the maximum size to 5 megabytes:

admin@InCenter:/> cc Settings 
admin@InCenter:/Settings> set LogFiles MaxFileSize=5

Similarly, the maximum number of compressed log files stored could be set to 20:

admin@InCenter:/Settings> set LogFiles NumberOfFiles=20

The mgmtsystem Service Logging

In addition to the log files described above, there is additional logging information saved in the Linux systemd journaling system. Its purpose is to record any critical system events when InCenter is unable to start at all. Normally the administrator will not use the information in this location. To see the logging information use the following command:

> journalctl -u mgmtsystem

This command will show a paged listing of the latest log record. To exit the view, press the Q key or enter Ctrl-C.

17.5. Downloading Log Files

Log files can be downloaded from InCenter using either an SCP or SFTP client. By using the default SSH port number of 22, transfers will be mediated by InCenter which means that the only directory accessible will be /mgmtsystem/storage. This section will show examples using SCP only.

Assuming that the InCenter IP address is 192.168.23.206, the following will download all log files:

> scp -r admin@192.168.23.206:./ .

To download just the system.log file:

> scp admin@192.168.23.206:./logs/system.log .

To download all the error files:

> scp -r admin@192.168.23.206:./errors .

Downloading all the error files like this at once may be more convenient because the error file names can be long.

17.6. Expanding Log Storage

By default, the InCenter virtual machine will provide 40 Gbytes of storage space for storing both system and OpenSearch database node logs. This may need to be expanded and this can be done by first adding an additional storage volume to the virtual machine and then moving the InCenter logging directory to this new volume.

The detailed steps for doing this are the following:

  1. Shutdown the InCenter virtual machine.

  2. Add the required additional storage volume to the virtual machine.

  3. Start the virtual machine again.

  4. Connect to the Linux CLI using SSH via port 2222 on the InCenter management IP address.

  5. Stop the InCenter logging service:

    $ sudo service opensearch stop
  6. Create new Linux physical volume. Note that the new Linux storage volume should be named something new like "sdb":

    $ sudo pvcreate /dev/sdb
  7. Create new volume group:

    $ sudo vgcreate vg-log-data /dev/sdb
  8. Create new logical volume:

    $ sudo lvcreate -l 100%FREE -n lv01 vg-log-data
  9. Create the file system for the new logical volume:

    $ sudo mkfs -t ext4 /dev/mapper/vg--log--data-lv01
  10. Create a temporary directory to copy the old log data into:

    $ sudo mkdir -p /mnt/log-data/opensearch/
  11. Mount the new logical volume to the temporary directory created:

    $ sudo mount /dev/vg-log-data/lv01 /mnt/log-data/opensearch/
  12. Copy the old log data to the temporary directory (in the new logical volume):

    $ sudo cp -a /var/lib/opensearch/. /mnt/log-data/opensearch/
  13. Once the copy is done, unmount the temporary directory:

    $ sudo umount /dev/vg-log-data/lv01 /mnt/log-data/opensearch
  14. Mount the InCenter log directory to the new logical volume:

    $ sudo mount /dev/vg-log-data/lv01 /var/lib/opensearch
  15. In order to maintain the mount point after restarts, add an fstab entry:

    $ echo '/dev/mapper/vg--log--data-lv01 /var/lib/opensearch
    	ext4 defaults,comment=cloudconfig _netdev
    	nofail  0 0' | sudo tee -a /etc/fstab
  16. Start the InCenter logging service again:

    $ sudo service opensearch start
  17. Reconnect with SSH to the InCenter CLI on port 22 and increase the maximum log database storage size:

    admin@InCenter:/> cc Settings  
    admin@InCenter:/Settings> set LogServer MaximumLogDatabaseSize=100

    The size specified will depend on the total amount of storage space allocated in to the virtual machine. It is recommended that the MaximumLogDatabaseSize is set to no more than 75% of the available space.

17.7. Displaying Uptime

The uptimes can be displayed with one of the following methods:

  • For the InCenter system and use the uptime command.
  • To display the uptime of a standalone node or HA cluster, use the status option of the node command.

Displaying InCenter Uptime

The CLI uptime command with no options displays InCenter uptime since the last restart:

admin@InCenter:/> uptime
1 minutes, 12 seconds

The -verbose option provides uptime information about both InCenter and the underlying host system:

admin@InCenter:/> uptime
System: 1 minutes, 16 seconds
Host:   1 minutes, 40 seconds

Displaying Node Uptime

The uptime for an individual node can be displayed using the status option with the node command:

admin@InCenter:/> node my-node1 status
This node is a standalone system
Uptime: 0 days 00:01:08
HostKey: SHA256:hPxm3mGe5K6Uy67g4+bg2kxDxbA+Jk9z

The same command form can be used to display the uptime for an HAPair object:

admin@InCenter:/> node my-hapair status
my-node1 (Primary node)
This node is an HA MASTER
Active: yes
HA cluster peer is alive
Uptime: 0 days 00:06:42
HostKey: SHA256:6Ce/v8ICQERhqFqWKupk1o+JzbDrbXW6hWD78
my-node2 (Secondary node)
This node is an HA SLAVE
Active: no
HA cluster peer is alive
Uptime: 0 days 00:01:57
HostKey: SHA256:IZ8xxmddfKFcZwOo2xJ9QPFXfM7kjwyZ4zYTL

17.8. System Backup and Restore

The InCenter database should be routinely backed up in case of a critical system failure.

A backup and restore should be performed using the appropriate script. This involves logging in to the host operating system and running one of the scripts that should already be present in the /home/administrator/scripts folder.

[Note] Note: The InCenter system will restart

During both the backup and restore procedures, the InCenter system is stopped and restarted. This means that any ongoing processes will be interrupted.

  • Performing a Backup

    Run the backup.sh script in the scripts folder of the administrator user.

  • Performing a Restore

    Run the restore.sh script in the scripts folder of the administrator user.

Both the backup and restore scripts are described next in Section 17.9, Scripts.

17.9. Scripts

After logging in to the host operating system, the administrator has a pre-installed folder called scripts with the file path /home/administrator/scripts. This folder contains a number of scripts written in the bash scripting language and have the filetype .sh. They are provided as tools for certain administrative operations and are described in this section.

The scripts found in the scripts folder are the following:

  • backup.sh

    This script will automatically create a backup of the InCenter system in the scripts folder. A suitable SCP or SFTP client can then be used to download the newly created backup file to a management computer from the server's disk if required.

    The backup.sh script will automatically stop and restart the InCenter services. A filename can be supplied as an argument in the command line. The file created will be in the GZIP format.

    An example of running this script is the following:

    $ ./backup.sh my-backup.tar.gz
  • restore.sh

    This script will restore a backup that was created using the backup.sh script described above. The backup file should be uploaded into the scripts folder using a suitable SCP or SFTP client if the file is not already present.

    The restore.sh script will automatically stop and restart the InCenter services. A filename can be supplied as an argument in the command line.

    An example of running this script is the following:

    $ ./restore.sh my-backup.tar.gz
  • staticIP.sh

    This script will set the IPv4 or IPv6 address which is used to communicate both with the InCenter system as well as the host operating system. It takes two parameters: the IP address and the netmask for the network. For the change to take effect, the virtual machine should be restarted.

    An IP address and netmask must be supplied as arguments in the command line when running this script. For example, to set the IPv4 address:

    $ ./staticIP.sh --ip4 203.0.113.10 --netmask4 255.255.255.0

    To set the IPv6 address:

    $ ./staticIP.sh --ip6 2001:db8::c0ca:1eaf --netmask6 64

    The full syntax for this script with all its options is the following:

    ./staticIP.sh [--ip4 <IP address> --netmask4 <netmask>
    		[--gateway4 <gateway>]]
    	[--ip6 <IP address> --netmask6 <netmask>
    		[--gateway6 <gateway>]]
  • backupLogs.sh and restoreLogs.sh

    These scripts are used for backing up and restoring log files. They are further explained in Section 18.4, Log Migration to a New Instance.

  • techsupport.sh

    This script performs a similar function to the InCenter CLI techsupport command, which is described in Section 17.11, The techsupport Command. However, the Linux script version of the command provides the ability to create comparable diagnostic output even if the InCenter CLI is unavailable. As described below, the script also provides the additional option to include the current InCenter configuration in the output generated.

    The following should be noted about using the techsupport.sh script:

    1. The output file from the script is created in the user's current directory when the script is run.

    2. The script must take one of the two following parameters:

      • -excludeCfg - Do not include the InCenter configuration.

        $ ./techsupport.sh -excludeCfg
      • -includeCfg - Include the InCenter configuration.

        $ ./techsupport.sh -includeCfg

        Note that this second option to include the configuration does not exist with the techsupport command in the InCenter CLI.

      If neither of the above parameters is specified in the command line after techsupport.sh then the script will terminate and no output will be created.

17.10. Upgrading the System

If the InCenter software version needs to be upgraded, the following steps should be used:

  1. Create a backup of the current InCenter configuration with the backup.sh script. Using this script is described in Section 17.9, Scripts.

  2. Download the backup file to the disk of the local management computer using SCP. The following is an example of the SCP command format (entered as a single line):

    > scp -P 2222 administrator@<sys_ip>:/home/administrator/<file>
      /<local_folder>/
  3. Shut down the old virtual InCenter instance so there can be no IP address conflicts with a new instance. Deleting the old instance is not recommended at this point, in case it is needed again later.

  4. Create a new virtual machine instance running the new InCenter version.

  5. Upload the backup file from the local management computer disk to the new instance using SCP. The following is an example of the SCP command format:

    > scp -P 2222 /<local_folder>/<filename> administrator@<sys_ip>:
  6. Restore the contents of the uploaded file into the new instance with the restore.sh script. Using this script is described in Section 17.9, Scripts.

  7. Once it is clear that the new InCenter instance is functioning as expected, the old instance can be deleted.

Transferring Log Data

If there is a need to transfer log data from an old virtual machine instance to a new one, doing this is described in Section 18.4, Log Migration to a New Instance.

17.11. The techsupport Command

The command techsupport creates a downloadable file which can be sent to product support personnel for troubleshooting purposes. The file is a compressed archive that contains information about the state of the InCenter system and/or nodes/HA clusters at the time the command is run. It is generally recommended to run the command immediately after the event being investigated.

The techsupport command options are described next.

For the InCenter System Only

The basic form of the techsupport command with no options creates a file only for the InCenter system and excludes any nodes controlled by InCenter:

admin@InCenter:/> techsupport
Collecting and storing Technical Support Information...
Technical Support file created:
InCenter-techsupport-2022-04-21T23_12_42.tar.gz

For the InCenter System and Selected Nodes

The -nodes option will add information for the specified nodes to the information about the InCenter system:

admin@InCenter:/> techsupport -nodes=my-node1,my-node2

All nodes can specified by using <all>:

admin@InCenter:/> techsupport -nodes=<all>

Note that the information included in the techsupport file for a node is the same as if the techsupport command had been run directly on the node.

Excluding InCenter Information

The -nosystem option excludes information about InCenter from the file generated. To generate a techsupport file for just the node called my-node1, the command would be:

admin@InCenter:/> techsupport -nosystem -nodes=my-node1

For High Availability Pairs

Instead of standalone nodes, the techsupport command can be applied to one or more HAPair objects. For example:

admin@InCenter:/> techsupport -nosystem -nodes=my-hapair

When applied to an HAPair, the techsupport command is run twice at the node level, once for each node in the pair, and both sets of information are then included in the file created.

Downloading the Techsupport File

The file created by the techsupport command can be downloaded from the InCenter server using SCP. It is saved into a directory called techsupport under the /mgmtsystem/storage directory. The file can therefore be downloaded by connecting on port 22 of the management IP address. A typical SCP download command could take the following form:

> scp admin@<mgmt-IP>:./techsupport/<techsupport-filename> .

Unavailable Nodes

When retrieving nodes may not be able to respond with the required information. A node might refuse the connection or may not respond to the request at all, possibly because it is not active.

Even if there are errors and no useful information is available, the techsupport command will still write a new file. Any error messages will be included in the file.

File Housekeeping

The files created by the techsupport command will not be deleted automatically by the system. Housekeeping of the files must be done manually by the administrator and it is recommended to do this routinely. Each file will take up disk space and if disk space runs low, the techsupport command will be unable to create an additional file.

All old techsupport files can be deleted to free up space with the -cleanup option:

admin@InCenter:/> techsupport -cleanup

Running techsupport Without the InCenter CLI

If, for some reason, the InCenter CLI is unavailable, the techsupport command can be run in the host Linux CLI by running the preinstalled script called techsupport.sh. This script is described further in Section 17.9, Scripts.

Note that the script has one feature that the command does not have which is the ability to include the current InCenter configuration in the output.

Chapter 18: Log Servers

As well as log messages sent by external nodes, the InCenter system itself generates log messages as various events occur. All these log messages can be sent to the following types of Syslog receivers:

  • The Internal InCenter Log Server

    By default, each InCenter installation has its own internal log server to which InCenter log events are automatically sent. This type of server is described in Section 18.1, The Internal Log Server.

  • External Syslog Servers

    The log messages generated only by InCenter itself can be sent to external Syslog receivers. This is described further in Section 18.3, Using Syslog Servers.

Note that when the InCenter software is upgraded, internal log files will not be preserved. For this reason, configuring an external log server can provide the advantage of preserving old log information.

Configuring External Nodes

For an external node to send log messages to InCenter, a LogReceiverSyslog object must be added in the node configuration in the normal way. For example:

Device:/> add LogReceiverSyslog Name=system_log_server
		IPAddress=203.0.113.5

Note that the IP address specified for the server is the same IP address used for InCenter management access.

18.1. The Internal Log Server

The InCenter internal log server is available as a standard component of the InCenter installation. The internal log server will store events from both InCenter itself as well as any external nodes that are configured to send Syslog messages to InCenter.

All log messages generated by InCenter itself will be automatically stored by the configured log server. The source IP address for these events will always be the loopback address 127.0.0.1.

Getting the Current Log Server Status

The log -status command will show the current server status:

admin@InCenter:/> log -status

Querying the Server Database

The log -query command is used to query the server database. The most basic form of the command has no additional parameters:

admin@InCenter:/> log -query
Time                 Source     Severity  Category  Message
-------------------  ---------  --------  --------  --------
2017-10-21 12:14:08  127.0.0.1  info      CONFIG    commit_started
user=admin ip=127.0.0.1 port=8000
Showing 1 out of 1.

The above output shows a single log message (wrapped to fit on the page) to illustrate the message format. The Source column shows the IP address of the node or the IP address 127.0.0.1 if the message is from InCenter itself.

In practice, too many messages may be displayed and the number can be limited with the -num option:

admin@InCenter:/> log -query -num=10

It is also possible to display log messages without any column formatting using the -no-table option. The following is an example with some typical output:

admin@InCenter:/> log -query -num=1 -no-table
2017-06-08 16:41:02.321 192.168.111.11 ARP,VALIDATE:
prio=warning id=00240 event=disallowed_by_access_rule
recviface=if2 srchw=10:00:00:02:10:00 srcip=127.0.0.3
destip=127.0.0.3 pkt_rec vif=if2
pkt_srchw=10:00:00:02:10:00 pkt_enetproto=ARP
rule=System_Block127Net action=drop logtrace=066089df

Filtering Queries

Filtering is possible to display only the log messages that match a set of criteria. For example, the source IP could be the criteria:

admin@InCenter:/> log -query -num=10 source=192.168.111.11

The criteria could be the log message severity:

admin@InCenter:/> log -query severity=warning

Alternatively the criteria could be the log message severity category. There can be more than one category associated with a log message but only one needs to match:

admin@InCenter:/> log -query category=ARP
Time                 Source          Severity  Category
-------------------  --------------  --------  ------------ 
2017-06-08 16:41:02  192.168.111.11  warning   ARP,VALIDATE
prio=warning id=00240 event=disallowed_by_access_rule
recviface=if2 srchw=10:00:00:02:10:00 srcip=127.0.0.3
destip=127.0.0.3 pkt_rec

Multiple criteria can be combined:

admin@InCenter:/> log -query severity=warning category=ARP

Pattern Filtering

A powerful method of filtering log messages is to use free text. This can be combined with the following wildcards:

  • * - Asterisk means any combination of characters.
  • ? - Question mark means any single character.

The following example will find all log messages that contain the word "warning" followed by any number of characters before the text "ip=10.11.12.?00", where the ? represents any character:

admin@InCenter:/> log -query
			-pattern="warning *ip=10.11.12.?00"

Log Server Settings

All log servers have the following adjustable settings:

  • LogServerPort - Listening port number of the log server.

  • LogDatabaseCleanupEnabled - This enables clean up of the log message database. The cleanup process is controlled using the other settings in this list. Default value: Yes.

  • MaximumLogDatabaseSize - Maximum size of the database in megabytes. If enabled, the oldest log messages are deleted if the total index size exceeds this amount. Default value: 10,000.

  • MaximumLogEventsAge - Delete log messages if they are older than this many days. Setting this to the minimum value of 1 will mean only the current day's logs are retained. Default value: 365.

  • MaximumAggregatedDatabaseSize - Aggregation indices will be deleted until the total disk used by them is lower than this limit, starting with the oldest index. Default value: 10,000.

  • MaximumAggregatedDataAge - Aggregation indices older than this age will be deleted. Default value: 365.

  • RateLimitBurst - Total number of logs allowed to be received over the time period specified by RateLimitInterval.

  • RateLimitInterval - The interval during which rate limiting counts logs and rejects logs once the number exceeds RateLimitBurst. The average number of logs that the log server accepts each second is the RateLimitBurst divided by the RateLimitInterval. For the default system this number is 1000/5 = 200 logs/second.

It should be noted that the automatic cleanup process runs once every hour.

Displaying the Current Server Properties

The command show LogServer in the Settings CLI context will display the current values of above settings for the currently configured log server. Below is example output for the internal server:

admin@InCenter:/> cc Settings 
admin@InCenter:/Settings> show LogServer
                      Property  Value
------------------------------  --------
                      Enabled:  Yes
                     Location:  Internal
    LogDatabaseCleanupEnabled:  Yes
              LogReceiverPort:  514
     MaximumAggregatedDataAge:  365
MaximumAggregatedDatabaseSize:  10000
       MaximumLogDatabaseSize:  10000
          MaximumLogEventsAge:  365
               RateLimitBurst:  1000
            RateLimitInterval:  5

18.2. Using External Log Servers

By default, InCenter uses a single internal log server for InCenter generated logs and logs received from external firewalls. However, it is possible to instead make use of a single external Open Distro log server. Any external server configured for this must be based on Open Distro Elasticsearch 0.7.0.1 or later.

The following command is an example of how such a server is configured and includes all the mandatory parameters:

admin@InCenter:/> Set LogServer Location=External
			LogServerIP=203.0.113.5
			AdministratorUsername=user1
			AdministratorPassword=pass1
			MaintainerUsername=user2
			MaintainerPassword=pass2
			ImpersonatorUsername=user3
			ImpersonatorPassword=pass3

The other optional command parameters are the same as the optional parameters used for the internal server and are described in Section 18.1, The Internal Log Server.

Using an external server does not change any of the other log management functions in InCenter. The location of the database is transparent to the user and such functions such as querying work in the same way as with the internal receiver.

Reverting to the Internal server

If it is required to revert back to using the internal InCenter server for event logging, the following command can be used:

admin@InCenter:/> Set LogServer Location=Internal

18.3. Using Syslog Servers

The log messages generated by InCenter itself can be sent to one or more external Syslog servers. InCenter Syslog messaging complies with RFC 5424. Note that the log messages sent to InCenter cannot be forwarded to an external Syslog server.

Configuring Syslog Servers

Sending messages to a Syslog server is set up with the CLI command: add SyslogReceiver.

Suppose that a Syslog server has the IPv4 address 192.0.2.10. The simplest CLI command to set up this server for logging InCenter events would be:

admin@InCenter:/> add SyslogReceiver Name=my_logger
			IP=192.0.2.10

Any number of SyslogReceiver objects can exist at the same time.

The optional SyslogReceiver object properties are the following:

  • Port - The port number for connection. The default is 514.

  • Severity - The severities to be sent. The possible values are: Informational, Notice, Warning, Error, Critical, Alert, Emergency and Debug.

    The default value is Informational, Notice, Warning, Error, Critical, Alert, Emergency.

  • Category - The categories of messages sent. The default is All but this could be any combination of: Config, DB, License, REST, SSH and System.

  • LogType - This can be Audit and/or System. The default is both.

  • Facility - This can be local0 through local7. The default is local0.

InCenter will scan all configured SyslogReceiver objects and send a log message to the receiver if the message matches the filtering criteria of the object.

18.4. Log Migration to a New Instance

When a new virtual instance of InCenter is created, perhaps because a new InCenter version has been installed, there may be a need to transfer the old log data to the new instance. This is achieved using the following steps:

  1. Find out the total disk space used by the old log data by running the following InCenter CLI command:

    admin@InCenter:/> log -status 

    Make sure that at least this amount of disk space is available for the new InCenter instance.

  2. Direct log messages to the new InCenter instance by changing the IP address property of the log receiver object on individual nodes.
  3. In the host operating system for the old InCenter instance, run the script backupLogs.sh which can be found in the scripts folder. This will save the old log data to a specific folder. For example:

    $ ./backupLogs.sh /mnt/cloudBackup/opensearch

    Note that an absolute path must be specified as the script parameter in the above command. If this is not done then it can lead to OpenSearch being unable to start. The same problem could occur if the backup script is terminated before it completes.

  4. Move this backup folder for the logs so it is available to the host operating system of the new InCenter instance.
  5. In the host operating system of the new instance, restore the saved log data in the folder by running the script restoreLogs.sh. For example:

    $ ./restoreLogs.sh /mnt/cloudBackup/opensearch

    Note that when either backupLogs.sh or restoreLogs.sh runs, InCenter will not be able to receive log messages for a period of time.

The new InCenter instance will now have all the old log data available to it.

Chapter 19: Redundancy Setup

The InCenter system can be configured to provide a redundant high availability (HA) cluster so that if one InCenter server running on one computer fails, an automatic failover will occur, allowing processing to continue on another computer with an identical InCenter configuration.

[Note] Note: This chapter covers clusters using InCenter 1.64 or later

This chapter describes setting up redundancy clusters where the HA nodes run InCenter version 1.64.00 or later. HA clusters created prior to 1.64 are not compatible but can be converted to this newer cluster type. Doing this is described at the end of this chapter.

HA cluster setup requires three separate instances of InCenter. Each instance acts as a high availability node (HA node) in an active-passive arrangement. The databases on these three HA nodes are configured to be synchronized. Located in front of the three HA nodes is a third-party load balancing proxy. The proxy polls the management IP of each HA node and sends traffic only to the active node.

If the active HA node fails, one of the other two HA nodes will automatically assume the active role. If two HA nodes were to fail, the remaining HA node will not accept traffic and the entire HA cluster will no longer function. This HA cluster arrangement is illustrated below.

A Redundant Cluster Configuration

Figure 19.1. A Redundant Cluster Configuration

Various proxy software products are available to perform the load balancing task and setting one up will not be described in this document. Only the base InCenter cluster setup will be covered later. The list of ports that must be forwarded by the balancer is also listed at the end of this section.

The rc Command

The main tool for setting up redundancy is the HA cluster configuration command rc in the host Linux system CLI. This is already included as part of the InCenter installation.

[Note] Note: Root privileges are required

When setting up an HA cluster using the rc command, root privileges are required.

The rc command can take the following options:

  • gen-certs - Generate the setup files needed for HA node synchronization.

  • initiate - Enable redundancy for this HA node. The extra parameter -f finalizes cluster setup.

  • disable - Disable redundancy for this HA node and remove it from the HA cluster.

  • status - Display the current status of this cluster.

The parameter -help can be used to display individual help text for a particular option. For example:

$ rc gen-certs -help

HA Cluster Setup Steps

The following steps should be followed to set up a high availability cluster:

  1. Have three InCenter instances ready to be joined together as HA nodes in a cluster. In this example, they will have the DNS resolvable host names hostname1, hostname2 and hostname3. All three should have the same InCenter version installed (1.64.00 or later) and they should be running on separate hardware platforms. The InCenter instances need not be newly installed but note that the current database image of the last HA node added to the cluster (in this example, hostname3) will overwrite the databases of the other two.

    The three instances will need to communicate with each other so the computers might be on the same network although it may be better to have at least one in a different physical location for maximum tolerance to local risks such as power supply failure.

  2. Log into the Linux CLI of the first InCenter instance (assume this is hostname1) and enter the following command:

    $ rc gen-certs -m hostname1 hostname2 hostname3

    This will generate three cluster setup files for the three instances with filenames of the form hostname1.tar.gz, hostname2.tar.gz and hostname3.tar.gz.

    As mentioned previously, hostname1, hostname2 and hostname3 are the DNS resolvable hostnames of the InCenter instances. For example, server1.example.com. The actual IP addresses cannot be used instead.

    It is important to note that a copy of the three files generated should be archived on disk storage outside of the cluster. This is because any of these files may be needed later when replacing a failed HA node with a new InCenter instance.

  3. Use SCP to upload two of the files to their respective instances. For example:

    $ scp -P 2222 hostname2.tar.gz administrator@hostname2:.
    $ scp -P 2222 hostname3.tar.gz administrator@hostname3:.
  4. On the first instance, make it become an HA node with the following command:

    $ rc initiate -c hostname1.tar.gz

    Repeat this on the second instance:

    $ rc initiate -c hostname2.tar.gz

    Repeat this again on the third instance, but this time adding the -f option to finalize the cluster setup and also to make this instance the HA node that is initially active:

    $ rc initiate -c hostname3.tar.gz -f

    Note that the database image of this final instance will automatically overwrite the databases in the other two instances.

  5. The HA cluster should now be operational with three HA nodes. The final step is to start the load balancer located in front of the three instances. The load balancer should be configured to send the following HTTP message to each instance to determine if it is the active HA node:

    	https://<ha-node-IP>:<rest-port>/rc/isPrimary

    Only the active HA node will send back a "200 OK" reply. This reply indicates that this is the HA node that should receive traffic from the load balancer. The <rest-port> value is always 8443.

Setting the Same Host SSH Keys On All HA Nodes

By default, all the different HA nodes in a cluster will use different host SSH keys (the SSH keys in the underlying Linux system used for management access). To avoid warning messages from SSH clients used for management, it can be convenient that all the HA nodes use the same host SSH keys. These keys are not synced in a cluster so to be all the same they need to be changed manually on each HA node by using SCP to upload new key files to the host Linux system but with the same file names.

Change the key files is described in more detail at the end of Section 3.1, SSH Access to the CLI. The recommended series of steps with a cluster is to first download the key files from one particular HA node using SCP and then upload these same files to the other HA nodes, also using SCP. The HA nodes that receive the uploads should then be restarted.

Dealing with HA Node Failure

Should one of the HA nodes fail then the remaining two HA nodes will continue to function as an HA cluster. If the failed HA node was active, the load balancer will automatically send traffic to the next functioning node. However, the failed HA node should be brought back online as soon as possible. This is because of the following risks that lead to cluster failure:

  • A second failure will leave only one HA node functional. However, a single HA node will always enter the passive state and will not accept traffic.

  • If there are two functional HA nodes remaining and the link between them is broken, both will behave like a single isolated HA node and both will enter the passive state, not accepting traffic.

When the failed HA node has been replaced with a functioning InCenter instance, the following steps are used to include the new instance into the cluster. This can be done without needing to stop traffic flowing to the other functioning nodes. In this example, it will be assumed that hostname2 has failed:

  1. Using SCP, upload the original setup file for the HA node to the new InCenter instance:

    $ scp -P 2222 hostname2.tar.gz administrator@hostname2:.

    This is the reason that a copy of the three setup files should have been archived on disk storage outside of the HA nodes, as described earlier.

  2. Log into the new instance and bring the HA node online.

    $ rc initiate -c hostname2.tar.gz

The database of the other HA nodes will now be automatically copied into the new HA node and the full three node HA cluster will have been restored.

If copies of the original setup files are not available then the procedure described previously to set up a new cluster with all three instances should be used instead. In this case, traffic processing will have to stop until the setup is complete and the last HA node initialized should not be the HA node that was replaced.

Removing Nodes from a Cluster

It is possible to remove an HA node from the cluster by logging into Linux on the instance and using the following command:

$ rc disable

Following this command, the HA node will now act as a standalone instance and will no longer respond to the polling sent by a load balancing proxy. It is completely removed from the cluster.

Although a cluster should optimally have three working nodes, removing one HA node might be needed in order to replace it. The replacement procedure after manual removal would be the same as that described previously for replacing a failed HA node.

If an HA node is disabled manually and it was the active HA node, the remaining two HA nodes will decide between themselves which node will now become the active one. Using the disable option on all three HA nodes will dismantle the cluster completely.

Creating and Restoring a System Backup

It should not matter which of the three InCenter instances a system backup is taken from. The database should be synchronized across all of the working HA nodes in a cluster. The steps to take a backup are the following:

  1. Disable one of the HA nodes so it is removed from the cluster (preferably not the active node). The following command is used to do this on the node:

    $ rc disable
  2. Take a backup from the HA node. Doing this is described in Section 17.8, System Backup and Restore.
  3. Add the HA node back into the cluster. If the backup was taken from the hostname3 HA node then this would be added back with the following command:

    $ rc initiate -c hostname3.tar.gz

    It is assumed that the file hostname3.tar.gz that was generated from the initial cluster setup is still stored on the HA node.

The steps to restore a backup are the following:

  1. Take the entire cluster offline by disabling all HA nodes. This is done by entering the rc disable command on each HA node separately.

  2. Restore the backup to just one HA node. Doing this is described in Section 17.8, System Backup and Restore.

  3. Set up the whole cluster from the beginning by adding each HA node, as described previously. Provided that the original .tar.gz files are still present on each HA node, this only requires the rc initiate command to be entered on each.

    Make sure that the HA node that received the backup is the last one to be added to the cluster and that the rc initiate command has the -f option. This will force the last HA node's database to be copied to the other HA nodes. For example, if hostname3 received the backup, the last initiate command would be:

    $ rc initiate -c hostname3.tar.gz -f

Node Logging in a Cluster

In the current InCenter version, the log messages sent by managed nodes are stored only on the HA node which is currently active. The logs collected are not mirrored across all HA nodes. This means that if a failover occurs, these log messages can become split across HA nodes.

[Note] Note: Communication between instances

When redundancy mode is enabled, secure communication between HA node instances in a cluster is done using TCP on port 27017 with authentication being done using the generated X.509 certificates and traffic encryption using TLS/SSL.

Load Balancer Port Forwarding

When setting up the load balancer, the following ports need to be forwarded:

  • 22 - The InCenter CLI.

  • 2222 - The Linux CLI.

  • 443 - The InCenter WebUI.

  • 8443 - The InCenter REST API.

  • 997 - The connections to managed nodes.

Other Notes About Load Balancer Functioning

The following should be noted about the functioning of the load balancer:

  • The InCenter URI is always available to the load balancer even if the REST API is disabled in InCenter.

  • The InCenter SSL certificate is automatically synced between cluster nodes so the load balancer always sees the same certificate.

Converting an Older Pre-1.64 Redundancy Cluster

Prior to InCenter version 1.64.00, redundancy was achieved used a different approach. Such older clusters will not function with InCenter version 1.64.00 or later. If upgrading a cluster to 1.64.00 or later, the old cluster should first be dismantled and then a new cluster created based on the latest InCenter version.

The following steps should be used to dismantle a pre-1.64 cluster:

  1. Disable redundancy on the secondary HA node in the old cluster.

  2. Disable redundancy on the primary HA node in the old cluster.

  3. If required to restore the current configuration later, take a configuration and log data backup from the old primary HA node and download the files to a management computer. Doing this is described in Section 17.8, System Backup and Restore.

  4. Shut down the HA nodes.

Continue with the following steps to create a new cluster:

  1. Make sure all the HA nodes in the new cluster are running the latest InCenter version.

  2. If required, restore the configuration and log files from the old cluster to any one of the HA nodes in the new cluster.

  3. Follow the procedure described earlier in this chapter to create a new redundancy cluster.

    If step 2 above has been performed and an older HA node configuration has been restored to one of the new HA nodes, make sure that that HA node is the last node added to the new HA cluster (and includes the -f option). In this way, the restored configuration will also overwrite the configuration of the other HA nodes.

Chapter 20: IPS

This section describes configuring the NetShield Intrusion Prevention System (IPS) feature using InCenter.

[Note] Note: NetWall nodes are not supported

This feature is only supported by NetShield nodes. In NetWall nodes, the cOS Core IDP feature provides the equivalent functionality. This is described in Chapter 21, IDP.

20.1. Overview

Computer servers can sometimes have vulnerabilities which leave them exposed to attacks carried by network traffic. Worms, trojans and backdoor exploits are examples of such attacks which, if successful; can potentially compromise or take control of a server. A generic term that can be used to describe these server orientated threats are intrusions.

Intrusions differ from viruses in that a virus is normally contained in a single file download and this is often downloaded to a client system which it then can infect. An intrusion manifests itself as a malicious pattern of Internet data aimed at bypassing server security mechanisms. Intrusions are not uncommon and they can constantly evolve because their creation can be automated by the attacker.

Intrusion Prevention with NetShield

The Intrusion Prevention System (IPS) provides an important defense against intrusion attempts. IPS works by scanning flows through the firewall, searching for data patterns that indicate an intrusion is being attempted. Once detected, the administrator can choose what action to take, including dropping the flow.

The data flows that IPS scans is determined by defining one or more IPSRule objects in the node configuration. The data patterns that IPS searches for are contained in one or more signature files which reside in the node's disk storage. These can consist of a single Clavister supplied signature file and/or multiple or a single custom signature file in SNORT format.

IPS Configuration Objects

The following configuration objects are used to configure IPS:

  • IntrusionPrevention

    This exists by default and is the parent object for all IPSRule objects created.

  • IPSRule

    IPSRule objects should be defined to determine what traffic should be scanned. They are independent of IPRule objects and all flows are checked against the IPSRule set.

    Each rule has a set of filtering criteria so that triggering traffic will be subject to the rule's IPSRuleAction children.

    IPS is enabled when one or more IPSRule objects exist in a node configuration. There is no global setting to enable or disable it.

  • IPSRuleAction

    One or more IPSRuleAction objects are added as children to an IPSRule object. The Action property of the object determines what action should be taken when the parent rule triggers AND one of the object's filtering criteria triggers. The filtering criteria are the IPS signatures specified by the SignatureCategory and/or SignatureGroup properties.

    The SignatureCategory and/or the SignatureGroup properties can be assigned a comma separated list. However, if a value is not specified for either property, the IPSRuleAction object will never be triggered and its associated action never taken.

    Note that a category exists in the vendor signature file for each signature. For custom signature files added by the administrator, the category of all signatures in the file is the filename itself. Multiple categories can be specified using the asterisk '*' wildcard character to represent any combination of characters. For example: IPS_WEB_*.

  • IPSSignatureGroup

    A list of one or more IPSSignatureGroup objects can be assigned to an IPSRuleAction object. Each group represents one or more IPS signatures that are to be used when the traffic is scanned. No signatures are installed in a default configuration.

    The IPSSignatureGroup object has many filtering options which are described next.

Using IPSSignatureGroup Filters

A list of one or more IPSSignatureGroup object can be assigned to the SignatureGroup property of an IPSAction object. An IPSSignatureGroup provides a means to specify individual or collections of signatures that are to be applied to the traffic specified by the associated IPSAction's parent IPSRule.

There are several filtering options for the IPSSignatureGroup which are specified using the following object properties:

  • IncludeVendorSignature

    This specifies a list of one or more individual signature SIDs which may be found in the active vendor signature file.

  • IncludeCustomSignature

    This specifies a list of one or more individual signature SIDs which may be found in the active custom signature file.

    If both this and the IncludeVendorSignature property are specified then the two are combined using a logical OR. The complete logic applied to all the filters is described at the end of this property list.

  • IncludeSignatureGroup

    This specifies a list of one or more other SignatureGroup objects. These member groups will then be filtered by the FilterBy properties specified below.

  • IncludeCategory

    The categories to be selected. Different signatures in the vendor file can have different categories. All signatures in the custom file have the same category and that category is the name of the active custom file. For example, if the active custom file is called my_sigs.sig then the assigning this category would be done with:

    IncludeCategory=my_sigs.sig
  • FilterByCVE

    The CVE (Common Vulnerabilities and Exposures) code is set for some of the signatures. The CVE for vendor signatures can be found in the advisory associated with a signature.

  • FilterBySeverity

    This property can take one of the following values:

    1. Attack or Scan for vendor signatures.
    2. Custom for all signatures in the custom file.
  • FilterByString

    This property provides the ability to perform a search for an arbitrary character sequence within the name and content of currently included filtering signatures.

  • CreatedAfter

    Only individual signatures created after a certain date will be included. All the vendor signatures will have a creation date. Custom signatures may have a creation date.

    The value specified for this property must be in the format yyyy-mm-dd. For example, to include the date 2017-03-22 and any later dates, the property would be set as:

    CreatedAfter="2017-03-21"

When multiple filter properties of the IPSSignatureGroup object are used, they are combined using the following logic:

(IncludeVendorSignature OR IncludeCustomSignature OR
		IncludeSignatureGroup OR IncludeCategory)
	AND FilterByCVE AND FilterBySeverity
	AND FilterByString AND CreatedAfter

IPS Processing Steps

The steps in IPS processing are the following:

  1. A packet arrives at an interface and the system performs normal verification. If the packet is part of a new flow then it is checked against the IP rule set first before being passed to the IPS subsystem. If the packet is part of an existing connection it is passed straight to the IPS system.

  2. Provided that the IP rules allow a flow, all traffic is checked against the IPSRule object list.

  3. If an IPSRule triggers, its child IPSRuleAction objects are processed in order. The signatures specified for each child action are applied to the traffic until one of them triggers.

  4. If the traffic finds a match in the IPSRuleAction signatures specified by the SignatureCategory or SignatureGroup, the Action property specifies what to do. The action performed can be be one of the following:

    • Ignore - Do nothing if an intrusion is detected and allow the flow to stay open.

    • Audit - Allow the flow to stay open but log the event.

    • Protect - This option drops the flow and logs the event. This is the default.

HTTP Normalization

An IPSRule object has a number of properties that can be used for HTTP normalization. This allows the administrator to choose the actions that should be taken when IPS finds inconsistencies in the URIs embedded in incoming HTTP requests. Some server attacks are based on creating URIs with sequences that can exploit weaknesses in some HTTP server products.

The IPSRule properties related to HTTP normalization are as follows:

  • URIInvalidUTF8

    This looks for any invalid UTF8 characters in a URI. The default action when triggered is DropLog.

  • URIInvalidHex

    A valid hex sequence is where a percentage sign is followed by two hexadecimal values to represent a single byte of data. An invalid hex sequence would be percentage sign followed by something which is not a valid hexadecimal value.

    The default action when triggered is DropLog.

  • URIDoubleEnc

    This looks for any hex sequence which itself is encoded using other hex escape sequences. An example would be the original sequence %2526 where %25 might be decoded by the HTTP server to '%' and results in the sequence '%26'. This is then finally decoded to '&'.

    The default action when triggered is Ignore.

Setting a Scan Limit

In some circumstances, it may not be desirable to apply IPS scanning to more than an initial portion of the data sent in a flow. This may be done for performance reasons.

The system provides the ability to impose a scan limit which means that IPS will only scan a specified number of bytes at the beginning of a flow and then scanning will stop for the remainder of the flow.

By default, the scan limit feature is disabled. To enable it, the following properties of an IPSRule object are used:

  • ScanLimit

    This is a boolean property that takes the value Yes or No. The default is No so no limit is applied. If set to Yes then a limit is applied to the number of initial bytes scanned by IPS in either direction.

  • ScanLimitBytes

    If ScanLimit is enabled then this property specifies how many bytes are scanned by IPS in a flow, in either direction, before scanning stops.

    The default value for this property is 800 bytes. This means that only the first 800 bytes for either direction of a flow will be scanned.

A CLI example of using these options can be found in Section 20.2, Setting Up IPS.

Notes About Setting a Scan Limit

The following points should be noted when using a scan limit with IPS:

  • The feature should be used with caution since an attacker might deliberately choose to insert an attack in a flow at some point after the limit given by ScanLimitBytes. However, typically it is more likely that an attack that can be detected by IPS will be present in the initial portion of a flow.

  • TCP flows are better candidates for applying a scan limit since the integrity of TCP flows is partially ensured by the mechanisms within the TCP protocol itself.

    It should be noted that if the HTTP keepalive feature (also known as HTTP persistent connection) is used on a protected webserver, it might be possible to insert an attack in HTTP requests that arrive after a scan limit is reached. For this reason, it is recommended to disable HTTP keepalive on the webserver when a scan limit is used.

  • Stateless protocols like UDP and ICMP are more exposed if a scan limit is applied since they lack the integrity mechanisms found in a stateful protocol like TCP.

  • Note that when a scan limit is enabled, no log event message is generated to indicate that a scan ended because the limit was reached. Doing this could generate an overwhelming number of log messages.

20.2. Setting Up IPS

The steps for setting up IPS are as follows:

  1. Create any required SignatureGroup objects as children of the predefined IntrusionPrevention object. This will not be needed if the signatures to be used can be specified by category.

  2. Add an IPSRule object as a child of the predefined IntrusionPrevention object to specify the traffic to be processed.

  3. Add one or more IPSRuleAction objects to the rule which specify:

    • The IPS signatures to be used when scanning the traffic targeted by the rule. This can be done by specifying one or more signature categories and/or specifying one or more SignatureGroup objects. If both a category and a group is specified then a logical OR is used in combining them.

    • The action to take when a signature triggers. As described previously, this can be one of of Protect (the default), Audit or Ignore.

  4. At least one IPS signature file must be activated. Doing this is described in Section 20.3, IPS Signature Management.

Example 20.1. Setting Up IPS for HTTP Server Protection

The following example details the steps needed to set up IPS for a simple scenario where an HTTP server is exposed to the Internet on the DMZ network with a public IPv4 address. The public Internet can be reached through the firewall on the WAN interface as illustrated below.

An IPSRule object called srv_rule will be created with the Service set to http. The SourceInterface and SourceNetwork defines where traffic is coming from, in this example the external network. The DestinationInterface and DestinationNetwork define where traffic is directed to, in this case the server.

Command-Line Interface

Change the CLI context to be the node. Assume the node name is my-node1:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1>

Change the context to be the IntrusionPrevention object which is predefined:

admin@InCenter:/my-node1> cc IntrusionPrevention 
admin@InCenter:/my-node1/IntrusionPrevention> 

Create a signature group for the signature as a child:

admin@InCenter:/my-node1/IntrusionPrevention> 
		add IPSSignatureGroup Name=my_group
		IncludeCategory=IPS_WEB_*

Create a rule as a child:

admin@InCenter:/my-node1/IntrusionPrevention> 
		add IPSRule
		SourceInterface=wan
		SourceNetwork=wannet
		DestinationInterface=dmz
		DestinationNetwork=http_server_ip
		Service=http
		Name=srv_rule
Added IPSRule/1(srv_rule)

Note that the Name property is optional.

Change the CLI context to be the rule:

admin@InCenter:/my-node1/IntrusionPrevention> 
		cc IPSRule 1(srv_rule) 
admin@InCenter:/my-node1/IntrusionPrevention/IPSRule/1(srv_rule)> 

Add the action as a child, including the group defined earlier:

admin@InCenter:/my-node1/IntrusionPrevention/IPSRule/1(srv_rule)> 
		add IPSRuleAction
		Action=Protect
		SignatureGroup=my_group

Note that the default action is Protect but it is included above for clarity.

Example 20.2. IPS Category Filtering Setup with Ignored Signatures

This example is similar to the previous but this time flows will be dropped if any signatures in the category IPS_WEB_POLICY are triggered. However, the vendor signature with the SID 64317, which is part of the IPS_WEB_POLICY category, will be an exception and will only cause a log message to be generated if it triggers.

To achieve this, two IPSRuleAction objects will be created. The first will trigger on the SID 64317 and the action taken will be Audit. The second will trigger on the category IPS_WEB_POLICY and the action will be Protect.

Command-Line Interface

Change the CLI context to be the node. Assume the node name is my-node1:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1>

Change the context to be the IntrusionPrevention object:

admin@InCenter:/my-node1> cc IntrusionPrevention 
admin@InCenter:/my-node1/IntrusionPrevention> 

Create a child signature group for the SID 64317 exception:

admin@InCenter:/my-node1/IntrusionPrevention> 
		add IPSSignatureGroup Name=my_sig_exception
		IncludeVendorSignature=64317

Add a single rule for the targeted traffic:

admin@InCenter:/my-node1/IntrusionPrevention> 
		add IPSRule
		SourceInterface=wan
		SourceNetwork=wannet
		DestinationInterface=dmz
		DestinationNetwork=http_server_ip
		Service=http
		Name=srv_rule
Added IPSRule/1(srv_rule)

Change the CLI context to be the rule:

admin@InCenter:/my-node1/IntrusionPrevention> 
		cc IPSRule 1(srv_rule) 
admin@InCenter:/my-node1/IntrusionPrevention/IPSRule/1(srv_rule)> 

Add an action so that SID 64317 is logged but not dropped:

admin@InCenter:/my-node1/IntrusionPrevention/IPSRule/1(srv_rule)>  
		add IPSRuleAction
		Action=Audit
		SignatureGroup=my_sig_exception

Add a second action so flows that trigger the IPS_WEB_POLICY category (with the exception of SID 64317) are dropped:

admin@InCenter:/my-node1/IntrusionPrevention/IPSRule/1(srv_rule)>  
		add IPSRuleAction
		Action=Protect
		SignatureCategory=IPS_WEB_POLICY

Note that the default action is Protect but it is explicitly stated above for clarity.

Example 20.3. Setting an IPS Scan Limit

This example shows how an IPS scan limit of 2048 bytes can be set in the IPSRule from the previous example.

Command-Line Interface

From the previous example, add a single rule for the targeted traffic:

admin@InCenter:/my-node1/IntrusionPrevention> 
		add IPSRule
		SourceInterface=wan
		SourceNetwork=wannet
		DestinationInterface=dmz
		DestinationNetwork=http_server_ip
		Service=http
		Name=srv_rule
		ScanLimit=Yes
		ScanLimitBytes=2048
Added IPSRule/1(srv_rule)

20.3. IPS Signature Management

The IPS feature relies on signature files which must be first uploaded to InCenter and which are then deployed to a node.

There are two kinds of IPS signature files:

  • Vendor Signature Files

    These are provided by Clavister as a set of predefined signatures in a single file which cannot be changed by the administrator. Updated versions of the vendor signature file are provided by Clavister.

    When a vendor file is activated for a node, it is automatically uploaded to that node, overwriting any previously activated vendor file. There can be only one active vendor file on a node at any one time.

  • Custom Signature Files

    These are files in the SNORT format that can be created or edited by the administrator and they can be sourced from third parties. More than one can be active on a node at the same time. These files can be used with or instead of an active vendor signature file.

    When a custom file is activated for a node, it is automatically uploaded to that node, overwriting any activated file with the same name. Multiple custom files can be active on a node at the same time.

    The SNORT conventions supported are described in Section 20.4, SNORT File Usage.

The following should be noted about the management of IPS signature files using InCenter:

  • No signature files exist in the default InCenter configuration.

  • Signature files are uploaded to the InCenter server using SCP/SFTP/REST. They are kept in the folder called ipsSignatures.

  • Multiple files of both the vendor and custom signature type can reside in InCenter storage at the same time.

  • Signature files residing on the InCenter server can be deployed to nodes by activating them.

  • Signature files residing on nodes by deactivating them.

Vendor Signature Types

The signatures in the vendor file can be further broken down into the following types:

  • Intrusion Protection Signatures (IPS)

    The category of these signatures has the IPS_ prefix. They are highly accurate and a match is almost certainly an indicator of a threat. Using the Protect action is recommended. These signatures can detect administrative actions and security scanners.

  • Intrusion Detection Signatures (IDP)

    The category of these signatures has the IDS_ prefix. They can detect events that may be intrusions. They have lower accuracy than the IPS signatures and may give some false positives so it is recommended that the Audit action is always used. Using them with the Protect action may interrupt normal traffic.

Activating Signature Files

Once the IPS objects are configured, IPS will not process any traffic until at least one signature file in the ipsSignatures folder is activated via the InCenter CLI using the ips -activate command. For example:

admin@InCenter:/> ips -activate my-sigs.dat my-node1,my-node2

This command will upload the signature file my-sigs.dat to the nodes my-node1 and my-node2 from InCenter and also makes the file active on the nodes so that the IPS subsystem can use them.

Deactivating Signature Files

Deactivating signature files removes the file from a node and so it will no longer be used for IPS processing.

The ips Command

The InCenter CLI command ips provides a set of options for managing and viewing the signature files kept in the ipsSignatures folder. It is similar to, but not exactly the same as, the ips command found in the node's CLI.

When browsing and displaying IPS signatures, the InCenter ips command will only look at the local signature files in InCenter storage unless a node name is specified then only the signature files on that node will be looked at.

Displaying the IPS Status

The ips -status command shows the signature files that are available in InCenter storage. For example:

admin@InCenter:/> ips -status
File                     Origin   Issued      Last Loaded  Status
------------------------ -------  ----------  -----------  ------
ipssigs_2017101109.dat   Custom               2017-10-19   Loaded
ipssigs_2017101715.dat   Custom               2017-10-19   Failed
ipssigs_2017101805.dat   Factory  2017-10-18  2017-10-19   Loaded

The Status column indicates if the file was successfully recognized as a signature file. A status of Failed means that it is either empty or invalid.

Displaying a Signature

The ips -show=signature command will display information about signatures:

admin@InCenter:/> ips -show=signature <sig_id> [-num=<num>]

Where <num> is the maximum number of lines to display. This option can be added to nearly all the forms of the ips command.

admin@InCenter:/> ips -show=signature 20040717
Property      Value
------------  -------------------------------------------------
         ID:  20040717
       Name:  Auth.SambaWebAdminTool.Buffer.Overflow
Description:  A buffer overflow vulnerability in the Samba Tool
   Category:  IDS_SMB_EXPLOIT
   Severity:  Scan
       File:  ipssigs_201710180526.dat
        CVE:  2004-0600
    Created:  2007-10-18
    Updated:  2010-11-30
Used in IPS signature group(s): my-node1/group2

Displaying a Group

The ips -show=group command will display information about a IPSSignatureGroup object on a node:

admin@InCenter:/> ips -show=group <node>/<group>

For example:

admin@InCenter:/> ips -show=group my-node1/group2 -num=2
Property         Value
---------------  -------
          Name:  group2
      Comments:  <empty>
Items in Group:  6
Type       ID        Name
---------  --------  --------------------------------------
Category             IDS_SMB_EXPLOIT
Signature  20040717  Auth.SambaWebAdminTool.Buffer.Overflow
Showing 2 out of 6.

Displaying a Category

The ips -show=category command will display information about the categories in the signature files in InCenter storage:

admin@InCenter:/> ips -show=category <category>

For example:

admin@InCenter:/>  ips -show=category IDS_SMB_EXPLOIT -num=1
Property     Value
-----------  ---------------
      Name:  IDS_SMB_EXPLOIT
Signatures:  5
Used in IPS signature group(s): my-node1/group2
Members:
ID        Name                                    Category
--------  --------------------------------------  ---------------
20040717  Auth.SambaWebAdminTool.Buffer.Overflow  IDS_SMB_EXPLOIT
Showing 1 out of 5.

Listing Signatures

The ips -list=signature will list signatures matching a query value:

admin@InCenter:/> ips -list=signature [query=<value>]

The value entered for any query for this and subsequent command is case insensitive and can use the wildcards asterisk '*' for any characters and question mark '?' for any single character.

For example:

admin@InCenter:/> ips -list=signature -query=attack -num=3
ID        Name                         Category
--------  ---------------------------  ----------------------- 
20057242  Trojan.DDoS.OrbitDownloader  IPS_MALWARE_COMMCONTROL
20057565  Trojan.p3rlbot.irc.ddos      IPS_MALWARE_COMMCONTROL
20057513  Trojan.psPerlBot.CnC.DDoS    IPS_MALWARE_COMMCONTROL
Showing 3 out of 15.

If the query= option is left out for this or subsequent commands then individual signatures are not listed but a summary of signature numbers instead.

Listing Groups

The ips -list=group command will list groups matching a query value:

admin@InCenter:/> ips -list=group [query=<value>]

For example:

admin@InCenter:/> ips -list=group -query=dev?/gr*2
Name              Comments
----------------  --------
my-node1/my-grp1
my-node1/my-grp2
Showing 2 out of 2.

Listing Categories

The ips -list=category command will list categories matching a query value:

admin@InCenter:/> ips -list=category [query=<value>]

For example:

admin@InCenter:/> ips -list=category -query=IDS -num=3
Name                    Signatures
----------------------  ----------
IDS_ANTIVIRUS_CLAMAV    2
IDS_ANTIVIRUS_SYMANTEC  2
IDS_APP_ADOBE           1
Showing 3 out of 110.

20.4. SNORT File Usage

As mentioned previously, a custom signature file is in SNORT format and these can be created by third parties or by the system administrator. SNORT is described in depth at the following link:

https://snort.org

The standard form of SNORT signatures is a filter followed by options:

action protocol src-net src-ports direction dest-net dest-ports ( options )

However, it is important to be aware of how some SNORT file conventions are interpreted by the system and what limitations exist. This is described next.

SNORT Filter Usage

The SNORT filter is interpreted as follows:

  • action

    Ignored. The actions associated with the triggering IPSRuleAction object are used instead.

  • protocol

    Only the values tcp, udp or icmp are supported.

  • source-net

    Ignored.

  • source-port

    Syntax:

    <min>
    <min>:<max>
    <min>:
    :<max>
    any

    If <min> is left out, it defaults to 0. If <max> is left out it defaults to <min>, unless <min>: is supplied then <max> defaults to 65535. Only a single port range is supported, negated ranges not supported. Commonly used variables, such as $HTTP_PORTS, are not supported.

  • direction

    Only -> is supported.

  • destination-net

    Ignored

  • destination-port

    This follows the same rules as source-port.

Supported Options

The following SNORT options are supported with any restrictions listed:

  • content:[!]"string |0d 0a|";

    Specifies a string to look for in the packet data. Hex data can be embedded, as shown. Some characters need to be escaped, like \, ", | and ;

  • uricontent:[!]"string |00|";

    Specifies a string to look for in the normalized HTTP URI. Hex data can be embedded, as shown. Some characters need to be escaped, like \, ", | and ;

  • http_uri;

    Modifies the preceding content option to look in the normalized HTTP URI instead of the raw packet data.

  • nocase;

    Modifies the preceding content or uricontent to be case insensitive.

  • offset:<uint>;

    Modifies the preceding content specifying that the search for it should start <uint> bytes into the packet. Only positive offsets supported.

  • depth:<uint>;

    Modifies the preceding content specifying that it will only be looked for within <uint> bytes from the start of the packet or the content's offset (see above) if set.

  • distance:<uint>;

    Modifies the preceding content specifying that the search for it should start <uint> bytes from the point where the previous content matched. Only positive distances are supported. This defaults to zero making all contents ordered.

  • within:<uint>;

    Modifies the preceding content limiting the number of bytes that will be searched to find it to <uint> relative to the previous content match plus distance (if specified, see above). Only positive within values supported.

  • dsize:[<|>]<number>;

    Packet size must be less than (<), equal to, or greater than (>) number. Packet size range in the syntax: dsize:<min><><max>; is not supported. isdataat is not supported.

  • flowbits:set,<flag>;

    Set a flag in the flow that can be tested by other signatures.

  • flowbits:noalert;

    This signature will never trigger log/drop action. Useful together with flowbits:set,<flag>; for signatures that will match quite often and in itself is not an attack but enables other signatures that otherwise would be prone to false-positives.

  • flowbits:isset,<flag>;

    Specifies that a certain flag must be set in the flow for this signature to match.

  • flow:<keywords>;

    A comma separated list of groups. Only to_client, from_client, to_server, from_server is supported.

  • msg:"message";

    Message/name included in logs.

  • sid:<uint>;

    Signature id, included in logs.

  • rev:<uint>;

    Signature revision, included in logs.

  • reference:<type>,<id>;

    Example:

    reference:cve,2004-0646;
    reference:cve,CAN-2005-1252;reference:bugtraq,9368;
    reference:url,www.securiteam.com/unixfocus/5FP0J15CKE.html;
    reference:nessus,11131;reference:cve,2017-5638;
  • metadata:<key> <value>(, <key> <value>)*;

    Key and value are separated by space and may not contain " " (space), "," or ";". Multiple key-value pairs are separated by commas.

  • classtype:<class name>;

    Defines the type of threat that this signature detects. Example:

    attempted-admin, shellcode-detect, trojan-activity,
    web-application-attack, misc-attack, network-scan

20.5. Best Practice Deployment

IPS Deployment Considerations

In order to have an effective and reliable IPS system, the following questions should be considered by the administrator:
  • What kinds of traffic should be analyzed?
  • What kinds of intrusions should be searched for in that traffic?
  • What action should be carried out if an intrusion attempt is detected?

IPS Deployment Recommendations

The following are the recommendations for IPS employment:

  • Enable only the IPS signatures for the traffic that is being allowed. For example, if the IP rule set is only allowing HTTP traffic then there is no point enabling FTP signatures.

  • Once the relevant IPS signatures are selected, initially run in Audit mode.

  • After running in Audit mode for a sample period with live traffic, examines the log messages generated. Check for the following:

    1. When IPS triggers, what kind of traffic is it triggering on?

    2. Is the correct traffic being identified?

    3. Are there any false positives with the signatures that have been chosen?

  • Adjust the signature selection and examine the logs again. There may be several adjustments before the logs demonstrate that the desired effect is being achieved, with the very minimum of false positives.

    If certain signatures are repeatedly triggering it may indicate a server is under attack.

  • After a short period running in Audit mode with satisfactory results showing in the logs, switch over IPS to Protect mode so that triggering connections are dropped.

Chapter 21: IDP

This section describes configuring the NetWall Intrusion Detection and Prevention (IDP) feature using InCenter.

[Note] Note: NetWall nodes are not supported

This feature is only supported by NetWall nodes. In NetShield nodes, the cOS Core IPS feature provides the equivalent functionality. This is described in Chapter 20, IPS.

21.1. Overview

Computer servers can sometimes have vulnerabilities which leave them exposed to attacks carried by network traffic. Worms, trojans and backdoor exploits are examples of such attacks which, if successful; can potentially compromise or take control of a server. A generic term that can be used to describe these server orientated threats are intrusions.

Intrusions differ from viruses in that a virus is normally contained in a single file download and this is often downloaded to a client system which it then can infect. An intrusion manifests itself as a malicious pattern of Internet data aimed at bypassing server security mechanisms. Intrusions are not uncommon and they can constantly evolve because their creation can be automated by the attacker.

IDP with NetWall

Intrusion Detection and Prevention (IDP) provides an important defense against intrusion attempts. IDP works by scanning traffic that flows through the NetWall node, searching for data patterns that indicate an intrusion is being attempted. Once detected, the administrator can configure what action should be taken, including dropping the connection.

The traffic that IDP scans is determined by defining one or more IDPRule objects in the cOS Core configuration. The data patterns or signatures that IDP searches for are contained in a single signature database which resides in cOS Core non-volatile storage.

IDP Signature Storage and Updates

It should be noted that the IDP signature database is not held centrally on the InCenter server. The entire database is replicated across individual NetWall nodes and is periodically updated automatically for each node over the Internet. Setting updating up requires the following cOS Core CLI command:
admin@InCenter:/my-node1> set UpdateCenter IDPEnabled=Yes
			EnableProxy=Yes
			HTTPProxyIP=<tpm-ipaddress>
			UpdateInterval=Daily
			UpdateHour=<hour-value>
			UpdateMinute=<minute-value>
Where nodes do not have direct access to the Internet, they can use InCenter as a proxy server for signature updates and this is specified by the HTTPProxy parameter in the command above. In addition, InCenter must be configured to act as the proxy and doing this is described in Section 6.10, Internet Proxy Setup.

Note that the complete documentation for the IDP feature in NetWall nodes can be found in the IDP section of the separate cOS Core Administration Guide.

IDP Configuration Objects

The following configuration objects are used to configure IDP:

  • IDPRule

    IDPRule objects should be defined to determine what traffic should be scanned. They are independent of IP rule set entries and all traffic flows are checked against the IDPRule set.

    Each rule has a set of filtering criteria so that triggering traffic will be subject to the rule's IDPRuleAction children.

    IDP is enabled when one or more IDPRule objects exist in a node configuration. There is no global setting to enable or disable it.

  • IDPRuleAction

    One or more IDPRuleAction objects are added as children to an IDPRule object. The Action property of the object determines what action should be taken when the parent rule triggers AND one of the object's filtering criteria triggers. The IDP signatures to apply are specified by the Signatures property.

    The Signatures property can be assigned a comma separated list of signature numbers. However, if a value is not specified for the property, the IDPRuleAction object will never be triggered and its associated action never taken.

IDP Processing Flow

The steps in IDP processing flow are the following:

  1. A packet arrives at an interface and the system performs normal verification. If the packet is part of a new connection then it is checked against the IP rule set first before being passed to the IDP subsystem. If the packet is part of an existing connection it is passed straight to the IDP system.

  2. Provided that the IP rules allow the new connection, the connection is then checked against the IDPRule object list. If an IDPRule matches, IDP scanning will begin on the connection using that rule and its child actions.

  3. If the IDP scanning begins, the rule's child IDPRuleAction objects are processed in order. The signatures specified for each child action are applied to the traffic.

  4. If the traffic triggers a match in one of the IDPRuleAction signatures, the Action property is used to determine what to do. The action performed can be be one of the following:

    • Protect - This option drops the connection and logs the event.

      This is the default.
    • Audit - Allow the connection to stay open but log the event.

    • Ignore - Do nothing if an intrusion is detected and allow the connection to stay open.

    • Pipe - Constrain the bandwidth coming from the host using a traffic shaping pipe.

HTTP Normalization

An IDPRule object has a number of properties that can be used for HTTP normalization. This allows the administrator to choose the actions that should be taken when IDP finds inconsistencies in the URIs embedded in incoming HTTP requests. Some server attacks are based on creating URIs with sequences that can exploit weaknesses in some HTTP server products.

The IDPRule properties related to HTTP normalization are as follows:

  • URIInvalidUTF8

    This looks for any invalid UTF8 characters in a URI. The default action when triggered is DropLog.

  • URIInvalidHex

    A valid hex sequence is where a percentage sign is followed by two hexadecimal values to represent a single byte of data. An invalid hex sequence would be percentage sign followed by something which is not a valid hexadecimal value.

    The default action when triggered is DropLog.

  • URIDoubleEnc

    This looks for any hex sequence which itself is encoded using other hex escape sequences. An example would be the original sequence %2526 where %25 might be decoded by the HTTP server to '%' and results in the sequence '%26'. This is then finally decoded to '&'.

    The default action when triggered is Ignore.

Setting a Scan Limit

In some circumstances, it may not be desirable to apply IDP scanning to more than an initial portion of the traffic in a connection. This might be done for performance reasons since applying IDP to all the traffic in a connection may consume too many system resources.

IDP provides the ability to impose a scan limit which means that IDP will only scan a specified number of bytes at the beginning of a flow and then scanning will stop for the remainder of the flow.

By default, the scan limit feature is disabled. To enable it, the following properties of an IDPRule object are used:

  • ScanLimit

    This is a boolean property that takes the value Yes or No. The default is No so no limit is applied. If set to Yes then a limit is applied to the number of initial bytes scanned by IDP in either direction.

  • ScanLimitBytes

    If ScanLimit is enabled then this property specifies how many bytes are scanned by IDP in a flow, in either direction, before scanning stops.

    The default value for this property is 800 bytes. This means that only the first 800 bytes for either direction of a flow will be scanned.

A CLI example of using these options can be found in Section 21.2, Setting Up IDP.

Notes About Setting a Scan Limit

The following points should be noted when using a scan limit with IDP:

  • The feature should be used with caution since an attacker might deliberately choose to insert an attack at some point after the limit given by ScanLimitBytes. However, typically it is more likely that an attack that can be detected by IDP will be present in the initial portion.

  • TCP flows are better candidates for applying a scan limit since the integrity of TCP flows is partially ensured by the mechanisms within the TCP protocol itself.

    It should be noted that if the HTTP keepalive feature (also known as HTTP persistent connection) is used on a protected webserver, it might be possible to insert an attack in HTTP requests that arrive after a scan limit is reached. For this reason, it is recommended to disable the HTTP keepalive option on the webserver when a scan limit is used.

  • Stateless protocols like UDP and ICMP are more exposed if a scan limit is applied since they lack the integrity mechanisms found in a stateful protocol like TCP.

  • Note that when a scan limit is enabled, no log event message is generated to indicate that a scan ended because the limit was reached. Doing so could generate an overwhelming number of log messages.

21.2. Setting Up IDP

The steps for setting up IDP are as follows:

  1. Create an IDPRule object to specify which traffic is to be processed by IDP.

  2. Add one or more IDPRuleAction objects to the rule which specify:

    • The IDP signatures to be used when scanning the traffic targeted by the rule. This can be done by specifying one or more signature categories and/or specifying a list of signatures. If both a category and a list is specified then a logical OR is used in combining them.

    • The action to take when a signature triggers. As described previously, this can be one of of Protect (the default), Audit or Ignore.

Example 21.1. Setting up IDP for HTTP Server Protection

The following example details the steps needed to set up IDP for a simple scenario where an HTTP server is exposed to the Internet on the DMZ network with a public IPv4 address. The public Internet can be reached through the firewall on the WAN interface as illustrated below.

An IDPRule object called srv_rule will be created with the Service set to http. The SourceInterface and SourceNetwork defines where traffic is coming from, in this example the external network. The DestinationInterface and DestinationNetwork define where traffic is directed to, in this case the server.

Command-Line Interface

Change the CLI context to be the node. Assume the node name is my-node1:

admin@InCenter:/> cc StandaloneNode my-node1 
admin@InCenter:/my-node1>

Create an IDP rule:

admin@InCenter:/my-node1> add IDPRule
		SourceInterface=wan
		SourceNetwork=wannet
		DestinationInterface=dmz
		DestinationNetwork=http_server_ip
		Service=http
		Name=srv_rule

Note that the Name property is optional but recommended for clarity.

Change the CLI context to be the rule:

admin@InCenter:/my-node1> cc IDPRule 1 
admin@InCenter:/my-node1/IDPRule/1> 

Add the action as a child:

admin@InCenter:/my-node1/IDPRule/1> add IDPRuleAction
		Action=Protect
		Signatures=72828,72829,58183,52543

Note that the default action is Protect but it is included above for clarity.

21.3. Best Practice Deployment

IDP Deployment Considerations

In order to have an effective and reliable IDP system, the following questions should be considered by the administrator:
  • What kinds of traffic should be analyzed?

  • What kinds of intrusions should be searched for in that traffic?

  • What action should be carried out if an intrusion attempt is detected?

IDP Deployment Recommendations

The following are the recommendations for IDP employment:

  • Enable only the IDP signatures for the traffic that is being allowed. For example, if the IP rule set is only allowing HTTP traffic then there is no point enabling FTP signatures.

  • Once the relevant IDP signatures are selected, initially run in Audit mode.

  • After running in Audit mode for a sample period with live traffic, examines the log messages generated. Check for the following:

    1. When IDP triggers, what kind of traffic is it triggering on?

    2. Is the correct traffic being identified?

    3. Are there any false positives with the signatures that have been chosen?

  • Adjust the signature selection and examine the logs again. There may be several adjustments before the logs demonstrate that the desired effect is being achieved, with the very minimum of false positives.

    If certain signatures are repeatedly triggering it may indicate a server is under attack.

  • After a short period running in Audit mode with satisfactory results showing in the logs, switch over IDP to Protect mode so that triggering connections are dropped.

Appendix A: Third Party Software

The InCenter product makes use of various third party software. A list of all third party software and the associated licenses can be found in a single text file called acknowledgments.txt which is included with each release of the software.

The file can be downloaded by first pressing the About button at the top pane of the display.

The About Button

Figure A.1. The About Button

This displays the About dialog. The acknowledgements button in the dialog can then be pressed to download the acknowledgements file. This is shown in the example screenshot below.

Downloading the Third Party Software File

Figure A.2. Downloading the Third Party Software File

As can be seen in the above screenshot, the About dialog includes the release notes for the version of InCenter that is running. These notes are also displayed automatically after user login if there has been a version upgrade of InCenter since the last login.