7.2. IP Reputation

Home Prev	cOS Core 14.00.17 Administration Guide	Next

Overview

cOS Core provides the ability to examine the source and destination IPv4 addresses for any connection and determine an IP Reputation value for those IPs. This can be done for public IPv4 addresses only. The feature is a subscription based service so it requires an appropriate cOS Core support agreement to function.

The IP reputation determination is done by looking up an IP in an IP reputation database, part of which is cached locally in the Clavister NetWall Firewall for high threat IPs. The remaining part of the database is stored on external servers. The database consists both of unique IP addresses as well as IP ranges which can pose a threat. Each IP or range is determined to pose a potential threat from a lookup in the database, it then has a Threat Category associated with it which defines the type of threat and also has a Threat Level which defines the severity of the threat.

The reputation database is provided by and constantly updated by the Webroot™ company which is well known for the quality and breadth of their IP reputation offering. This database is accessed via Clavister servers and therefore requires Internet access by cOS Core.

IP Reputation Requires a Valid Subscription

IP reputation is a subscription based feature and the current cOS Core license must include a valid subscription date for the feature to work. See Appendix A, Subscription Based Features for more details about this and for details about the behavior when a subscription expires.

IP Reputation Threat Levels

The IP reputation system assigns a threat level score to an IP address from 1 to 100. The following table shows the threat classifications in this range:

Range	Threat Level	Information
1-20	High risk	6-12 million IPs in the local database
21-40	Suspicious	Unclassified IPs have a score of 40
41-60	Moderate risk
61-80	Low risk
81-100	Trustworthy

It is the High Risk part of the IP database that is stored locally in the Clavister NetWall Firewall for fast access by cOS Core.

Threat Categories

When an IP address is assigned a score and therefore a threat level by the IP reputation subsystem, it is done so by placing it into one or more categories and these are listed below. If an IP does not get a category associated with it then it is considered not to be a threat. The categories associated with an IP are listed in the log message generated for a connection.

Threat Category	Use Cases	Description
Spam sources	Inbound	Tunneling Spam messages through proxy, anomalous SMTP activities, Forum Spam activities.
Windows Exploits	Inbound	Active IP addresses offering or distributing malware, shell code, rootkits, worms or viruses.
Web Attacks	Inbound	Cross site scripting, iFrame injection, SQL injection, cross domain injection or domain password brute force.
BotNets	Inbound Outbound	Botnet C&C channels and infected zombie machines controlled by bot master.
Scanners	Inbound	All reconnaissance such as probes, host scan, domain scan and password brute force.
Denial of Service	Inbound	DOS, DDOS, anomalous SYN flood, anomalous traffic detection.
Reputation	Inbound	IP addresses currently known to be infected with malware. This category also includes IPs hosting URLs with consistently low reputation scores.
Phishing	Inbound	IP addresses hosting phishing sites and other kinds of fraud activities such as ad click fraud or gaming fraud.
Proxy	Inbound	IP addresses providing proxy and anonymizer services. This category also includes TOR anonymizer IP addresses.
Mobile Threats	Inbound	IP addresses of malicious and unwanted mobile applications.
Package	Inbound	This type contains all other types.
TOR Proxy	Inbound	IP addresses acting as exit nodes for the Tor network. Exit nodes are the last point along the proxy chain and make a direct connection to the originator’s intended destination.

IP Reputation Characteristics

The following characteristics of the IP reputation subsystem should be noted.

The IP reputation logging feature is controlled by a single global setting and this is enabled by default. How to change this setting is described in Example 7.2, “Enabling IP Reputation Logging”.
When enabled, all public (global unicast) source and destination IPv4 addresses for all interfaces are checked against the reputation database. The exception is the public IPs of cOS Core interfaces which are assumed to be trusted.
One log message is generated per public IP address for every database lookup. This message contains both the reputation score of the IP as well as the categories that the IP triggered for the score calculation.
The source and destination IP addresses will be looked up only once for a connection tracked by the cOS Core state table.

Data packets that are subject to a FwdFast IP rule or a Stateless IP policy are not tracked in the cOS Core state table and will never be evaluated by the IP reputation subsystem.

Excluded IP Addresses

The following IPv4 addresses are never looked up by the IP reputation subsystem:

The public IP addresses assigned to cOS Core interfaces (as mentioned already).
All private IP addresses (192.168.0.0/16 etc.) according to RFC-1918 and the localhost IP (127.0.0.1).
Any broadcast or multicast addresses.

Database Updating

The entire IP reputation database is accessed via Clavister servers across the Internet. It contains the IPs and networks for all threat levels and this external database is constantly updated throughout the day.

A portion of this database is mirrored on local cOS Core disk to avoid the latency of cloud lookups. This local database portion only contains IPs and networks with a threat level of High Risk, where the reputation score is between 1 and 20. This local database portion is updated in its entirety once every 24 hours.

cOS Core also maintains a smaller cache in fast memory of the most recent IP reputation lookups in order to further improve lookup performance. In addition, there can be partial local database updates sent by Clavister servers which can occur between the 24 hour complete updates. These partial updates are also stored in the cache until the entire high risk local database is updated.

In summary, there are three levels of the IP reputation database: the entire database which exists externally in the cloud; the local database copy kept locally on cOS Core disk containing only high threat IPs; the cOS Core cache which is kept in fast memory and contains recent lookups plus any partial database updates.

IP Reputation Log Messages

With IP reputation logging enabled, an IP_reputation log message is generated for each IP lookup. A typical message is shown below:

CONN prio=Info id=00600120 rev=1 event=ip_reputation action=none
ip=203.0.113.4 score=10 categories="Spam Sources, Windows Exploits,
Web Attacks" connipproto=ICMP connrecvif=core connsrcip=203.0.113.4
connsrcid=27507 conndestif=wan conndestip=192.168.98.14 conndestid=27507

Log Message Display in the Web Interface

IP reputation log messages can also be viewed in the cOS Core Web Interface by going to Status > IP Reputation Logging. However, this status page will only display log messages where the IP reputation score is 40 or below (in other words, either of the two highest threat levels).

IP Reputation Lookup Latency

Since only high threat IPs are kept in the local portion of the IP reputation database, there can be some time delay if the IP is not found locally and a query needs to be sent to the part of the database kept in the cloud. This means that the IP reputation log message may be generated after the connection has opened and, in the case where the connection is open momentarily (such as an ICMP ping exchange), after the connection has closed.

Forcing Database Updates

It is possible to force an update of locally stored IP reputation data by using the following CLI command:

Device:/> updatecenter -update ipreputation

This action can also be performed through the Web Interface by going to Status > Maintenance > Update Center and pressing the Update button for IP reputation.

When an update is forced, the cache will be partially cleared. All high risk cache entries are removed (those with a threat level of 20 or less) which have had a recent positive hit (all cache entries have a maximum lifetime of 60 hours). This ensures a continued rapid lookup for commonly accessed IPs that are harmless.

However, a forced update does not mean all partial local database updates stored in the cache are loaded into the local IP reputation database on disk. That will only happen every 24 hours when there is a complete update of the local database. However, the forced update does provide a way to update the cache when, for example, cOS Core has not had access to the Clavister servers for a period of time and partial updates may be missing.

Download and Updating is Automatic After Starting Up

When cOS Core starts, if the database has never been downloaded before or the last 24 hour update was missed, cOS Core will automatically download the most recent 24 hour full local database update. cOS Core also downloads on startup any partial updates that had been missed since the last 24 hour update and stores them in the IP reputation cache.

Note that this automatic downloading of the database will occur if the IP reputation logging feature is enabled or not. As long as a valid subscription exists for the feature, the portions of the IP reputation database that are stored locally will be automatically downloaded.

Local Database Preservation During a Restart

Reconfiguring cOS Core will not affect the IP reputation local database and cache. However, restarting cOS Core will completely clear the cache but will leave the local database unaffected. As explained above, any high threat partial updates will be automatically downloaded from Clavister servers after starting up and stored in the cache.

The ipreputation CLI Command

The CLI command ipreputation provides the administrator a way to directly interact with the IP reputation to perform IP queries and to examine the state of the subsystem.

To query an IP address, the -query option can be used:

Device:/> ipreputation -query 203.0.113.6
		
Result of query
---------------
Address            :  203.0.113.6
Response IP        :  203.0.113.6
Reputation         :  10
Categories         :  Spam_Sources Windows_Exploits Web_Attacks 
In IP Threat List  :  1
Found in           :  Local Database

The -category option could be added to this command to search within just a particular category.

Note that the Found in line above indicates where the lookup found a match. The possible values are the following:

Cloud lookup - The IP was found on an external database server.
Local Database - The IP was found in the high risk part of the IP reputation database that is kept locally by cOS Core.
Cache - The IP was found in the local cOS Core cache, meaning that either it had been already looked up recently or was contained in a partial database update (as previously explained, partial updates to the high risk local database are initially stored in the cache).

Note also that IP reputation logging does not need to be enabled in order to use the -query option.

The -show -verbose option displays the statistics for the IP reputation engine since the last restart or since it was enabled along with details of the last database updates:

Device:/> ipreputation -show
		
IP Reputation Engine Status
 ----------------------------------------------
 Database Entries

 IP Addresses        :   1489822
 IP Ranges           :   16783
 Offline DB total    :   1506605
 In latest partial   :   1
 Cache Entries       :   9

 Database updates

 Database date       :   2021-02-07 12:56:00
 Partial update date :   2021-02-07 12:56:00
 Last update check   :   2021-03-16 13:11:49
 Last update         :   2021-03-16 13:11:49

 Server              :   203.0.113.1
 Status              :   Connected

The -subsystems option list those parts of cOS Core that are making use of the IP reputation engine:

Device:/> ipreputation -subsystems
		
Name
-------------
State Engine
CLI

The value State Engine means that logging is switched on. The value CLI means that the engine has been used in a CLI lookup. Other values could be threat prevention objects, such as a DoS Protection object, that are performing IP reputation lookups. As explained later in the discussion of threat prevention objects, the usage by different subsystems is independent from each other.

All the options for the ipreputation command can be found in the separate cOS Core CLI Reference Guide.

Example 7.2. Enabling IP Reputation Logging

This example enables IP reputation evaluation and logging for all interfaces.

Command-Line Interface

Device:/> set Settings StateSettings IPReputationLogs=Yes

InControl

Follow similar steps to those used for the Web Interface below.

Web Interface

Go to: System > Advanced Settings > State Settings
Enable Log IP Reputation
Click OK
Go to Status > IP Reputation Log to check that the feature is functioning.

IP Reputation Usage by Threat Prevention Objects

The IP reputation subsystem is used by the following cOS Core configuration objects for threat prevention to determine if a connection should be dropped:

DoS Protection

The DoS Protection object provides a defense against denial of service attacks against a server. This object is described further in Section 7.4, DoS Protection.
Scanner Protection

The Scanner Protection object provides a defense against an external host scanning the ports of a server looking for vulnerabilities. This object is described further in Section 7.5, Scanner Protection.
Botnet Protection

The Botnet Protection object provides a defense against both internal and external clients or servers being part of a botnet. This object is described further in Section 7.3, Botnet Protection.
Phishing Protection

The Phishing Protection object provides a defense against phishing attempts directed towards protected clients. This object is described further in Section 7.6, Phishing Protection.
Spam Protection

The Spam Protection object provides a defense against Spam traffic directed towards protected clients. This object is described further in Section 7.7, Spam Protection.

Threat Prevention Objects Use the Local Part of the IP Reputation Database

When the IP reputation subsystem is used by threat prevention objects such as Botnet Protection, only the local portion of the IP reputation database is referenced. This means:

The IP reputation score is determined immediately with no latency problems. The lookup time is also kept to a minimum by the local IP reputation database cache.
Since the locally stored portion of the database only contains IPs that present the most serious threats, the chances of false positives are minimized.

IP Reputation Logging is Independent of Threat Prevention Objects

The IP reputation logging feature and the threat prevention objects operate independently but there is no duplication of IP reputation lookups. If IP reputation logging is enabled and one or more threat prevention objects, such as DoS Protection, are also enabled, the order of processing is as follows for a new connection:

Any enabled threat prevention objects will be applied first. The first one that triggers will drop the connection, add the IP to the blacklist and generate a blacklist log message. No further processing is done by the IP reputation subsystem.
If a threat prevention object has not dropped the connection, the IP reputation logging subsystem will generate a lookup log message for it if that feature is enabled.

Turning Off IP Reputation

When turning IP reputation off, the following checklist should be followed:

Make sure no feature is making use of IP reputation in the configuration. For example, make sure it is not being used by any of the configured DoS Protection, Scanner Protection, Botnet Protection Phishing Protection or Spam Protection objects.
In the Web Interface, go to System > Advanced Settings > State Settings and disable Log IP Reputation.
Go to Status > IP Reputation Log to check that the feature has been disabled.

Disabling IP reputation is also discussed in an article in the Clavister Knowledge Base at the following link:

https://kb.clavister.com/324735676

IP Reputation Blocking "Trusted" Websites

It's possible that the administrator gets feedback from internal users that the IP Reputation subsystem is blocking websites that they consider to be "trusted". There is a discussion of this topic in the Clavister Knowledge Base at the following link:

https://kb.clavister.com/329091873

Q&A Discussion of IP Reputation Usage

A general question and answer discussion of using the IP reputation feature can be found in a Clavister Knowledge Base article at the following link:

https://kb.clavister.com/329099584