System:/> set Settings HWMONSettings SensorRefreshInterval=20

In contrast to System Hardware Monitoring, the feature called User Hardware Monitoring is used to generate log event messages for specific sensors and conditions that are defined by the administrator.

Enabling User Hardware Monitoring

To enable user hardware monitoring for a sensor, a HWMONMonitor object must be created in the configuration and associated with the sensor. Only one sensor can be associated with one HWMONMonitor object.

Assume that monitoring of the CPU temperature is required. If the sensor name is CPU_Temp and the new object is to be called mon1, user monitoring for this sensor can be enabled with the following CLI command:

System:/> add HWMONMonitor mon1 SensorID=CPU_Temp

The HWMONMonitor object created will generate an event message when its sensor has a value that moves outside the thresholds defined for the object (in the case of temperatures and fan speeds) or when the status changes (in the case of a failed power supply).

Another log message is generated as the sensor value moves back to a normal value and becomes equal to the threshold value. In the above command, no minimum or maximum threshold values are specified so cOS Stream will use the default thresholds for this sensor. All sensors have default thresholds already defined. Note that the SensorID property must always be specified when creating an HWMONMonitor object.

By default, a severity of Warning is used for all log messages generated. The severity can be explicitly set by changing the Severity property of an HWMONMonitor object. For example, to set the severity to Emergency for the mon1 object, the CLI command would be:

System:/> set HWMONMonitor mon1 Severity=Emergency

Sensor Types

Sensors are of two types:

Understanding the thresholds of a numeric sensor is straightforward but binary sensors require some additional explanation. Binary sensors have a value which is either 1 or 0. However, it depends on the sensor which value indicates a "normal" condition. For example, one sensor could indicate that a PSU is installed and its normal value is 1 (the PSU is present). Another sensor might indicate that a PSU has failed and its normal value is 0 (the PSU is operational and has not failed).

The following should be noted about HWMONMonitor object thresholds for binary sensors:

Changing Threshold Values

The lower and upper thresholds for an HWMONMonitor object associated with a numeric sensor can be changed from the default, as shown in the following example:

System:/> set HWMONMonitor mon1 LowThresh=-5 HighThresh=100

Note that negative values can be used for temperatures.

Changing the Monitoring Interval for the HWMONMonitor Object

The Interval property of an HWMONMonitor object decides how often, in seconds, the data from the object's associated sensor is examined. A higher value for this property will cause less log messages to be sent. For example, to examine the CPU temperature every 30 seconds, the CLI command would be as follows:

System:/> set HWMONMonitor mon1 Interval=30

Note that this interval does not affect the rate at which the sensors are polled, which is controlled by the global setting SensorRefreshInterval (described at the beginning of this topic). The Interval property only affects the frequency with which the currently stored sensor values are examined by the HWMONMonitor object to determine if a log message should be generated.

Log Messages

A message is generated when the sensor value passes outside the thresholds specified. A second log message is generated when the sensor value passes back inside the thresholds specified. Similarly, for a binary sensor, a log message is generated when the state of the sensor changes in either direction.

The log messages generated by HWMONMonitor objects all belong to the HWMON log message category. As described previously, the severity of the message is Warning by default but this can be changed by setting the Severity property of the HWMONMonitor object generating the message. Below are some examples of the log messages created by user hardware monitoring.

A sensor value above a maximum threshold:

SYSTEM,HWMON: prio=warning id=1081
event=sensor_value_above_monitor_threshold sensorid=”CPU_Temp”
description="CPU Temperature" name="n2" value=81
threshold=80 action=none

A sensor value below a minimum threshold:

SYSTEM,HWMON: prio=warning id=1079
event=sensor_value_below_monitor_threshold sensorid=”CPU_Temp”
description="CPU Temperature" name="n2" value=29
threshold=30 action=none

A sensor value crossing a threshold back to a normal value:

SYSTEM,HWMON: prio=warning id=1082
event=sensor_returned_to_normal sensorid=”CPU_Temp”
description="CPU Temperature" name="n2" value=80
action=none

The Log Repeat Interval

While the sensor value remains outside of its thresholds, a HWMONMonitor object will regenerate the log message, by default, every 6 hours so that the administrator continues to be reminded that the abnormal condition exists.

The default log repeat interval value of 6 hours can be changed by assigning a new value to the LogRepeatInterval property of the HWMONMonitor object. The value is specified as an integer number of seconds. For example, the following command would change the log repeat interval for the monitor called mon1 to become 12 hours:

System:/> Set HWMONMOnitor mon1 LogRepeatInterval=43200

Note that the minimum allowed value for the LogRepeatInterval property is 30 seconds and the maximum is 86,400 seconds (24 hours).

Displaying the Available Sensors

To see all the current available sensors, the hwmon -sensorlist command can be used. The following shows some typical output (the last two right hand columns showing lowest and highest have been truncated to fit the page width):

System:/> hwmon -sensorlist
		
Sensor ID       Description               Unit  Value  Monitor Min    Max
--------------  ------------------------  ----  -----  ------- -----  -----
CPU_Temp        CPU Temperature           C     58     yes     0      98    
System_Temp     System Temperature        C     24     no      0      55    
System_Power    System Power Consumption  W     224    no      0      500   
System_12V      System Internal Power     mV    12126  no      11500  13000 
FAN1_RPM        System FAN1 Speed         RPM   4200   no      1500   6800   
FAN2_RPM        System FAN2 Speed         RPM   4000   no      1500   6800  
FAN3_RPM        System FAN3 Speed         RPM   4100   no      1500   6800  
PSU1_Avail      PSU1 Available            -     1      no      1      1     
PSU1_Fail       PSU1 Failure Detected     -     0      no      0      0    
PSU1_Input_Lost PSU1 Power Input Lost     -     0      no      0      0     
PSU2_Avail      PSU2 Available            -     1      no      1      1   
PSU2_Fail       PSU2 Failure Detected     -     0      no      0      0   
PSU2_Input_Lost PSU2 Power Input Lost     -     0      no      0      0

The following should be noted about the above output:

Displaying the Current HWMONMonitor List

To see all the current HWMONMonitor objects, the hwmon is used with no parameters. The following shows some typical output for two configured HWMONMonitor objects called mon1 and mon2 that are monitoring CPU temperature:

System:/> hwmon
		
Name  Sensor    Description       Value  Low  High  Status  #Low  #High
----  --------  ----------------  -----  ---  ----  ------  ----  -----
mon1  CPU_Temp  CPU Temperature   59     0    98    NORMAL  0     0
mon2  CPU_Temp  CPU Temperature   59     20   70    NORMAL  0     0

Here, the Low and High columns are the currently defined thresholds for these HWMONMonitor objects. The #Low and #High columns are the total number of alarms triggered when the low and high thresholds are passed. Alarms are explained later in this section.

Note that in the above list, there are two HWMONMonitor objects defined for the same sensor. This is permissible and means that separate log messages can be generated for different sensor ranges.

The Status column in the above output indicates the status of the sensor value when it was last examined by the HWMONMonitor object and can show the following values:

Displaying HWMONMonitor Properties

To see all the properties for an HWMONMonitor object, the hwmon <monitor-name> command can be used. For example, to see all the values for the HWMONMonitor called mon1, the command would be:

System:/> hwmon mon1
Name                       : mon1
Sensor id                  : CPU_Temp
Log severity setting       : warning
Description                : CPU Temperature
Value                      : 58
Low Threshold              : 0
Recommended low threshold  : 0
High Threshold             : 98
Recommended high threshold : 98
Action interval (seconds)  : 5
ALARM:LOW counter          : 1
ALARM:HIGH counter         : 3
Last status                : NORMAL

Note that the Recommended low threshold and the Recommended high threshold in the above output are the default threshold values for the associated sensor.

Sensor Alarms

As shown in the output above, each HWMONMonitor object has two alarm counters associated with it, the ALARM:LOW counter and the ALARM:HIGH counter. These properties of the object can only be read and cannot be manipulated by the administrator.

The following should be noted about the alarm counters:

The sensor information that is gathered by cOS Stream is available to an SNMP client using the SNMP protocol. This type of monitoring is known as System Monitoring.

System monitoring only has one option available that can be set by the administrator, and that is if it is enabled or disabled. All hardware monitoring is enabled by default, but this can be disabled by the following CLI command:

System:/> set Settings HWMONSettings MonitorEnable=No

This will disable both system and user monitoring. Both can be re-enabled with the following command:

System:/> set Settings HWMONSettings MonitorEnable=Yes

This setting affects both system monitoring and user monitoring

The following are the available sensors for the current range of Clavister hardware products running cOS Stream. All fan speeds are given in RPM and temperatures are in degrees centigrade.