Home  Prev  cOS Stream 4.00.03 Log Reference Guide  Next

2.19. HA

These log messages refer to the HA category.

2.19.1. [ID: 259] HA sync message reassembly failed

Log Categories
HA
Log Message
HA sync message reassembly failed.
Default Log Severity
Warning
Parameters
 
Explanation
Some of the fragments matching this message reassembly were invalid or unexpected resulting in the whole message being dropped.
Gateway Action
Drop
Action Description
None
Proposed Action
Search for logs of invalid or unexpected HA sync message fragment and take action accordingly.

2.19.2. [ID: 398] HA sync message reassembly failed due to lack[...]

Log Categories
HA
Log Message
HA sync message reassembly failed due to lack of resources.
Default Log Severity
Notice
Parameters
 
Explanation
The reassembly of a HA sync message could not be initiated or was prematurely aborted due to lack of resources.
Gateway Action
Drop
Action Description
None
Proposed Action
If this happens frequently try increasing the setting HASyncFragSettings:MaxConcurrentReass.

2.19.3. [ID: 597] HA sync message reassembly failed due to[...]

Log Categories
HA
Log Message
HA sync message reassembly failed due to timeout.
Default Log Severity
Notice
Parameters
 
Explanation
Not all fragments of a HA sync message was received before the reassembly was aborted due to timeout.
Gateway Action
Drop
Action Description
None
Proposed Action
This event can be the result of packet loss on the HA sync interfaces. If it happens infrequently it should not be a problem but if it happens more frequently then it might be a good idea to check if the sync interfaces are fully functional. It could also indicate that the reassembler's configuration is not suitable for the current load so another possible action is to review the HASyncFragSettings and increase the number of concurrent reassemblies or adjust the timing values.

2.19.4. [ID: 364] Received invalid HA sync message fragment

Log Categories
HA
Log Message
Received invalid HA sync message fragment.
Default Log Severity
Warning
Parameters
 
Explanation
A packet that is not a valid HA sync message fragment was received.
Gateway Action
Drop
Action Description
None
Proposed Action
Verify that the HA sync interfaces are properly secured. Untrusted systems should not be able to inject packets on the sync interfaces. Other potential causes are software or hardware malfunction.

2.19.5. [ID: 609] Received unexpected HA sync message fragment

Log Categories
HA
Log Message
Received unexpected HA sync message fragment.
Default Log Severity
Warning
Parameters
 
Explanation
The fragment did not fit the partially reassembled message that it matched.
Gateway Action
Drop
Action Description
None
Proposed Action
If this happens frequently then it may be a sign that the load on the reassembler is too high or that reassembly timeouts are too low. Review the HASyncFragSettings and increase the number of concurrent reassemblies or adjust the timing values.

2.19.6. [ID: 1044] Active-active scenario detected

Log Categories
HA,SYSTEM
Log Message
Active-active scenario detected. Remaining active since role is master.
Default Log Severity
Notice
Parameters
 
Explanation
The cluster have ended up in a scenario where both members wants to be active. In these situations the master will remain active while the slave go inactive.
Gateway Action
None
Action Description
None
Proposed Action
This is a erroneous scenario but, unless occurring frequently, can occur normally under special conditions.

2.19.7. [ID: 1047] Active-active scenario detected

Log Categories
HA,SYSTEM
Log Message
Active-active scenario detected. Going inactive since role is slave.
Default Log Severity
Notice
Parameters
 
Explanation
The cluster have ended up in a scenario where both members wants to be active. In these situations the master will remain active while the slave go inactive.
Gateway Action
Failover
Action Description
None
Proposed Action
This is a erroneous scenario but, unless occurring frequently, can occur normally under special conditions.

2.19.8. [ID: 1049] Config in sync

Log Categories
HA,SYSTEM
Log Message
Config in sync.
Default Log Severity
Notice
Parameters
localcfgver, remotecfgver
Explanation
Config versions match in both nodes of the HA cluster.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.9. [ID: 1051] Config not in sync

Log Categories
HA,SYSTEM
Log Message
Config not in sync.
Default Log Severity
Warning
Parameters
localcfgver, remotecfgver
Explanation
Config changes are done but not synced to the other node.
Gateway Action
None
Action Description
Config is not the same between the nodes in the HA cluster
Proposed Action
Set AutoSyncCfg in HighAvailability to True or send/receive configure on one node to make sure both nodes are running the same configuration.

2.19.10. [ID: 580] Failed to establish sync connection

Log Categories
HA,SYSTEM
Log Message
Failed to establish sync connection.
Default Log Severity
Error
Parameters
 
Explanation
The handshake to establish a new sync connection failed. A new handshake will be initiated.
Gateway Action
None
Action Description
None
Proposed Action
If the cluster fails to recover from this state on its own consider rebooting the inactive node.

2.19.11. [ID: 1736] No contact with monitored target

Log Categories
HA,IFACE
Log Message
No contact with monitored target.
Default Log Severity
Information
Parameters
targetip, physiface, targetiface
Explanation
This HA node was not able to establish bi-directional contact with the monitor target targetip, over targetiface. This log never shows up unless the HA cluster has failed to establish bidirectional heart-beat connectivity over the underlying physiface.
Gateway Action
None
Action Description
None
Proposed Action
Check interface cabling and load and verify proper operation. If the interface pair has been intentionally left unconnected then it is possible set the HAType property on the interfaces to NonCritical to stop the log from triggering, however, it is probably desirable to restore the HAType to Critical when the interface is taken into service again to get normal monitoring of the interface.

2.19.12. [ID: 546] Failure indication cleared

Log Categories
HA,SYSTEM
Log Message
Failure indication cleared.
Default Log Severity
Notice
Parameters
id, failure
Explanation
A previously indicated failure was reported to be resolved.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.13. [ID: 1726] Monitored target reachable

Log Categories
HA,IFACE
Log Message
Monitored target reachable.
Default Log Severity
Information
Parameters
targetip, physiface, targetiface
Explanation
The HA nodes were unable to successfully communicate through the heart-beats sent over the specific interface. This HA node was still able to establish bi-directional contact with the monitor target targetip, over targetiface, so network access for physiface is considered functional.
Gateway Action
None
Action Description
None
Proposed Action
Check interface cabling and load and verify proper operation. If the interface pair has been intentionally left unconnected then it is possible set the HAType property on the interfaces to NonCritical to stop the log from triggering, however, it is probably desirable to restore the HAType to Critical when the interface is taken into service again to get normal monitoring of the interface.

2.19.14. [ID: 605] Scheduling HA initiated system restart

Log Categories
HA,SYSTEM
Log Message
Scheduling HA initiated system restart.
Default Log Severity
Critical
Parameters
node, reason
Explanation
The HA monitor system detected a failure and initiated a system restart of the inactive HA member with the purpose of restoring system functionality.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.15. [ID: 281] Failure indication set

Log Categories
HA,SYSTEM
Log Message
Failure indication set.
Default Log Severity
Notice
Parameters
id, failure
Explanation
A failure was reported to the HA system which may result in a HA hand-over.
Gateway Action
None
Action Description
None
Proposed Action
Review the listed failure parameter and check for related logs to determine the cause of the malfunction.

2.19.16. [ID: 237] Going HA ACTIVE since HA peer is dead

Log Categories
HA,SYSTEM
Log Message
Going HA ACTIVE since HA peer is dead.
Default Log Severity
Alert
SNMP Trap Category
HA
SNMP Trap MIB name
ssmHAActivePeerDead
SNMP Trap MIB OID
1.3.6.1.4.1.5089.3.0.2010.0.1005   (STREAM-TRAPS-MIB)
Parameters
node
Explanation
The previously active HA peer became offline and the inactive node took over in a failover.
Gateway Action
Failover
Action Description
None
Proposed Action
If this was an unplanned event, check the logs for hints on why the peer node became offline.

2.19.17. [ID: 317] Going HA ACTIVE since outranking peer

Log Categories
HA,SYSTEM
Log Message
Going HA ACTIVE since outranking peer.
Default Log Severity
Alert
SNMP Trap Category
HA
SNMP Trap MIB name
ssmHAActiveOutrankingPeer
SNMP Trap MIB OID
1.3.6.1.4.1.5089.3.0.2010.0.1010   (STREAM-TRAPS-MIB)
Parameters
node, decider, localcriteria, peercriteria
Explanation
The previously active HA peer had some sort of malfunction and the inactive node took over.
Gateway Action
Failover
Action Description
None
Proposed Action
If this was an unplanned event, check the logs for hints on what type of malfunction occurred on the peer node.

2.19.18. [ID: 275] Going HA ACTIVE due to user request

Log Categories
HA,SYSTEM
Log Message
Going HA ACTIVE due to user request.
Default Log Severity
Notice
SNMP Trap Category
HA
SNMP Trap MIB name
ssmHAActiveUserRequest
SNMP Trap MIB OID
1.3.6.1.4.1.5089.3.0.2010.0.1015   (STREAM-TRAPS-MIB)
Parameters
node
Explanation
The system administrator triggered a hand-over and the node became HA active.
Gateway Action
Failover
Action Description
None
Proposed Action
None

2.19.19. [ID: 130] Going HA INACTIVE due to being outranked by[...]

Log Categories
HA,SYSTEM
Log Message
Going HA INACTIVE due to being outranked by peer.
Default Log Severity
Alert
SNMP Trap Category
HA
SNMP Trap MIB name
ssmHAInactiveOutrankedByPeer
SNMP Trap MIB OID
1.3.6.1.4.1.5089.3.0.2010.0.1020   (STREAM-TRAPS-MIB)
Parameters
node, failure, decider, localcriteria, peercriteria
Explanation
A hand-over was performed due to the peer node being deemed to be better fit to be active.
Gateway Action
Failover
Action Description
None
Proposed Action
Review the listed failures and check for related logs to determine the cause of the hand-over.

2.19.20. [ID: 146] Going HA INACTIVE due to user request

Log Categories
HA,SYSTEM
Log Message
Going HA INACTIVE due to user request.
Default Log Severity
Notice
SNMP Trap Category
HA
SNMP Trap MIB name
ssmHAInactiveUserRequest
SNMP Trap MIB OID
1.3.6.1.4.1.5089.3.0.2010.0.1025   (STREAM-TRAPS-MIB)
Parameters
node
Explanation
The system administrator triggered a hand-over and the node became HA inactive.
Gateway Action
Failover
Action Description
None
Proposed Action
None

2.19.21. [ID: 663] HA bidir heart-beat communication over[...]

Log Categories
HA,IFACE
Log Message
HA bidir heart-beat communication over interface failed.
Default Log Severity
Notice
Parameters
physiface
Explanation
The HA nodes were unable to successfully communicate through the heart-beats sent over the specific interface. This communication is used to monitor the health of the peer node and is the primary mean of monitoring the health of the interfaces.
Gateway Action
None
Action Description
None
Proposed Action
Check interface cabling and load and verify proper operation. If the interface pair has been intentionally left unconnected then it is possible set the HAType property on the interfaces to NonCritical to stop the log from triggering, however, it is probably desirable to restore the HAType to Critical when the interface is taken into service again to get normal monitoring of the interface.

2.19.22. [ID: 177] HA interface offline

Log Categories
HA,SYSTEM
Log Message
HA interface offline.
Default Log Severity
Alert
Parameters
iface
Explanation
An interface became offline due to loss of connectivity.
Gateway Action
None
Action Description
None
Proposed Action
Check interface cabling and verify proper operation.

2.19.23. [ID: 475] HA interface online

Log Categories
HA,SYSTEM
Log Message
HA interface online.
Default Log Severity
Alert
Parameters
iface
Explanation
An interface which was previously offline became online.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.24. [ID: 1046] Inactive-inactive situation detected

Log Categories
HA,SYSTEM
Log Message
Inactive-inactive situation detected. Going active since role is master.
Default Log Severity
Notice
Parameters
 
Explanation
The cluster has ended up in a scenario where both members wants to be inactive. In these situations the master will go active while the slave remains inactive.
Gateway Action
Failover
Action Description
None
Proposed Action
This is an erroneous scenario but, unless occurring frequently, can occur normally under special conditions.

2.19.25. [ID: 1045] Inactive-inactive situation detected

Log Categories
HA,SYSTEM
Log Message
Inactive-inactive situation detected. Remaining inactive since role is slave.
Default Log Severity
Notice
Parameters
 
Explanation
The cluster has ended up in a scenario where both members wants to be inactive. In these situations the master will go active while the slave remains inactive.
Gateway Action
None
Action Description
None
Proposed Action
This is an erroneous scenario but, unless occurring frequently, can occur normally under special conditions.

2.19.26. [ID: 629] Scheduling HA initiated system restart to[...]

Log Categories
HA,SYSTEM
Log Message
Scheduling HA initiated system restart to resynchronize.
Default Log Severity
Warning
Parameters
node
Explanation
The HA system is restarting the inactive HA member in order to prepare the member to receive new sync data.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.27. [ID: 1509] No matching HA interface id found during HA[...]

Log Categories
HA,IFACE,SYSTEM
Log Message
No matching HA interface id found during HA Peer MAC learning.
Default Log Severity
Warning
Parameters
hwaddr
Explanation
When exchanging hardware addresses with HA peer, the node received data on one interface that could not be matched to existing interfaces.
Gateway Action
None
Action Description
None
Proposed Action
Verify if both HA nodes have the same number of interfaces and that their HA Types match. Also verify that the sync interface is not on a shared network with other HA clusters.

2.19.28. [ID: 378] HA peer offline

Log Categories
HA,SYSTEM
Log Message
HA peer offline.
Default Log Severity
Notice
Parameters
 
Explanation
The previously online HA peer is now offline.
Gateway Action
None
Action Description
None
Proposed Action
If this was an unplanned event, check the logs for hints on why the peer node became offline.

2.19.29. [ID: 403] HA peer have an incompatible HA version

Log Categories
HA,SYSTEM
Log Message
HA peer have an incompatible HA version.
Default Log Severity
Alert
Parameters
version, min
Explanation
Contact was established with a HA peer with an incompatible HA implementation.
Gateway Action
Abort
Action Description
HA synchronization will not be performed
Proposed Action
Make sure the HA peers run core binaries with the same version. All flows/states will be lost when upgrading.

2.19.30. [ID: 1510] Invalid peer MAC received during HA Peer MAC[...]

Log Categories
HA,IFACE,SYSTEM
Log Message
Invalid peer MAC received during HA Peer MAC learning.
Default Log Severity
Warning
Parameters
newhw, iface
Explanation
When exchanging hardware addresses with HA peer, the node received one invalid hardware address from one of the peer configured interfaces.
Gateway Action
None
Action Description
None
Proposed Action
Verify if the configuration is correct, and that interfaces using HA over Interface MAC are correctly paired.

2.19.31. [ID: 1507] HA Peer MAC learning successful

Log Categories
HA,IFACE,SYSTEM
Log Message
HA Peer MAC learning successful.
Default Log Severity
Information
Parameters
learnt
Explanation
The node received a packet containing all peer interfaces hardware address and updated values where needed.
Gateway Action
None
Action Description
None
Proposed Action
If automatic peer discovery is not intended, then the peer MAC addresses should be set using property EthernetAddress on the EthernetInterface.

2.19.32. [ID: 1508] HA Peer MAC learning incomplete

Log Categories
HA,IFACE,SYSTEM
Log Message
HA Peer MAC learning incomplete.
Default Log Severity
Warning
Parameters
learnt
Explanation
The node received a packet containing all peer interfaces hardware address but some interfaces contained errors. These errors were separately logged.
Gateway Action
None
Action Description
None
Proposed Action
If automatic peer discovery is not intended, then the peer MAC addresses should be set when configuring the interface, using property 'EthernetAddress'. Also verify if both HA nodes have matching interface HA Types.

2.19.33. [ID: 114] HA peer online

Log Categories
HA,SYSTEM
Log Message
HA peer online.
Default Log Severity
Notice
Parameters
 
Explanation
The previously offline HA peer is now online.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.34. [ID: 630] Dataplane shutting down

Log Categories
HA,SYSTEM
Log Message
Dataplane shutting down.
Default Log Severity
Notice
Parameters
 
Explanation
Dataplane is shutting down.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.35. [ID: 808] Main resynchronization aborted

Log Categories
HA,SYSTEM
Log Message
Main resynchronization aborted.
Default Log Severity
Notice
Parameters
node, state, count
Explanation
The resynchronization of all HA state from the active node to the inactive node was aborted. Not all HA state was synchronized. One of the nodes must be restarted to begin a new full resynchronization.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.36. [ID: 323] Main resynchronization done

Log Categories
HA,SYSTEM
Log Message
Main resynchronization done.
Default Log Severity
Information
Parameters
node, state, count
Explanation
Resynchronization of all HA state from the active node to the inactive node has finished. The inactive node now has sufficient HA state information to take over if the active node fails.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.37. [ID: 285] Commencing main resynchronization

Log Categories
HA,SYSTEM
Log Message
Commencing main resynchronization.
Default Log Severity
Information
Parameters
node, state, count
Explanation
Resynchronization of all HA state from the active node to the inactive node has started. Once an inactive node is booted up, it has no HA state information. The active node then starts to send all HA state information to its inactive peer.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.38. [ID: 206] Sync connection established

Log Categories
HA,SYSTEM
Log Message
Sync connection established.
Default Log Severity
Information
Parameters
id
Explanation
The two HA peer nodes have successfully completed a handshake over the sync interface(s) and are now ready for state synchronization.
Gateway Action
None
Action Description
None
Proposed Action
None

2.19.39. [ID: 436] Sync connection failed

Log Categories
HA,SYSTEM
Log Message
Sync connection failed.
Default Log Severity
Critical
Parameters
 
Explanation
The device was unable to communicate with the HA peer node over the sync interface(s). Until this condition is resolved no state will be synchronized from the active node to the inactive node.
Gateway Action
None
Action Description
None
Proposed Action
Urgently investigate why the communication is not working.

2.19.40. [ID: 1425] System versions not equal

Log Categories
HA,SYSTEM
Log Message
System versions not equal.
Default Log Severity
Warning
Parameters
localsystemver, remotesystemver
Explanation
Config changes are done but not synced to the other node.
Gateway Action
None
Action Description
System versions are not the same in the HA cluster nodes
Proposed Action
Update the nodes in the HighAvailability cluster to the same system version.

2.19.41. [ID: 636] All flows closed due to HA activation or[...]

Log Categories
HA,SYSTEM,FLOW
Log Message
All flows closed due to HA activation or deactivation.
Default Log Severity
Warning
Parameters
state
Explanation
Enabling or disabling HA is a major configuration change. The event requires all flows to be closed.
Gateway Action
Close
Action Description
All flows are closed
Proposed Action
None

Home  Prev   Next