TECHZEN Zenoss User Community ARCHIVE  

Monitoring Template: events automatically cleared the moment they are created

Subject: Monitoring Template: events automatically cleared the moment they are created
Author: [Not Specified]
Posted: 2015-08-06 05:03

Hi, I'd like to forward a question from my IT operations director:

We have exceeded threshold events (> maximum threshold) that get cleared in the same second they are created. (firstTime, lastTime and stateChange are all the same timestamp).
There is no clearid, and count is 1.
However escalation_count is being incremented for every (similar) event, as after the event is cleared, in the next cycle, its happening again, and a new event is created & cleared.

There is no record (log) who or what changed the state; it happens (unwanted!) automagically
(Im referring to the records in the log section of the event like pete 2015-08-05 04:05:48: state changed to Acknowledged)

However not met threshold (< minimum threshold) events are not automagically clearedand both the count and escalation_count are incremented.

Agent: zenperfsql

Any clue why this happens
How to prevent Zenoss from doing this

Thanks in advance,
Robert



Subject: Check your transforms, could
Author: Jay Stanley
Posted: 2015-08-06 07:36

Check your transforms, could be something in there



Subject: Thanks, but I'm afraid we don
Author: [Not Specified]
Posted: 2015-08-06 22:08

Thanks, but I'm afraid we don't have any transforms under the problematic EventClass :(



Subject: Solved.
Author: [Not Specified]
Posted: 2015-08-10 23:51

Solved.
He had zEventAction set to "history" instead of "status" in this particular Event Class.



Subject: you may also want to turn on
Author: Andrew Kirch
Posted: 2015-08-11 09:16

you may also want to turn on Event Flap Detection in case this happens again. We do have some code to help identify and event on flapping events that you might not otherwise see. http://wiki.zenoss.org/Detecting_Event_Flaps

Andrew Kirch

akirch@gvit.com

Need Zenoss support, consulting or custom development Look no further. Email or PM me!

Ready for Distributed Topology (collectors) for Zenoss 5 Coming May 1st from GoVanguard



Subject: Events generated by traps will not clear from event console
Author: [Not Specified]
Posted: 2015-11-18 15:05

I'm having a similar issue to this one in 4.2.5 trap with OID ending in .312 = problem trap with OID ending in .313 = problem cleared. I wrote a transform to change the dedupid to 'evt.device|evt.eventClassKey|evt.severity|component name pulled out of the message with a regex grouping' The .312 events come in, the fields the transform is supposed to update seem to be working. The fingerprint matches my expectation. Exactly 3 seconds later, the event is updated by unknown and the message is "state changed to Cleared". There is a clearid for the event, and the eventState shows cleared. Unfortunately, I cannot close the event. The event severity remains at 4, and the count is 1. As best I can tell, the .313 trap has not come in and cleared the event. Transform as exists on /Unknown:

big_ip_device = re.search(r'BIG-IP',str(evt.DeviceClass))

if big_ip_device:

evt.eventClass = '/BIG-IP'

if evt.message.startswith('snmp trap'):

#Pool Resource Related - 1.3.6.1.4.1.3375.2.4.0.312 & 1.3.6.1.4.1.3375.2.4.0.313

pool_resources = re.search(r'1[.]3[.]6[.]1[.]4[.]1[.]3375[.]2[.]4[.]0[.]31[23]',getattr(evt,"oid"))

if pool_resources:

evt.summary = str(getattr(evt,"bigipNotifyObjMsg"))

pool_name = re.search(r'[Pp]ool /(\S+)',getattr(evt,"bigipNotifyObjMsg"))

evt.eventClassKey = 'BIG-IP_Pool'

pool_drained = re.search(r'1[.]3[.]6[.]1[.]4[.]1[.]3375[.]2[.]4[.]0[.]312',getattr(evt,"oid"))

if pool_drained:

evt.severity = 4

pool_avail = re.search(r'1[.]3[.]6[.]1[.]4[.]1[.]3375[.]2[.]4[.]0[.]313',getattr(evt,"oid"))

if pool_avail:

evt.severity = 0

# Now that severity is set for the fingerprint, join and set the thumbprint

dedupfields = [evt.device, evt.eventClassKey, evt.severity, pool_name.group(1)]

mydedupid = '|'.join(map(str, dedupfields))

evt.dedupid = mydedupid

if evt.severity == 0:

evt._action = 'history'

#Virtual Socket Related - 1.3.6.1.4.1.3375.2.4.0.135 & 1.3.6.1.4.1.3375.2.4.0.136

# Info only traps for now, not requested

virtual_socket = re.search(r'1[.]3[.]6[.]1[.]4[.]1[.]3375[.]2[.]4[.]0[.]13[56]',getattr(evt,"oid"))

if virtual_socket:

evt.summary = re.sub(r'SNMP_TRAP: ','',getattr(evt,"bigipNotifyObjMsg"))

socket_name = re.search(r'Virtual (\S+) has',getattr(evt,"bigipNotifyObjMsg"))

evt.eventClassKey = 'BIG-IP_Virtual_Socket'

socket_unavail = re.search(r'1[.]3[.]6[.]1[.]4[.]1[.]3375[.]2[.]4[.]0[.]136',getattr(evt,"oid"))

if socket_unavail:

evt.severity = 2

socket_available = re.search(r'1[.]3[.]6[.]1[.]4[.]1[.]3375[.]2[.]4[.]0[.]135',getattr(evt,"oid"))

if socket_available:

evt.severity = 0

# Now that severity is set for the fingerprint, join and set the thumbprint

dedupfields = [evt.device, evt.eventClassKey, evt.severity, socket_name.group(1)]

mydedupid = '|'.join(map(str, dedupfields))

evt.dedupid = mydedupid

if evt.severity == 0:

evt._action = 'history'



< Previous
How do I get value from CompositeData in ZenJMX
  Next
Monitoring Template: Data Source ?Cycle Time? not followed
>