TECHZEN Zenoss User Community ARCHIVE  

Rabbitmq queues backingup

Subject: Rabbitmq queues backingup
Author: Devon Solomon
Posted: 2017-11-01 15:06

We are running Zenoss core 5.3.0 and our rabbitmq queues are getting backed up. Nothing is being processed out of rwaevents queue. 

[root@rbt0 /]# rabbitmqctl list_queues -p /zenoss
Listing queues ...
4812b399e658.celery.pidbox 0
aliveness-test 0
celery 0
zenoss.queues.zep.heartbeats 0
zenoss.queues.zep.migrated.archive 0
zenoss.queues.zep.migrated.summary 0
zenoss.queues.zep.modelchange 0
zenoss.queues.zep.rawevents 81315
zenoss.queues.zep.signal 0
zenoss.queues.zep.zenevents 0

I have purged them, dumped and answered them, restarted RM and rabbitmq. Also went through the How to Recover Control Center from Hardware Failure  guide in an attempt to get things back on track but no luck. what else can i do to get events to be processed and rabbit not backed up. 
Zenoss remove preview
How to Recover Control Center from Hardware Failure
Applies To Zenoss 5.x Summary Hardware failure in Control Center can take various forms, including: Running out of disk space on one or more of the partitions that store Control Center, Docker o...
View this on Zenoss >

Devon Solomon

Subject: RE: Rabbitmq queues backingup
Author: Jane Curry
Posted: 2017-11-02 04:49

Check zeneventd - that's the daemon that takes events off the rawevents queue, processes them and then puts them on the zenevents queue to be processed by zeneventserver.  There's nothing in your zenevents queue so zeneventd must be the culprit. 

Is it running???
If so, turn up logging in the zeneventd container with "zeneventd debug" and see what is going on in $ZENHOME/logs/zeneventd.log.

Have you changed any transforms recently?  They are processed by zeneventd and a really injudicious transform could completely block the pipeline.


Jane Curry
Skills 1st United Kingdom

< Previous
Zenoss 5 core download and install
Email Alert/Notification not working