|
|
| Subject: | Pending Jobs for Discovery Never Go Active |
| Author: | [Not Specified] |
| Posted: | 2014-08-06 10:09 |
As part of product evaluation we installed Zenoss Core using autodeply on Centos 6.5 x64. The initial network discovery worked extremely well, so we moved on to adding MIBs and additional ZenPacks (for Procurve switches and LLDP modelling). We wanted to move it into a more production like environment and redo the discovery and modelling, so we deleted all of the existing Devices. Now when we try to start the Discovery again all the jobs stay Pending.
As far as I can see, rabbitmq and all the zenoss related services are running.
Troubleshooting steps thus far:
1. Restarted entire server.
2. Checked logs. No errors that specifically point to a job or queue failure, but these entries may be interesting:
event.log:2014-08-06T09:27:50 INFO zen.JobManager Created job
/opt/zenoss/log/jobs/bbedad39-8026-4d41-bbc2-ddcc8231190d.log:
2014-08-06 10:01:48,332 INFO zen.Job: Job bbedad39-8026-4d41-bbc2-ddcc8231190d (Products.Jobb
er.jobs.SubprocessJob) received
2014-08-06 10:01:48,332 DEBUG zen.Job: Waiting for job bbedad39-8026-4d41-bbc2-ddcc8231190d t
o appear in database
The last update to zenjobs.log was at 07:00, roughly three hours prior.
Could it possibly be related to this:
2014-08-06 10:33:13,881 INFO zen.ZenHub: Worker (4252) reports 2014-08-06 10:33:13,880 CRITICAL zen.Events: Unable to publish event to
Thanks,
Simon
| Subject: | Check filesystems utilization |
| Author: | Jan Garaj |
| Posted: | 2014-08-10 05:32 |
Check filesystems utilization (df -ha). RabbitMQ denies any connections, when its filesystem is almost full (10% is default threshold).
And then check RabbitMQ queues http://wiki.zenoss.org/Working_with_Queues
Devops Monitoring Expert advice:
Dockerize/automate/monitor all the things.
DevOps stack:
Docker / Kubernetes / Mesos / Zabbix / Zenoss / Grafana / Puppet / Ansible / Vagrant / Terraform /
Elasticsearch
| Subject: | Bonded interfaces? |
| Author: | [Not Specified] |
| Posted: | 2014-08-11 06:20 |
It is working now and it may be related to the bonded interface that I added *post* installation. Once I removed the bonded interface and the slaves it started working again. Does the connection to RabbitMQ use the loopback/localhost/127.0.0.1 or is it bound to an interface IP or host name lookup
| Subject: | Simon, |
| Author: | Andrew Kirch |
| Posted: | 2014-08-11 12:05 |
Simon,
One of our community members has this issue on occasion and it seems to be related to a stuck zenjobs. The next time this happens please restart zenjobs and let me know if the issues resolves.
Andrew Kirch
akirch@gvit.com
Need Zenoss support, consulting or custom development Look no further. Email or PM me!
Ready for Distributed Topology (collectors) for Zenoss 5 Coming May 1st from GoVanguard
| Subject: | Andrew, |
| Author: | [Not Specified] |
| Posted: | 2014-08-11 14:31 |
Andrew,
The job queue was empty (or at least appeared to be so). Also, restarting zenjobs and zenhub did not seem to get the job to move to pending. I should have also updated the original post with the log entries showing disconnected rabbitMQ sessions. As soon as I removed the bonded interfaces and restarted, the jobs worked fine.
I'll see if I can re-create the environment once the machine reaches its destination and repost the results.
Thanks,
Simon
| Subject: | zenjobs hanging Zenoss Core 4 - Cannot add new devices |
| Author: | Ken Jenkins |
| Posted: | 2016-04-08 19:39 |
Encountering zenjobs in Pending on Core 4.
I am looking for suggestions. I cannot add new devices.
Thanks,
Ken
| Subject: | Jenkinsk007, |
| Author: | [Not Specified] |
| Posted: | 2016-04-21 01:23 |
Jenkinsk007,
Let's see if we can find some more symptoms. Could you clear all the jobs, stop zenhub and zenjobs then start zenhub and zenjobs from the cli in this manner:
zenhub start -v10
zenjobs start -v10
After that submit a handful of jobs and have a look at /opt/zenoss/log/zenhub.log and /opt/zenoss/log/zenjobs.log. If you see anything out of the ordinary post it in a reply and we'll keep digging with the new lead.
Thanks man!
ZenMaster Shane William Scott (Hackman238)
CTO
GoVanguard Inc.
sscott@gvit.com
Need Zenoss support, consulting or custom development Look no further. Email or PM me!
Ready for Distributed Topology (collectors) for Zenoss 5 Coming May 1st from GoVanguard
| < |
Previous Set thresolds for VMS |
Next Need help to figure out the thresolds for cloud stack zenpack? |
> |