TECHZEN Zenoss User Community ARCHIVE  

Process not running: zencommand

Subject: Process not running: zencommand
Author: [Not Specified]
Posted: 2015-10-14 06:14

Hi everyone I have facing a little undesirable problem with zenoss mail alerts saying that zencommand is not running

message:
---------------------------------
Device: dbcontroler
Component: usr_bin_perl fba04951fda76b915671af901bb874b5
Severity: 4
Time: 2015/10/13 17:21:10.000
Message:
Process not running: zencommand
Using regex'.*zencommand.py.*--daemon.*'
All Processes have stopped since the last model occurred. Last Modification time (2015/10/13 05:22:41)
---------
but when I check to see if the process is really down using the same regular expression in the message, I can see the process its working.

[root@dbcontroler ~]# ps -ef|grep '.*zencommand.py.*--daemon.*'
root 3363 7043 0 13:04 pts/1 00:00:00 grep .*zencommand.py.*--daemon.*
zenoss 3675 1 1 11:39 00:00:55 /opt/zenoss/bin/python /opt/zenoss/Products/ZenRRD/zencommand.py --configfile /opt/zenoss/etc/zencommand.conf --cycle --daemon
[root@dbcontroler ~]#

I would like to know what is happening and how can I solve this problem.

Thanks in advanced for your help.
CrossX



Subject: Most (all I think) of our
Author: Andrew Kirch
Posted: 2015-10-14 11:34

Most (all I think) of our daemons have an internal watchdog that will restart it. Take a look at $ZENHOME/log/zencommand.log to determine if it might be crashing.

Andrew Kirch

akirch@gvit.com

Need Zenoss support, consulting or custom development Look no further. Email or PM me!

Ready for Distributed Topology (collectors) for Zenoss 5 Coming May 1st from GoVanguard



Subject: Checking logfile
Author: [Not Specified]
Posted: 2015-10-14 12:33

Checking the log $ZENHOME/log I've foud an error that correspond in time with the Alert (Process not running: zencommand) But I dont know if it could be the reason why the zencommand its stoping nither how to prevents that happend.
Find below the extract of $ZENHOME/log file:

2015-10-13 05:22:50,801 ERROR zen.RRDUtil: rrdtool reported error rrdcached: illegal attempt to update using time 1444706570.000000 when last up
date time is 1444706570.000000 (minimum one second step) Devices/oracdbsrv/fping_avg
2015-10-13 05:22:50,803 ERROR zen.RRDUtil: rrdtool reported error rrdcached: illegal attempt to update using time 1444706570.000000 when last up
date time is 1444706570.000000 (minimum one second step) Devices/oracdbsrv/fping_loss
2015-10-13 05:22:50,803 ERROR zen.RRDUtil: rrdtool reported error rrdcached: illegal attempt to update using time 1444706570.000000 when last up
date time is 1444706570.000000 (minimum one second step) Devices/oracdbsrv/fping_max
2015-10-13 05:22:50,804 ERROR zen.RRDUtil: rrdtool reported error rrdcached: illegal attempt to update using time 1444706570.000000 when last up
date time is 1444706570.000000 (minimum one second step) Devices/oracdbsrv/fping_min
2015-10-13 05:22:50,804 ERROR zen.RRDUtil: rrdtool reported error rrdcached: illegal attempt to update using time 1444706570.000000 when last up
date time is 1444706570.000000 (minimum one second step) Devices/oracdbsrv/fping_rcv
2015-10-13 05:22:50,805 ERROR zen.RRDUtil: rrdtool reported error rrdcached: illegal attempt to update using time 1444706570.000000 when last up
date time is 1444706570.000000 (minimum one second step) Devices/oracdbsrv/fping_xmt
2015-10-13 05:23:02,558 INFO zen.maintenance: Performing periodic maintenance
2015-10-13 05:23:02,559 INFO zen.zencommand: Counter eventCount, value 102795714
2015-10-13 05:23:02,563 INFO zen.zencommand: 254 devices processed (3224 datapoints)
2015-10-13 05:23:02,590 INFO zen.collector.scheduler: Tasks: 273 Successful_Runs: 324 Failed_Runs: 0 Missed_Runs: 0 Queued_Tasks: 0 Running_Task



Subject: I doubt that that's the cause
Author: Andrew Kirch
Posted: 2015-10-14 12:58

I doubt that that's the cause. You can crank up the debuggign by editing $ZENHOME/etc/zencommand.conf, or simply make a note of the PID and check to see if it changes on the next alert.

Andrew Kirch

akirch@gvit.com

Need Zenoss support, consulting or custom development Look no further. Email or PM me!

Ready for Distributed Topology (collectors) for Zenoss 5 Coming May 1st from GoVanguard



Subject: The process IID does not have changed and the alert is back!!!
Author: [Not Specified]
Posted: 2015-10-26 04:08

The process IID does not have changed and the alert is back, I dont know what could be happening.
Its really strange because the process is running since oct 14th and the alarms came up.
If you have any idea of what could be happenning please let me know.
thanks in advance
regards

----------------old-------
Last login: Thu Oct 15 09:38:14 2015 from 10.6.1.98
[root@dbcontroler ~]# cat /opt/zenoss/var/zencommand-localhost.pid
3675
[root@dbcontroler ~]#

[root@dbcontroler ~]# ps -ef|grep /zencommand
zenoss 3675 1 1 Oct14 03:21:24 /opt/zenoss/bin/python /opt/zenoss/Products/ZenRRD/zencommand.py --configfile

/opt/zenoss/etc/zencommand.conf --cycle --daemon
root 27416 22170 0 09:56 pts/0 00:00:00 grep /zencommand
[root@dbcontroler ~]#
----------------today---------
Last login: Mon Oct 19 10:15:33 2015 from 10.6.2.58
[root@dbcontroler ~]# cat /opt/zenoss/var/zencommand-localhost.pid
3675
[root@dbcontroler ~]#

[root@dbcontroler ~]# ps -ef|grep /zencommand
zenoss 3675 1 1 Oct14 03:21:24 /opt/zenoss/bin/python /opt/zenoss/Products/ZenRRD/zencommand.py --configfile

/opt/zenoss/etc/zencommand.conf --cycle --daemon
root 27416 22170 0 09:56 pts/0 00:00:00 grep /zencommand
[root@dbcontroler ~]#
------------------------
Device: dbcontroler
Component: usr_bin_perl fba04951fda76b915671af901bb874b5
Severity: 4
Time: 2015/10/24 11:43:57.000
Message:
Process not running: zencommand
Using regex '.*zencommand.py.*--daemon.*'
All Processes have stopped since the last model occurred. Last Modification time (2015/10/23 23:45:44)



Subject: Process not running: zencommand
Author: Tommy
Posted: 2015-10-26 10:46

IMHO your regex is a bit bizarre...
In Zenoss you should think like python programmer :)
Why you used a dot at the beginning of the regex



Subject: Hi IMHO
Author: [Not Specified]
Posted: 2015-10-26 18:50

A dot at the beginning means "no matter what or whatever" :) Im trying to find the cause of the problem and I guess that is more complicated than a simple regular expression thanks for your help



Subject: Run zencommand from the CLI
Author: [Not Specified]
Posted: 2015-10-27 09:49

Run zencommand from the CLI to see what is causing it to die.
Either do
"zencommand run " on its own
or
Run it against a specific device -
"zencommand run -d hostname.domain.com -v 10"



< Previous
Error INFO zen.zenoss.protocols.amqp: RabbitMQ connection was closed: [Errno 104 ...
  Next
ZENOSS 4.2.4 "EVENT AGING TASK AGED OUT THE EVENT" HOW TO STOP NOTIFICATION
>