TECHZEN Zenoss User Community ARCHIVE  

ZenRRDCache Randomly dies - Zenoss 4.2.5

Subject: ZenRRDCache Randomly dies - Zenoss 4.2.5
Author: Tim Meads
Posted: 2017-09-04 15:06

I've noticed that my graphs completely die out. When I check zenoss status, I see rrdcache has stopped. I refire RRD cache and the graphs come back with no data, after a few minutes, it dies again. When I refire rrdcache, there is a small blip of data but thats it. Then it dies again. Looking through the logs, all I can see is something that shoes me the connection is refused. But nothing as to why the service itself has died. 

/opt/zenoss/var/log/event.log shows;

2017-09-04T14:42:39 ERROR zen.RenderServer Failed to generate a graph

Traceback (most recent call last):

  File "/opt/zenoss/Products/ZenRRD/RenderServer.py", line 184, in render

    rrdtool_fn()

  File "/opt/zenoss/Products/ZenUtils/Utils.py", line 1700, in _inner

    return fn(*args,**kwargs)

  File "/opt/zenoss/Products/ZenRRD/RenderServer.py", line 183, in rrdtool_fn

    rrdtool.graph(*(gopts + list(rrd_daemon_args())))

error: Unable to connect to rrdcached: Connection refused


It's the only instance of rrdcahe in the logfiles dir. Not sure where else to look or whatever could be causing it to go stale. 

If I run the command manually, you get the following output. If I run it as the zenrrdcache start, it tells me the service is already running and it can't create the PID file. 

[zenoss@zen2 ~]$ /usr/bin/rrdcached -b /opt/zenoss/perf -p /opt/zenoss/var/rrdcached.pid -l /opt/zenoss/var/rrdcached.sock -j /opt/zenoss/var/rrd_journals

rrdcached: can't create pid file '/opt/zenoss/var/rrdcached.pid' (File exists)

rrdcached: removed stale PID file (no rrdcached on pid 24930)

rrdcached: starting normally.

[zenoss@zen2 ~]$ ps -ef | grep cache

492       2029     1  0 Sep02 ?        00:09:10 memcached -d -p 11211 -u memcached -m 64 -c 1024 -P /var/run/memcached/memcached.pid

zenoss   27047     1 16 14:42 ?        00:00:03 /usr/bin/rrdcached -b /opt/zenoss/perf -p /opt/zenoss/var/rrdcached.pid -l /opt/zenoss/var/rrdcached.sock -j /opt/zenoss/var/rrd_journals

zenoss   27079 20929  0 14:43 pts/0    00:00:00 grep cache

[zenoss@zen2 ~]$


Zenoss Zenoss 4.2.5
OS Linux (x86_64) 2.6.32 (Linux zen2.mwtn.net 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64)
Zope Zope 2.13.13
Python Python 2.7.2
Database MySQL 5.5.42 (5.5.42)
RRD RRDtool 1.4.7
Twisted Twisted 11.0.0
RabbitMQ RabbitMQ 2.8.7
Erlang Erlang 5.8.5
NetSnmp NetSnmp 5.5.0
PyNetSnmp PyNetSnmp 0.30.7
WMI Wmi 1.3.15


------------------------------
Tim Meads
NOC Supervisor
Mountain West Technologies Network
------------------------------


< Previous
Error when a login to the admin portal
  Next
Maintenance Window not working (Zenoss 5.2.1)
>