![]() |
![]() |
Subject: | ZenOSS 4.2.5: zencommand issue (password has expired) |
Author: | [Not Specified] |
Posted: | 2015-04-27 05:37 |
Hi,
My ZenOSS/OS version:
###########################################
Product: zenoss-core-4.2.5 (id = zenoss-core-4.2.5)
Home: /opt/zenoss
Revision: 0
Upgrading: 203
Last Attempted Step: post/run_migrates 4.2.5
Updated On: Mon Nov 3 10:36:44 2014
###########################################
Red Hat Enterprise Linux Server release 6.1 (Santiago)
###########################################
Monitored host version:
###########################################
Solaris 10
###########################################
I had several events about file descriptors out of range like this:
An error occurred while connecting: [Failure instance: Traceback (failure with no frames):
I increased soft/hard limits:
[zenoss@X]$ cat /etc/security/limits.conf | grep zen
zenoss soft nofile 4096
zenoss hard nofile 10240
I checked and increased a twisted maxfds limitation:
"/opt/zenoss/lib/python/twisted/internet/process.py":
BUG: We call select no matter what the reactor.
If the reactor is pollreactor, and the fd is > 1024, this will fail.
(only occurs on broken versions of linux, though).
maxfds = resource.getrlimit(resource.RLIMIT_NOFILE)[1] + 1
# OS-X reports 9223372036854775808. That's a lot of fds
# to close
if maxfds > 1024:
#maxfds = 1024
maxfds = 2048
return xrange(maxfds)
Finally I noticed an issue with a specific situacion: I had a monitored host using SSH (zencommand) which its password had expired alerting like this:
The command ps -eo args returned stderr data (1) from the device: WARNING: Your password has expired. Password change required but no TTY available.
I checked lots of SSH established connections on ZenOSS host to monitored host:
[root@X]# ls -l /proc/`pgrep -f zencommand`/fd | wc -l
701
[root@X]# netstat -anp | grep "`pgrep -f zencommand`" | grep X.X.X.X | wc -l
639
I suppose it's a zencommand bug:
"/opt/zenoss/Products/ZenRRD/zencommand.py":
class SshRunner(object):
"""
Run a single command across a cached SSH connection
"""
EXPIRED_MESSAGES = ("WARNING: Your password has expired.\nPassword"\
" change required but no TTY available.\n",)
...
if not self._connection.is_expired \
and self.stderr in SshRunner.EXPIRED_MESSAGES:
log.debug('Connection %s expired, cleaning up pool',
self._connection.connection_description)
self._connection.is_expired = True
self._connection.cleanUpPool()
return self
Regards,
Carlos Ramos
< |
Previous Zenoss Analytics fct_event, vba embeded in excel |
Next Decommissioned server shows 100 availability on reports |
> |