Zenoss ZenTech Community

I have there are more than 2,5 thousands devices in our Zenoss instance (4.2.4). It's consists of master-collector (8CPU, 32GB RAM), few collectors (6 CPU 16GB RAM), remote DB (8 CPU 32GB RAM) and LDAP integration. After additional 100 devices performance of web gui degrese. Every menu opened very slow, more than 2 minutes. More than 50 users used Zenoss in our company.

P.S: All advices by Zenoss Wiki performance tuning page was done early for thousands devices, but it not helping now.

Subject:	Monitor it :-) Every part,
Author:	Jan Garaj
Posted:	2015-01-19 12:42

Monitor it :-) Every part, which is involved (zenwebserver: apache/nginx, zope). Statistics, response times, but also per instance (worker).
In my case it's complicated: Loadbalancer->Apache->Nginx->Zopes. I monitor response times from every level. Zenpacks for nginx and apache are available so install them.

I expect that your problem is Zope (own experience). Common response zope time is <0.3sec, but sometimes they work ~4sec or have also timeout (5 sec in my case). And zope instance was in idle mode (no job assignment in progress) at that time, just one TCP connection in CLOSE-WAIT.

Normal state is 1-2 timeouts per day. Because I have 11 zope instances, so I can live with this problem :-) But it looks funny sometimes - http://i.imgur.com/CeJAEyy.png

Also I recommend you to monitor web use cases - (e.g. GET index, POST events,...) - http://i.imgur.com/TQtXqXG.png, because it can be something specific (e.g. a huge amount of events, so only Events page response time has increased).

Definitely check also all zenoss logs. It can be IMHO also some zenhub performance issue.

If you have commercial version, then raise also support ticket - support has some nice zope stat script.

Devops Monitoring Expert advice: Dockerize/automate/monitor all the things.

DevOps stack: Docker / Kubernetes / Mesos / Zabbix / Zenoss / Grafana / Puppet / Ansible / Vagrant / Terraform / Elasticsearch

Subject:	Hello Jan,
Author:	[Not Specified]
Posted:	2015-01-21 00:11

Hello Jan,

Tell me please, what count of devices monitors by your Zenoss instance

Subject:	Similar: 2.5k+ devices, but
Author:	Jan Garaj
Posted:	2015-01-21 03:26

Similar: 2.5k+ devices, but count of devices is irrelevant metric for the problem with zenwebserver - you can have 1M devices and if only 1 user will use Web GUI, it'll be IMHO fine.

More suitable metrics in my case: avg 70-100 unique users per day with avg 250k-300k requests per day. I saw also some "special" user with 70k requests per user and per day (=one user has generated 25% of day requests - without any scripts)

Subject:	Zenoss Web GUI Performance
Author:	[Not Specified]
Posted:	2015-01-19 09:23

Zenoss Web GUI Performance