How to get Multiple Zopes in Zenoss 4.2.x

Hi Zenoss Users,

I have been trying to figure out for awhile a way to get multiple zope process to run on one zenoss master server. I have many users of zenoss and zope is always the bottleneck. I got this working in my test environment and plan to promote these changes to production in the new year. Right now I GSLB to two varnish server then back to zenoss (zope). This will allow me to split to load further to two different zope processes.

I was able to figure this out looking at a patch hackman238 created (but did not work on zenoss core 4.2.x)
https://github.com/Hackman238/Random/blob/master/multiplezope.patch

Also consider tuning zope, especially the python-check-interval

If people are interested I can create a formal patch but I want to test this further under a production load.

You will need to create new directories under /opt/zenoss/var/ called zope plus the zope instance starting with 1. Create the amount of directories depending on the number of zope processes you will spin up. In this example I am going to start 2 zope processes.

[zenoss@a-tst-zen01 bin]$ ls -lsa /opt/zenoss/var/ | grep zope
4 drwxr-xr-x 2 zenoss zenoss 4096 Dec 30 12:51 zope1
4 drwxr-xr-x 2 zenoss zenoss 4096 Dec 30 12:51 zope2

[zenoss@a-tst-zen01 bin]$ ls -lsa /opt/zenoss/var/zope*
/opt/zenoss/var/zope1:
total 20
4 drwxr-xr-x 2 zenoss zenoss 4096 Dec 30 12:51 .
4 drwxr-xr-x 9 zenoss zenoss 4096 Dec 30 13:01 ..
4 -rw-r--r-- 1 zenoss zenoss 4 Dec 30 12:51 Z2.lock
4 -rw-r--r-- 1 zenoss zenoss 4 Dec 30 12:51 Z2.pid
4 -rw-rw-r-- 1 zenoss zenoss 82 Dec 29 13:39 zope.conf
0 srwx------ 1 zenoss zenoss 0 Dec 30 12:51 zopectlsock

/opt/zenoss/var/zope2:
total 20
4 drwxr-xr-x 2 zenoss zenoss 4096 Dec 30 12:51 .
4 drwxr-xr-x 9 zenoss zenoss 4096 Dec 30 13:01 ..
4 -rw-r--r-- 1 zenoss zenoss 4 Dec 30 12:51 Z2.lock
4 -rw-r--r-- 1 zenoss zenoss 4 Dec 30 12:51 Z2.pid
4 -rw-rw-r-- 1 zenoss zenoss 82 Dec 29 13:39 zope.conf
0 srwx------ 1 zenoss zenoss 0 Dec 30 12:51 zopectlsock

Also create a zope.conf under these new directories per zope instance

[zenoss@a-tst-zen01 bin]$ cat /opt/zenoss/var/zope*/zope.conf
%include /opt/zenoss/etc/zope.conf
clienthome /opt/zenoss/var/zope1
port-base 100
%include /opt/zenoss/etc/zope.conf
clienthome /opt/zenoss/var/zope2
port-base 101

per comments in the /opt/zenoss/etc/zope.conf
# Directives: port-base
#
# Description:
# Offset applied to the port numbers used for ZServer
# configurations. For example, if the http-server port is 8080 and
# the port-base is 1000, the HTTP server will listen on port 9080.
# This makes it easy to change the complete set of ports used by a
# Zope server process

Edit two files in /opt/zenoss/bin, zopectl and runzope

[zenoss@a-tst-zen01 ~]$ cd /opt/zenoss/bin/
[zenoss@a-tst-zen01 bin]$ cat zopectl
#!/bin/sh
PYTHON="/opt/zenoss/bin/python"
INSTANCE_HOME="/opt/zenoss"
export INSTANCE_HOME PYTHON

ZDCTL="/opt/zenoss/zopehome/zopectl"

ZOPES=2

for i in $(seq 1 $ZOPES); do
##Setup enviornment to start zope
clienthome="$INSTANCE_HOME/var/zope$i"
cfg="$clienthome/zope.conf"
CONFIG_FILE=$cfg
export CONFIG_FILE
##Start Zope
"$PYTHON" "$ZDCTL" -C "$cfg" "$@"
done

[zenoss@a-tst-zen01 bin]$ cat runzope
#!/bin/sh
CONFIG_FILE="${CONFIG_FILE:-/opt/zenoss/etc/zope.conf}"
ZOPE_RUN="/opt/zenoss/zopehome/runzope"
exec "$ZOPE_RUN" -C "$CONFIG_FILE" "$@"

Start!

[zenoss@a-tst-zen01 bin]$ zopectl start
.
daemon process started, pid=9937
.
daemon process started, pid=9942

[zenoss@a-tst-zen01 bin]$ tail -12 ../log/event.log
------
2014-12-30T13:14:28 INFO ZServer HTTP server started at Tue Dec 30 13:14:28 2014
Hostname: 0.0.0.0
Port: 8180
------
2014-12-30T13:14:29 INFO ZServer HTTP server started at Tue Dec 30 13:14:29 2014
Hostname: 0.0.0.0
Port: 8181
------
2014-12-30T13:14:32 INFO Zope Ready to handle requests
------
2014-12-30T13:14:33 INFO Zope Ready to handle requests

[zenoss@a-tst-zen01 bin]$ netstat -an | grep 818
tcp 0 0 0.0.0.0:8180 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:8181 0.0.0.0:* LISTEN

[zenoss@a-tst-zen01 bin]$ ps -ef | grep zope
zenoss 10078 1 0 13:14 00:00:00 /opt/zenoss/bin/python /opt/zenoss/lib/python/zdaemon/zdrun.py -S /opt/zenoss/lib/python/Zope2/Startup/zopeschema.xml -b 10 -s /opt/zenoss/var/zope1/zopectlsock -x 0,2 -z /opt/zenoss /opt/zenoss/bin/runzope
zenoss 10080 10078 2 13:14 00:00:04 /opt/zenoss/bin/python /opt/zenoss/zopehome/runzope -C /opt/zenoss/var/zope1/zope.conf
zenoss 10083 1 0 13:14 00:00:00 /opt/zenoss/bin/python /opt/zenoss/lib/python/zdaemon/zdrun.py -S /opt/zenoss/lib/python/Zope2/Startup/zopeschema.xml -b 10 -s /opt/zenoss/var/zope2/zopectlsock -x 0,2 -z /opt/zenoss /opt/zenoss/bin/runzope
zenoss 10104 10083 2 13:14 00:00:04 /opt/zenoss/bin/python /opt/zenoss/zopehome/runzope -C /opt/zenoss/var/zope2/zope.conf
zenoss 10325 8591 0 13:17 pts/0 00:00:00 grep zope