|
|
| Subject: | Zenoss Broken after 5.0.2 to 5.0.3 upgrade |
| Author: | [Not Specified] |
| Posted: | 2015-06-04 13:18 |
Hi there Some time ago i performed an upgrade from 5.0 to 5.0.2, and i ended up with a broken installation (more or less like i will report now, but since it was a new installation, i started clean from 5.0.2 and reported nothing).
Now, with my system correctly configured, i created a snapshot (so i am safe :P), and went on to upgrade following the documentation.
Before i begin, i want to mention that the scripts to upgrade to 5.0.3 do not have the version number before their name, so the instructions are wrong (Again).
I have a community zenpack installed (WMVare esxi monitor, compatible with 5.0.x), and i had manually updated the microsoft windows zenpack (5.0.2 came with 2.3.2, and 2.4.1 was available back then). I noticed some error duing the upgrade regarding those zenpacks not bein installable or something.
After starting zennos, the following services display a RED exclamation mark:
zencommand zenjmx zenmodeler zenfersnmp zenping zenprocess zenpython zenstatus zensyslog zentrap zenhub
Trying to install either the vmware zenpack, or the upgraded microsoft windows 2.4.4 zenpack fails:
[root@zentoss5 zenpacks]# serviced service run zope zenpack install ZenPacks.community.VMwareES XiMonitor-2.0.5.egg
I0604 14:52:27.995923 09618 server.go:341] Connected to the control center at port 10.11.12.2:4 979
I0604 14:52:28.190516 09618 server.go:435] Acquiring image from the dfs...
I0604 14:52:28.192678 09618 server.go:437] Acquired! Starting shell
E0604 17:52:29.375999 00001 endpoint.go:514] No proxy for 8f5ai2cxhweox6378ongmj2oo_localhost_z enhubPB - no need to set empty address list
E0604 17:52:29.379290 00001 endpoint.go:514] No proxy for 8f5ai2cxhweox6378ongmj2oo_zproxy - no need to set empty address list
Trying to connect to logstash server... 127.0.0.1:5042
Connected to logstash server.
INFO:zen.Relations:Ignoring unresolvable object '
ERROR:zen.ZenPackCMD:Error installing ZenPack ZenPacks.community.VMwareESXiMonitor-2.0.5.egg
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenUtils/ZenPackCmd.py", line 252, in InstallEggAndZenPack
serviceId=serviceId)
File "/opt/zenoss/Products/ZenUtils/ZenPackCmd.py", line 487, in InstallDistAsZenPack
zenPack, deferFileDeletion, existing = transactional_actions()
File "/opt/zenoss/lib/python2.7/site-packages/ZODB/transact.py", line 44, in g
r = f(*args, **kwargs)
File "/opt/zenoss/Products/ZenUtils/ZenPackCmd.py", line 361, in transactional_actions
existing = dmd.ZenPackManager.packs._getOb(packName, None)
File "/opt/zenoss/Products/ZenRelations/ToManyContRelationship.py", line 204, in _getOb
return self._objects[id].__of__(self)
AttributeError: 'ZenPack' object has no attribute '__of__'
ERROR:zen.ZenPackCmd:zenpack command failed
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenUtils/zenpack.py", line 800, in
zp.run()
File "/opt/zenoss/Products/ZenUtils/zenpack.py", line 227, in run
serviceId=self.options.serviceId)
File "/opt/zenoss/Products/ZenUtils/ZenPackCmd.py", line 268, in InstallEggAndZenPack
raise e
AttributeError: 'ZenPack' object has no attribute '__of__'
Command returned non-zero exit code 3. Container not commited.
[root@zentoss5 zenpacks]# serviced service run zope zenpack install ZenPacks.zenoss.Microsoft.W indows-2.4.4.egg
I0604 14:53:57.592885 11044 server.go:341] Connected to the control center at port 10.11.12.2:4 979
I0604 14:53:57.794435 11044 server.go:435] Acquiring image from the dfs...
I0604 14:53:57.796166 11044 server.go:437] Acquired! Starting shell
E0604 17:53:58.875712 00001 endpoint.go:514] No proxy for 8f5ai2cxhweox6378ongmj2oo_localhost_z enhubPB - no need to set empty address list
E0604 17:53:58.878926 00001 endpoint.go:514] No proxy for 8f5ai2cxhweox6378ongmj2oo_zproxy - no need to set empty address list
Trying to connect to logstash server... 127.0.0.1:5042
Connected to logstash server.
INFO:zen.Relations:Ignoring unresolvable object '
INFO:zen.ZenPackCMD:Previous ZenPack exists with same name ZenPacks.zenoss.Microsoft.Windows
ERROR:zen.ZenPackCMD:Error installing ZenPack ZenPacks.zenoss.Microsoft.Windows-2.4.4.egg
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenUtils/ZenPackCmd.py", line 252, in InstallEggAndZenPack
serviceId=serviceId)
File "/opt/zenoss/Products/ZenUtils/ZenPackCmd.py", line 487, in InstallDistAsZenPack
zenPack, deferFileDeletion, existing = transactional_actions()
File "/opt/zenoss/lib/python2.7/site-packages/ZODB/transact.py", line 44, in g
r = f(*args, **kwargs)
File "/opt/zenoss/Products/ZenUtils/ZenPackCmd.py", line 396, in transactional_actions
for p in existing.packables():
File "/opt/zenoss/Products/ZenRelations/ToManyRelationship.py", line 71, in __call__
return self.objectValuesAll()
File "/opt/zenoss/Products/ZenRelations/ToManyRelationship.py", line 174, in objectValuesAll
return list(self.objectValuesGen())
File "/opt/zenoss/Products/ZenRelations/ToManyRelationship.py", line 179, in
return (obj.__of__(self) for obj in self._objects)
AttributeError: 'PortCheckDataSource' object has no attribute '__of__'
ERROR:zen.ZenPackCmd:zenpack command failed
Traceback (most recent call last):
File "/opt/zenoss/Products/ZenUtils/zenpack.py", line 800, in
zp.run()
File "/opt/zenoss/Products/ZenUtils/zenpack.py", line 227, in run
serviceId=self.options.serviceId)
File "/opt/zenoss/Products/ZenUtils/ZenPackCmd.py", line 268, in InstallEggAndZenPack
raise e
AttributeError: 'PortCheckDataSource' object has no attribute '__of__'
Command returned non-zero exit code 3. Container not commited.
[root@zentoss5 zenpacks]#
My control center still says "zenoss 5.0.2", buy in the "versions" screen in zenoss, i see 5.0.3. The template in control center still says 5.0.2 too, for new deployments.
Do i need to stay in 5.0.2 forever, or could there be a fix The logstash purging bug in 5.0.2 makes it not desirable to stay there T_T
I can easily go back to 5.0.2 with the vmware snapshot, but i will keep the system broken for a while, just in case we can work on it.
| Subject: | before you try installing |
| Author: | Andrew Kirch |
| Posted: | 2015-06-04 13:30 |
before you try installing ZenPacks you need to fix the red services. Can you provide log dumps
Andrew Kirch
akirch@gvit.com
Need Zenoss support, consulting or custom development Look no further. Email or PM me!
Ready for Distributed Topology (collectors) for Zenoss 5 Coming May 1st from GoVanguard
| Subject: | I can provide anything, but i |
| Author: | [Not Specified] |
| Posted: | 2015-06-04 13:44 |
I can provide anything, but i am new to zenoss, how do i dump them :P (i see logs.. logs everywhere :P)
| Subject: | If what i need to do is |
| Author: | [Not Specified] |
| Posted: | 2015-06-04 13:50 |
If what i need to do is download the logs of each red service, here they are:
https://www.dropbox.com/sh/k3wsva4idvxyfah/AAAew1FTYeKsEc4Ncc2xEbEsadl=0
| Subject: | any further help would be |
| Author: | [Not Specified] |
| Posted: | 2015-06-05 07:05 |
any further help would be appreciated :P
| Subject: | I was wondering... no upgrade |
| Author: | [Not Specified] |
| Posted: | 2015-06-05 09:42 |
I was wondering... no upgrade step asks us to "update zenos-core".
Just for fun, i restored the snapshot, and updathed that, now i see the 5.0.3 template, and i will see how things go.
| Subject: | I see now that the "upgrade" |
| Author: | [Not Specified] |
| Posted: | 2015-06-05 12:50 |
I see now that the "upgrade" fails to install custom zenpacks "because it does not find them", dunno if related to my further issues:
Script started on Fri 05 Jun 2015 01:15:29 PM ART
[94mI0605 13:15:29.826687 07140 runner.go:160] executing step 0: DESCRIPTION Zenoss Core 5.0.3 upgrade
[0m[94mI0605 13:15:29.826795 07140 runner.go:160] executing step 1: VERSION core-5.0.3
[0m[94mI0605 13:15:29.826806 07140 runner.go:160] executing step 2: REQUIRE_SVC
[0m[94mI0605 13:15:29.826816 07140 eval.go:329] checking service requirement
[0m[94mI0605 13:15:29.826824 07140 eval.go:333] verifying service 8f5ai2cxhweox6378ongmj2oo
[0m[94mI0605 13:15:29.827284 07140 eval.go:339] found 8f5ai2cxhweox6378ongmj2oo tenant id for service 8f5ai2cxhweox6378ongmj2oo
[0m[94mI0605 13:15:29.827303 07140 runner.go:160] executing step 3: SNAPSHOT
[0m[94mI0605 13:15:29.827320 07140 eval.go:22] performing snapshot
[0m[94mI0605 13:15:30.777262 07140 eval.go:41] snapshot id: 8f5ai2cxhweox6378ongmj2oo_20150605-161530
[0m[94mI0605 13:15:30.777299 07140 runner.go:160] executing step 4: SVC_USE zenoss/core_5.0:5.0.3
[0m[94mI0605 13:15:30.777313 07140 eval.go:59] preparing to use image: zenoss/core_5.0:5.0.3
[0m[94mI0605 13:15:30.777332 07140 service_client.go:31] Pulling zenoss/core_5.0:5.0.3, tagging to latest, and pushing to registry localhost:5000 - this may take a while
[0m[94mI0605 13:15:36.513188 07140 eval.go:68] Successfully tagged new image localhost:5000/8f5ai2cxhweox6378ongmj2oo/core_5.0:latest
[0m[94mI0605 13:15:36.513218 07140 runner.go:160] executing step 5: SVC_START Zenoss.core/mariadb
[0m[94mI0605 13:15:36.972666 07140 eval.go:159] starting service Zenoss.core/mariadb cnvmtsnrssvopggaxhpr5i3xu
[0m[94mI0605 13:15:37.001665 07140 runner.go:160] executing step 6: SVC_START Zenoss.core/RabbitMQ
[0m[94mI0605 13:15:37.448918 07140 eval.go:159] starting service Zenoss.core/RabbitMQ 2z61g5h2tq5bw41xawu93kcxy
[0m[94mI0605 13:15:37.495122 07140 runner.go:160] executing step 7: SVC_START Zenoss.core/zeneventserver
[0m[94mI0605 13:15:37.998267 07140 eval.go:159] starting service Zenoss.core/zeneventserver 10967bzuwz79aben7zozommab
[0m[94mI0605 13:15:38.042143 07140 runner.go:160] executing step 8: SVC_START Zenoss.core/redis
[0m[94mI0605 13:15:38.478582 07140 eval.go:159] starting service Zenoss.core/redis 2vlazl534yvvqbhwuwxbuzrf4
[0m[94mI0605 13:15:38.595789 07140 runner.go:160] executing step 9: SVC_WAIT Zenoss.core/mariadb Zenoss.core/RabbitMQ Zenoss.core/zeneventserver Zenoss.core/redis started 600
[0m[94mI0605 13:15:40.695275 07140 eval.go:123] waiting 600 for services Zenoss.core/mariadb, Zenoss.core/RabbitMQ, Zenoss.core/zeneventserver, Zenoss.core/redis to be started
[0m[94mI0605 13:15:41.217714 07140 runner.go:160] executing step 10: SVC_RUN Zenoss.core/Zope upgrade
[0m[94mI0605 13:15:41.620580 07140 eval.go:259] running: serviced service run apok3u8zkdk3540650m0ipjme upgrade
[0m[94mI0605 13:15:42.106413 07788 server.go:341] Connected to the control center at port 10.11.12.2:4979
[0m[94mI0605 13:15:42.307572 07788 server.go:435] Acquiring image from the dfs...
[0m[94mI0605 13:15:42.309075 07788 server.go:437] Acquired! Starting shell
[0mE0605 16:15:42.953346 00001 endpoint.go:514] No proxy for 8f5ai2cxhweox6378ongmj2oo_localhost_zenhubPB - no need to set empty address list
E0605 16:15:42.954582 00001 endpoint.go:514] No proxy for 8f5ai2cxhweox6378ongmj2oo_zep - no need to set empty address list
E0605 16:15:42.956677 00001 endpoint.go:514] No proxy for 8f5ai2cxhweox6378ongmj2oo_zproxy - no need to set empty address list
Trying to connect to logstash server... 127.0.0.1:5042
Connected to logstash server.
Not starting service yet, waiting on prereq: MariaDB connectivity
INFO:zen.ZenPackCmd:Restoring zenpacks
INFO:zen.ZenPackCmd:Attempting to install packs: ZenPacks.Firestar.CiscoASA, ZenPacks.TwoNMS.PrinterMIB, ZenPacks.community.VMwareESXiMonitor, ZenPacks.zenoss.Microsoft.Windows
INFO:zen.ZenPackCmd:looking for /opt/zenoss/.ZenPacks/ZenPacks.Firestar.CiscoASA-2.1.2-*
INFO:zen.ZenPackCmd:looking for /opt/zenoss/packs/ZenPacks.Firestar.CiscoASA-2.1.2-*
INFO:zen.ZenPackCmd:could not find install candidate for ZenPacks.Firestar.CiscoASA 2.1.2
INFO:zen.ZenPackCmd:looking for /opt/zenoss/.ZenPacks/ZenPacks.TwoNMS.PrinterMIB-2.0.0-*
INFO:zen.ZenPackCmd:looking for /opt/zenoss/packs/ZenPacks.TwoNMS.PrinterMIB-2.0.0-*
INFO:zen.ZenPackCmd:could not find install candidate for ZenPacks.TwoNMS.PrinterMIB 2.0.0
INFO:zen.ZenPackCmd:looking for /opt/zenoss/.ZenPacks/ZenPacks.community.VMwareESXiMonitor-2.0.5-*
INFO:zen.ZenPackCmd:looking for /opt/zenoss/packs/ZenPacks.community.VMwareESXiMonitor-2.0.5-*
INFO:zen.ZenPackCmd:could not find install candidate for ZenPacks.community.VMwareESXiMonitor 2.0.5
INFO:zen.ZenPackCmd:looking for /opt/zenoss/.ZenPacks/ZenPacks.zenoss.Microsoft.Windows-2.4.4-*
INFO:zen.ZenPackCmd:looking for /opt/zenoss/packs/ZenPacks.zenoss.Microsoft.Windows-2.4.4-*
INFO:zen.ZenPackCmd:could not find install candidate for ZenPacks.zenoss.Microsoft.Windows 2.4.4
INFO:zen.ZenPackCmd:Successfully restored zenpacks!
INFO:zen.migrate:Database going to version Zenoss 5.0.3
INFO:zen.migrate:Installing setDMDVersion (5.0.3)
INFO:zen.migrate:Loading Reports
[94mI0605 13:16:07.931277 07788 shell.go:225] Committing container
[0m[94mI0605 13:16:19.970415 07140 runner.go:152] Executing exit functions
[0m
Script done on Fri 05 Jun 2015 01:16:19 PM ART
| Subject: | My conclusions: |
| Author: | [Not Specified] |
| Posted: | 2015-06-08 07:10 |
My conclusions:
1) The upgrade script (at least on my setup) fails to find/reinstall custom or upgraded zenpacks
2) The vmware esxi monitor zenpack needs you to reinstall the dependencies after an upgrade
3) The "not tested" script for removing broken zenpacks removes ALL the zenpacks :P
So, i reinstalled all the zenpacks (hp, dell and mysql monitor had to be reinstalled twice, or maybe they might not, if i install them last).
Now i have 5.0.3 running, apparently fine.
Is there any way to "free" the space occupied by the 5.0.2 image that is no longer running
Also, shouln't we upgrade/update "zenoss-core" with "yum"
Finally, why does control center still say "zenoss core 5.0.2"
| < |
Previous Graphing PgSQL query result set from a remote machine |
Next Monitor Login failure / success in Active Directory domains? |
> |