TECHZEN Zenoss User Community ARCHIVE  

KeyError on Device

Subject: KeyError on Device
Author: [Not Specified]
Posted: 2016-10-26 00:34

Hi

Running Zenoss Core4.2.5.

I have a problem where I get a "KeyError" flare in the webinterface when trying to set productionState on a device. I have tried running findposkeyerror, zenchkrels etc. (all the tools from the zenosstoolbox) but I'm unable to find any error thus not able to recover it.



Subject: I have pasted the stacktrace
Author: [Not Specified]
Posted: 2016-10-26 00:38

I have pasted the stacktrace on:

http://pastebin.com/Q5WuAZNz

I have been unable to paste in here in the forums.
I have tried to delete the device and many other things, all end in the same KeyError. I have tried all the tricks that I can find, but apparently I'm not able to remove this.

I have replicated this on my test-server (by restoring my backup) so any tips are more than welcome... I plan to test it on the test-system and then implement it in production when I'm confident that it has solved the problem.

Thomas



Subject: So all the tools report no
Author: Jane Curry
Posted: 2016-10-26 05:13

So all the tools report no errors Even zencatalogscan

There is also a zenindextool utility in later versions of toolbox which takes a "-t" parameter for type to reindex. Have you tried this with both "-t Device" and "-t DeviceSearch" The documentation here -https://support.zenoss.com/hc/en-us/articles/203263689 - says you should stop zenhub, zenactiond and webserver (zopectl in Zenoss Core) first. Suggesting this as your stacktrace looks like it is failing around reindexing.

What sort of a device is it

What tricks have you tried to remove it

Cheers,

Jane

Email: jane.curry@skills-1st.co.uk    Web: https://www.skills-1st.co.uk



Subject: Yes, I have run zodbscan,
Author: [Not Specified]
Posted: 2016-10-27 03:51

Yes, I have run zodbscan, findposkeyerror, zenrelationscan and zencatalogscan and all reported no problems with my device.

I have tried to run zenindextool on both Devices and DeviceSearch, and this did not help either.

The device is a Unix server with some Modeler plugins (Zeus Loadbalancer zenpack). It is infact also a device which is associated to a remote collector.

I have tried to tricks with described on:

http://www.zenoss.org/forum/8976

as well as:

import sys
from ZODB.POSException import POSKeyError
from BTrees.OOBTree import OOSet
from Products.ZenModel.ZenStatus import ZenStatus
dev=find('servername')
dev.macaddresses = OOSet()
commit()
sync()
dev._delOb("_lastPollSnmpUpTime")
commit()
dev._lastPollSnmpUpTime = ZenStatus(0)
commit()
sync()
quit()



Subject: Can you move this device to
Author: Jane Curry
Posted: 2016-10-27 13:33

Can you move this device to the localhost collector

Can you use the ZMI to see characteristics / change the ProdState For example, my server is zen42.class.example.org, the ZMI is accessed at https://zen42.class.example.org/zport/dmd/manage - basically stick "zport/dmd/manage" after your normal http / server / port url. Navigate to the offending device and click the Properties tab (far right). You will need a Zenoss GUI user with Manager role to access the ZMI. If you are really lucky, you may be able to change the production state in the ZMI.

Is it just changing prodState that is an issue with this device or does it have other issues

Can you use zendmd to change prod state so, for my zen42 server,:

zendmd

d=find('zen42.class.example.org')

d.productionState responds with 300 ie Maintenance

d.setProdState(1000)

d.productionState responds with 1000 ie Production

This hasn't changed anything in the GUI yet. If this has worked so far, you might commit this in zendmd so it is reflected in the GUI:

commit()

I suspect these commands won't actually succeed but it might give us some more error messages to work on.

Cheers,

Jane

Email: jane.curry@skills-1st.co.uk    Web: https://www.skills-1st.co.uk



Subject: I have moved it to the
Author: [Not Specified]
Posted: 2016-10-28 01:53

I have moved it to the localhost collector and this worked.

I have tried via the ZMI, and it responds with a keyError, so this did not work.

I get a keyError if I try and change DeviceClass also...

I have tried via zendmd and it responds with a KeyError when I run d.setProdState(300) (without the commit).



Subject: Presumably you have tried
Author: Jane Curry
Posted: 2016-10-29 07:41

Presumably you have tried remodelling it

If you have managed to change it's collector then it isprobably worth re-running the tools again.

I'm afraid my experience with this sort of issue is that you keep trying stuff and eventually you do manage to delete the wretched thing.

When using zendmd, did it report correctly with d.productionState

Have you seen this append -http://devarthur.blogspot.co.uk/2010/03/deleting-all-devices-from-zenoss-with.html . It is obviously doing a lot more than you need but it gives you an example of the deleteDevice method.

There are also some zendmd examples here -http://zenpackers.readthedocs.io/en/latest/zendmd.html

Cheers,

jane

Email: jane.curry@skills-1st.co.uk    Web: https://www.skills-1st.co.uk



Subject: Yep, modeling has been tried,
Author: [Not Specified]
Posted: 2016-10-31 03:33

Yep, modeling has been tried, no luck.

It did not help running the tools after changing collector.

d.productionState returns fine (1000), but keyError when calling d.setProdState(300).

deleteDevice also returns KeyError.

I will take a look at the zendmd examples to see if I can find the solution... but I'm still very much grateful for any and all tips and pointers to how I can solve this.

Thomas



Subject: I think I found a way to fix
Author: [Not Specified]
Posted: 2016-11-02 03:45

I think I found a way to fix the problem, some testing shows that if I stop zenoss completely and then enters zendmd and runs a reindex() (takes like 30 minutes) and then a commit() (takes 2-3 minutes) then I am able to change productionstate on the device...

Running the reindex() while zenoss was running was not successful.

Thanks for the pointers given here... :-)

Thomas



< Previous
SUP update for 4.2.5 overdue?
  Next
Zenrender not starting
>