TECHZEN Zenoss User Community ARCHIVE  

Zenoss 4.2.5 - global_catalog contains broken object

Subject: Zenoss 4.2.5 - global_catalog contains broken object
Author: Paul Manno
Posted: 2020-04-20 17:55

Hello,

Can't seem to get rid of a broken device tree in global_catalog.  Toolkit apps return:

zodbscan: 1 dangling reference
PATH: dmd/Devices/deviceSearch/_catalog/path/_index
TYPE: <type 'BTrees.OOBTree.OOBucket'>
OID: 0x00022090 '\x00\x00\x00\x00\x00\x02 \x90' 139408
Refers to a missing object:
NAME: None
TYPE: <type 'BTrees.IOBTree.IOBTree'>
OID: 0x00582743 "\x00\x00\x00\x00\x00X'C" 5777219

findposkeyerror and zenrelationscan return zero errors.

How can I remove the item from the tree?  I want to remove the entire /zport/dmd/Devices/Server/iDRAC node.

ERROR zencatalogscan: Catalog global_catalog contains broken object /zport/dmd/Devices/Server/iDRAC/devices/NNN.NNN.NNN.NNN/dellSensors/System_Board_SYS_Usage

Thanks!
Paul


Subject: RE: Zenoss 4.2.5 - global_catalog contains broken object
Author: Laurent Hemeryck
Posted: 2020-04-21 02:51

Hello Paul, 

zencatalogscan has reported a broken object. Have you fixed it ? 

Launch the command: zencatalogscan -f -n 10

Cheers, 

Laurent


------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------


Subject: RE: Zenoss 4.2.5 - global_catalog contains broken object
Author: Paul Manno
Posted: 2020-04-21 08:13

Hi Laurent,

I didn't run fix because the doc says:

"WARNING: If any catalog issues are detected, before you re-run with the --fix flag, first confirm that zodbscanfindposkeyerror, and zenrelationscan all return with no issues found."

And zodbscan was reporting an error... Is it ok to run fix even with the zodbscan error ?

Paul


------------------------------
Paul Manno
------------------------------


Subject: RE: Zenoss 4.2.5 - global_catalog contains broken object
Author: Laurent Hemeryck
Posted: 2020-04-21 08:24

You didn't mention zodbscan. First run a fix with zodbscan.

Then scan again with findposkeyerror, zodbscan, zenrelationscan and zencatalogscan in that order. If any of those reports an error, run it again with those arguments: "-f -n 10"

Laurent

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------


Subject: RE: Zenoss 4.2.5 - global_catalog contains broken object
Author: Paul Manno
Posted: 2020-04-21 10:03

Hi Laurent,

Thanks for the quick reply.  I ran both findposkeyerror and zenrelationscan and they both return zero errors.  So the only toolkit apps that are showing an error are zodbscan and zencatalogscan.

from my first post:
zodbscan: 1 dangling reference
PATH: dmd/Devices/deviceSearch/_catalog/path/_index
TYPE: <type 'BTrees.OOBTree.OOBucket'>
OID: 0x00022090 '\x00\x00\x00\x00\x00\x02 \x90' 139408
Refers to a missing object:
NAME: None
TYPE: <type 'BTrees.IOBTree.IOBTree'>
OID: 0x00582743 "\x00\x00\x00\x00\x00X'C" 5777219

and

ERROR zencatalogscan: Catalog global_catalog contains broken object /zport/dmd/Devices/Server/iDRAC/devices/NNN.NNN.NNN.NNN/dellSensors/System_Board_SYS_Usage

Given the doc says "don't run zencatalogscan -f without having zero errors on the other runs" how am I supposed to fix this?  Any ideas?

Thanks again,
Paul



------------------------------
Paul Manno
------------------------------


Subject: RE: Zenoss 4.2.5 - global_catalog contains broken object
Author: Laurent Hemeryck
Posted: 2020-04-21 10:28

First run a fix with zodbscan: 
zodbscan -f -n 10

Then again, go through all tools in the same order. 


------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------


Subject: RE: Zenoss 4.2.5 - global_catalog contains broken object
Author: Paul Manno
Posted: 2020-04-21 10:34

zodbscan doesn't seem to have a fix option:

[zenoss@zenoss4 ~]$ zodbscan -f -n 10
usage: zodbscan [-h] [-v] [-v10] [--tmpdir TMPDIR] [-s]
zodbscan: error: unrecognized arguments: -f -n 10

[zenoss@zenoss4 ~]$ zodbscan -v
2.0.0



------------------------------
Paul Manno
------------------------------


Subject: RE: Zenoss 4.2.5 - global_catalog contains broken object
Author: Laurent Hemeryck
Posted: 2020-04-21 10:48

OK. So, in a summary we have: 
- zodbscan that reports an error within a catalog
- zencatalogscan that reports an error within the global catalog
- the other tools don't report any error

You can still try zenossdbpack, but the chance that it cleans up your object is small. Make sure to have a backup and yes, indeed, I would run zencatalogscan -f -n 10, because you have two tools that report an error within a catalog.

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------


Subject: RE: Zenoss 4.2.5 - global_catalog contains broken object
Author: Paul Manno
Posted: 2020-04-21 11:50

Hi Laurent,

Thanks for the help.  I snapshot my instance and executed: zencatalogscan -f -n 10.  That did fix the error.

All the best,
Paul

------------------------------
Paul Manno
------------------------------


< Previous
ZenOSS Core 6.2.1 runaway resources
  Next
Graphs contain no data
>