TECHZEN Zenoss User Community ARCHIVE  
FORUMS / ZenPacks, JSON API, and Integrations / DISCUSSION

Modeling times out in Custom ZenPack

Subject: Modeling times out in Custom ZenPack
Author: Karl Lentz
Posted: 2021-04-22 17:01

Hello everyone,

I am developing a ZenPack so that I can monitor a Veritiv Environmental Sensor.
I have my device and component classes defined in yaml I believe correctly.  And when I only have the device modeler plugin active the device properly models.  But as soon as I add my component modeler the modeling process times out.  I'm not sure what I am missing.  Any help is greatly appreciated.

Here is my component modeler
from Products.DataCollector.plugins.CollectorPlugin import SnmpPlugin, GetTableMap

class VeritivSensorMap(SnmpPlugin):

    relname = 'veritivSensors'
    modname = 'Zenpacks.atldc.Veritiv.VeritivSensor'

    snmpGetTableMaps = (
        GetTableMap('internalSensorTable',
                    '.1.3.6.1.4.1.21239.5.1.2.1',
                    {
                        '.1': 'internalIndex',
                        '.2': 'serialNumber',
                        '.3': 'sensorName',
                    }
            ),
        )

    def process(self, device, results, log):
        log.info('processing %s for device %s', self.name(), device.id)

        getdata, tabledata = results

        internalSensorTable = tabledata.get('internalSensorTable', {})
        log.info('SensorTable: %s', internalSensorTable)

        if not internalSensorTable:
            log.warn('No SNMP response from %s for the %s plugin', device.id, self.name())
            log.warn('Table Data= %s', tabledata)
            return

        rm = self.relMap()

        for oid, data in internalSensorTable.items():
            try:
                om = self.objectMap(data)
                om.id = self.prepId(om.sensorName)
                om.snmpindex = oid
                log.info('Object Map: %s', om)
                rm.append(om)
            except (KeyError, IndexError, AttributeError, TypeError), errorInfo:
                log.warn('Error in %s modeler plugin %s', self.name(), errorinfo)
                continue
        return rm
​


And here is my yaml file:

name: ZenPacks.atldc.Veritiv

classes:
  VeritivDevice:
    base: [zenpacklib.Device]
    label: Veritiv Device
    properties:
      deviceCount:
        label: Device Count
        type: int

  VeritivSensor:
    base: [zenpacklib.Component]
    label: Veritiv Sensor
    properties:
      serialNumber:
        label: Serial Number
      sensorName:
        label: Sensor Name

class_relationships:
  - VeritivDevice(veritivSensors) 1:MC VeritivSensor(veritivDevice)

device_classes:
  /Network/Watchdog/Veritiv:
    zProperties:
      zPythonClass: ZenPacks.atldc.Veritiv.VeritivDevice
      zSnmpMonitorIgnore: false
      zCollectorPlugins:
        - atldc.snmp.VeritivDeviceMap
        - atldc.snmp.VeritivSensorMap
        - zenoss.snmp.NewDeviceMap
        - zenoss.snmp.DeviceMap
        - zenoss.snmp.InterfaceMap

And here is the output from Zenmodeler:
[zenoss@localhost z]$ zenmodeler run --device=10.103.42.5
2021-04-22 20:01:34,081 INFO zen.ZenModeler: Connecting to localhost:8789
2021-04-22 20:01:34,107 INFO zen.ZenModeler: Connected to the zenhub/0 instance
2021-04-22 20:01:34,108 INFO zen.ZenModeler: Getting configuration from ZenHub...
2021-04-22 20:01:34,238 INFO zen.ZenModeler: Starting modeling in 0 seconds.
2021-04-22 20:01:34,238 INFO zen.ZenModeler: Starting collector loop #001...
2021-04-22 20:01:34,238 INFO zen.ZenModeler: Collecting for device 10.103.42.5
2021-04-22 20:01:34,239 INFO zen.ZenModeler: Got 1 devices to be scanned during collector loop #001
2021-04-22 20:01:34,340 INFO zen.ZenModeler: Filled collection slots for 1 of 1 devices during collector loop #001
2021-04-22 20:01:34,341 INFO zen.ZenModeler: skipping WMI-based collection, PySamba zenpack not installed
2021-04-22 20:01:34,341 INFO zen.ZenModeler: Collect on device 10.103.42.5 for collector loop #001
2021-04-22 20:01:34,382 INFO zen.ZenModeler: No Python plugins found for 10.103.42.5
2021-04-22 20:01:34,385 INFO zen.ZenModeler: No command plugins found for 10.103.42.5
2021-04-22 20:01:34,388 INFO zen.ZenModeler: SNMP collection device 10.103.42.5
2021-04-22 20:01:34,388 INFO zen.ZenModeler: plugins: atldc.snmp.VeritivDeviceMap, zenoss.snmp.NewDeviceMap, zenoss.snmp.DeviceMap, zenoss.snmp.InterfaceMap, atldc.snmp.VeritivSensorMap
2021-04-22 20:01:34,392 INFO zen.ZenModeler: No portscan plugins found for 10.103.42.5
2021-04-22 20:01:40,024 INFO zen.SnmpClient: snmp client finished collection for 10.103.42.5
2021-04-22 20:01:40,025 INFO zen.ZenModeler: processing atldc.snmp.VeritivDeviceMap for device 10.103.42.5
2021-04-22 20:01:40,025 INFO zen.ZenModeler: Device Properties post: <ObjectMap {'classname': '',
 'compname': '',
 'deviceCount': 2,
 'modname': 'Zenpacks.atldc.Veritiv.VeritivDevice',
 'setHWModel': 'Geist WD100',
 'setHWSerialNumber': 'TB19070494'}>
2021-04-22 20:01:40,026 INFO zen.ZenModeler: Processing zenoss.snmp.NewDeviceMap for device 10.103.42.5
2021-04-22 20:01:40,027 INFO zen.ZenModeler: processing zenoss.snmp.DeviceMap for device 10.103.42.5
2021-04-22 20:01:40,027 INFO zen.ZenModeler: Modeler zenoss.snmp.InterfaceMap processing data for device 10.103.42.5
2021-04-22 20:01:40,027 WARNING zen.ZenModeler: Unable to get data for 10.103.42.5 from either ipAddrTable or ipNetToMediaTable
2021-04-22 20:01:40,028 INFO zen.ZenModeler: processing atldc.snmp.VeritivSensorMap for device 10.103.42.5
2021-04-22 20:01:40,028 INFO zen.ZenModeler: SensorTable: {'1': {'serialNumber': 'AB801F12ACA2A9C3', 'sensorName': 'POD-4'}}
2021-04-22 20:01:40,028 INFO zen.ZenModeler: Object Map: <ObjectMap {'classname': '',
 'compname': '',
 'id': 'POD-4',
 'modname': 'Zenpacks.atldc.Veritiv.VeritivSensor',
 'sensorName': 'POD-4',
 'serialNumber': 'AB801F12ACA2A9C3',
 'snmpindex': '1'}>
2021-04-22 20:02:04,082 INFO zen.ZenModeler: Counter eventCount, value 7
2021-04-22 20:02:34,082 INFO zen.ZenModeler: Counter eventCount, value 13
2021-04-22 20:03:04,082 INFO zen.ZenModeler: Counter eventCount, value 19
2021-04-22 20:03:34,082 INFO zen.ZenModeler: Counter eventCount, value 25
2021-04-22 20:04:04,082 INFO zen.ZenModeler: Counter eventCount, value 31
2021-04-22 20:04:34,082 INFO zen.ZenModeler: Counter eventCount, value 37
2021-04-22 20:04:34,422 WARNING zen.ZenModeler: Client 10.103.42.5 timeout
2021-04-22 20:04:34,423 INFO zen.ZenModeler: Scan time: 180.18 seconds for collector loop #001
2021-04-22 20:04:34,423 INFO zen.ZenModeler: Scanned 1 of 1 devices during collector loop #001
2021-04-22 20:04:34,424 INFO zen.ZenModeler: Daemon ZenModeler shutting down
​


------------------------------
Karl Lentz
Sr. Network Engineer
ATL Data Centers LLC
College Park GA
4049611009
------------------------------

Subject: RE: Modeling times out in Custom ZenPack
Author: Ryan Matte
Posted: 2021-04-22 19:48

Try increasing zCollectorClientTimeout to a higher value.  That's what controls the overall modelling timeout (I know that the name of the zProperty isn't the most intuitive).  It's possible that your modelling is just legitimately taking longer than the currently configured timeout.  If you crank zCollectorClientTimeout up to some really high value like 5000 and the modelling still fails then there's possibly something wrong with your modeler code, but hopefully increasing the timeout will fix it.  Note that you should only increase zCollectorClientTimeout specifically on devices that need it, also you'll want to start out with some really high value for testing but then reduce it to a more realistic value that's closer to the actual time the modelling takes once you figure that out.

------------------------------
Ryan Matte
------------------------------ 
from Products.DataCollector.plugins.CollectorPlugin import SnmpPlugin, GetTableMapclass VeritivSensorMap(SnmpPlugin):    relname = 'veritivSensors'    modname = 'Zenpacks.atldc.Veritiv.VeritivSensor'    snmpGetTableMaps = (        GetTableMap('internalSensorTable',                    '.1.3.6.1.4.1.21239.5.1.2.1',                    {                        '.1': 'internalIndex',                        '.2': 'serialNumber',                        '.3': 'sensorName',                    }            ),        )    def process(self, device, results, log):        log.info('processing %s for device %s', self.name(), device.id)        getdata, tabledata = results        internalSensorTable = tabledata.get('internalSensorTable', {})        log.info('SensorTable: %s', internalSensorTable)        if not internalSensorTable:            log.warn('No SNMP response from %s for the %s plugin', device.id, self.name())            log.warn('Table Data= %s', tabledata)            return        rm = self.relMap()        for oid, data in internalSensorTable.items():            try:                om = self.objectMap(data)                om.id = self.prepId(om.sensorName)                om.snmpindex = oid                log.info('Object Map: %s', om)                rm.append(om)            except (KeyError, IndexError, AttributeError, TypeError), errorInfo:                log.warn('Error in %s modeler plugin %s', self.name(), errorinfo)                continue        return rm​


And here is my yaml file:

name: ZenPacks.atldc.Veritivclasses:  VeritivDevice:    base: [zenpacklib.Device]    label: Veritiv Device    properties:      deviceCount:        label: Device Count        type: int  VeritivSensor:    base: [zenpacklib.Component]    label: Veritiv Sensor    properties:      serialNumber:        label: Serial Number      sensorName:        label: Sensor Nameclass_relationships:  - VeritivDevice(veritivSensors) 1:MC VeritivSensor(veritivDevice)device_classes:  /Network/Watchdog/Veritiv:    zProperties:      zPythonClass: ZenPacks.atldc.Veritiv.VeritivDevice      zSnmpMonitorIgnore: false      zCollectorPlugins:        - atldc.snmp.VeritivDeviceMap        - atldc.snmp.VeritivSensorMap        - zenoss.snmp.NewDeviceMap        - zenoss.snmp.DeviceMap        - zenoss.snmp.InterfaceMap

And here is the output from Zenmodeler:
[zenoss@localhost z]$ zenmodeler run --device=10.103.42.52021-04-22 20:01:34,081 INFO zen.ZenModeler: Connecting to localhost:87892021-04-22 20:01:34,107 INFO zen.ZenModeler: Connected to the zenhub/0 instance2021-04-22 20:01:34,108 INFO zen.ZenModeler: Getting configuration from ZenHub...2021-04-22 20:01:34,238 INFO zen.ZenModeler: Starting modeling in 0 seconds.2021-04-22 20:01:34,238 INFO zen.ZenModeler: Starting collector loop #001...2021-04-22 20:01:34,238 INFO zen.ZenModeler: Collecting for device 10.103.42.52021-04-22 20:01:34,239 INFO zen.ZenModeler: Got 1 devices to be scanned during collector loop #0012021-04-22 20:01:34,340 INFO zen.ZenModeler: Filled collection slots for 1 of 1 devices during collector loop #0012021-04-22 20:01:34,341 INFO zen.ZenModeler: skipping WMI-based collection, PySamba zenpack not installed2021-04-22 20:01:34,341 INFO zen.ZenModeler: Collect on device 10.103.42.5 for collector loop #0012021-04-22 20:01:34,382 INFO zen.ZenModeler: No Python plugins found for 10.103.42.52021-04-22 20:01:34,385 INFO zen.ZenModeler: No command plugins found for 10.103.42.52021-04-22 20:01:34,388 INFO zen.ZenModeler: SNMP collection device 10.103.42.52021-04-22 20:01:34,388 INFO zen.ZenModeler: plugins: atldc.snmp.VeritivDeviceMap, zenoss.snmp.NewDeviceMap, zenoss.snmp.DeviceMap, zenoss.snmp.InterfaceMap, atldc.snmp.VeritivSensorMap2021-04-22 20:01:34,392 INFO zen.ZenModeler: No portscan plugins found for 10.103.42.52021-04-22 20:01:40,024 INFO zen.SnmpClient: snmp client finished collection for 10.103.42.52021-04-22 20:01:40,025 INFO zen.ZenModeler: processing atldc.snmp.VeritivDeviceMap for device 10.103.42.52021-04-22 20:01:40,025 INFO zen.ZenModeler: Device Properties post: <ObjectMap {'classname': '', 'compname': '', 'deviceCount': 2, 'modname': 'Zenpacks.atldc.Veritiv.VeritivDevice', 'setHWModel': 'Geist WD100', 'setHWSerialNumber': 'TB19070494'}>2021-04-22 20:01:40,026 INFO zen.ZenModeler: Processing zenoss.snmp.NewDeviceMap for device 10.103.42.52021-04-22 20:01:40,027 INFO zen.ZenModeler: processing zenoss.snmp.DeviceMap for device 10.103.42.52021-04-22 20:01:40,027 INFO zen.ZenModeler: Modeler zenoss.snmp.InterfaceMap processing data for device 10.103.42.52021-04-22 20:01:40,027 WARNING zen.ZenModeler: Unable to get data for 10.103.42.5 from either ipAddrTable or ipNetToMediaTable2021-04-22 20:01:40,028 INFO zen.ZenModeler: processing atldc.snmp.VeritivSensorMap for device 10.103.42.52021-04-22 20:01:40,028 INFO zen.ZenModeler: SensorTable: {'1': {'serialNumber': 'AB801F12ACA2A9C3', 'sensorName': 'POD-4'}}2021-04-22 20:01:40,028 INFO zen.ZenModeler: Object Map: <ObjectMap {'classname': '', 'compname': '', 'id': 'POD-4', 'modname': 'Zenpacks.atldc.Veritiv.VeritivSensor', 'sensorName': 'POD-4', 'serialNumber': 'AB801F12ACA2A9C3', 'snmpindex': '1'}>2021-04-22 20:02:04,082 INFO zen.ZenModeler: Counter eventCount, value 72021-04-22 20:02:34,082 INFO zen.ZenModeler: Counter eventCount, value 132021-04-22 20:03:04,082 INFO zen.ZenModeler: Counter eventCount, value 192021-04-22 20:03:34,082 INFO zen.ZenModeler: Counter eventCount, value 252021-04-22 20:04:04,082 INFO zen.ZenModeler: Counter eventCount, value 312021-04-22 20:04:34,082 INFO zen.ZenModeler: Counter eventCount, value 372021-04-22 20:04:34,422 WARNING zen.ZenModeler: Client 10.103.42.5 timeout2021-04-22 20:04:34,423 INFO zen.ZenModeler: Scan time: 180.18 seconds for collector loop #0012021-04-22 20:04:34,423 INFO zen.ZenModeler: Scanned 1 of 1 devices during collector loop #0012021-04-22 20:04:34,424 INFO zen.ZenModeler: Daemon ZenModeler shutting down​


------------------------------
Karl Lentz
Sr. Network Engineer
ATL Data Centers LLC
College Park GA
4049611009
------------------------------

Subject: RE: Modeling times out in Custom ZenPack
Author: Karl Lentz
Posted: 2021-04-23 09:37

Thank you for the suggestion.  I tried setting the zCollectorClientTimeout to 5000 and still got the same timeout message as with the default timeout value.

So something is obviously wrong in my modeler code for the Sensor modeler but I can't figure out what the issue is.

I tried running zenmodeler with -v 10 but did not see any reported Errors.  I'm not sure if my problem is in my modeler or if the definition of the relationship in my yaml file is wrong.

------------------------------
Karl Lentz
Sr. Network Engineer
ATL Data Centers LLC
College Park GA
------------------------------

Subject: RE: Modeling times out in Custom ZenPack
Author: Karl Lentz
Posted: 2021-04-23 20:21

Disregard this, I see this message when I run zenmodeler with my modeler plugin removed.  Back to the drawing board.

So I am seeing the following error when I run zenmodeler with -v 10
2021-04-24 00:17:30,153 DEBUG zen.pbclientfactory: clientConnectionLost [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.
]
​

Any thoughts what this error means or would be causing it?



------------------------------
Karl Lentz
Sr. Network Engineer
ATL Data Centers LLC
College Park GA
------------------------------ 
from Products.DataCollector.plugins.CollectorPlugin import SnmpPlugin, GetTableMapclass VeritivSensorMap(SnmpPlugin):    relname = 'veritivSensors'    modname = 'Zenpacks.atldc.Veritiv.VeritivSensor'    snmpGetTableMaps = (        GetTableMap('internalSensorTable',                    '.1.3.6.1.4.1.21239.5.1.2.1',                    {                        '.1': 'internalIndex',                        '.2': 'serialNumber',                        '.3': 'sensorName',                    }            ),        )    def process(self, device, results, log):        log.info('processing %s for device %s', self.name(), device.id)        getdata, tabledata = results        internalSensorTable = tabledata.get('internalSensorTable', {})        log.info('SensorTable: %s', internalSensorTable)        if not internalSensorTable:            log.warn('No SNMP response from %s for the %s plugin', device.id, self.name())            log.warn('Table Data= %s', tabledata)            return        rm = self.relMap()        for oid, data in internalSensorTable.items():            try:                om = self.objectMap(data)                om.id = self.prepId(om.sensorName)                om.snmpindex = oid                log.info('Object Map: %s', om)                rm.append(om)            except (KeyError, IndexError, AttributeError, TypeError), errorInfo:                log.warn('Error in %s modeler plugin %s', self.name(), errorinfo)                continue        return rm​


And here is my yaml file:

name: ZenPacks.atldc.Veritivclasses:  VeritivDevice:    base: [zenpacklib.Device]    label: Veritiv Device    properties:      deviceCount:        label: Device Count        type: int  VeritivSensor:    base: [zenpacklib.Component]    label: Veritiv Sensor    properties:      serialNumber:        label: Serial Number      sensorName:        label: Sensor Nameclass_relationships:  - VeritivDevice(veritivSensors) 1:MC VeritivSensor(veritivDevice)device_classes:  /Network/Watchdog/Veritiv:    zProperties:      zPythonClass: ZenPacks.atldc.Veritiv.VeritivDevice      zSnmpMonitorIgnore: false      zCollectorPlugins:        - atldc.snmp.VeritivDeviceMap        - atldc.snmp.VeritivSensorMap        - zenoss.snmp.NewDeviceMap        - zenoss.snmp.DeviceMap        - zenoss.snmp.InterfaceMap

And here is the output from Zenmodeler:
[zenoss@localhost z]$ zenmodeler run --device=10.103.42.52021-04-22 20:01:34,081 INFO zen.ZenModeler: Connecting to localhost:87892021-04-22 20:01:34,107 INFO zen.ZenModeler: Connected to the zenhub/0 instance2021-04-22 20:01:34,108 INFO zen.ZenModeler: Getting configuration from ZenHub...2021-04-22 20:01:34,238 INFO zen.ZenModeler: Starting modeling in 0 seconds.2021-04-22 20:01:34,238 INFO zen.ZenModeler: Starting collector loop #001...2021-04-22 20:01:34,238 INFO zen.ZenModeler: Collecting for device 10.103.42.52021-04-22 20:01:34,239 INFO zen.ZenModeler: Got 1 devices to be scanned during collector loop #0012021-04-22 20:01:34,340 INFO zen.ZenModeler: Filled collection slots for 1 of 1 devices during collector loop #0012021-04-22 20:01:34,341 INFO zen.ZenModeler: skipping WMI-based collection, PySamba zenpack not installed2021-04-22 20:01:34,341 INFO zen.ZenModeler: Collect on device 10.103.42.5 for collector loop #0012021-04-22 20:01:34,382 INFO zen.ZenModeler: No Python plugins found for 10.103.42.52021-04-22 20:01:34,385 INFO zen.ZenModeler: No command plugins found for 10.103.42.52021-04-22 20:01:34,388 INFO zen.ZenModeler: SNMP collection device 10.103.42.52021-04-22 20:01:34,388 INFO zen.ZenModeler: plugins: atldc.snmp.VeritivDeviceMap, zenoss.snmp.NewDeviceMap, zenoss.snmp.DeviceMap, zenoss.snmp.InterfaceMap, atldc.snmp.VeritivSensorMap2021-04-22 20:01:34,392 INFO zen.ZenModeler: No portscan plugins found for 10.103.42.52021-04-22 20:01:40,024 INFO zen.SnmpClient: snmp client finished collection for 10.103.42.52021-04-22 20:01:40,025 INFO zen.ZenModeler: processing atldc.snmp.VeritivDeviceMap for device 10.103.42.52021-04-22 20:01:40,025 INFO zen.ZenModeler: Device Properties post: <ObjectMap {'classname': '', 'compname': '', 'deviceCount': 2, 'modname': 'Zenpacks.atldc.Veritiv.VeritivDevice', 'setHWModel': 'Geist WD100', 'setHWSerialNumber': 'TB19070494'}>2021-04-22 20:01:40,026 INFO zen.ZenModeler: Processing zenoss.snmp.NewDeviceMap for device 10.103.42.52021-04-22 20:01:40,027 INFO zen.ZenModeler: processing zenoss.snmp.DeviceMap for device 10.103.42.52021-04-22 20:01:40,027 INFO zen.ZenModeler: Modeler zenoss.snmp.InterfaceMap processing data for device 10.103.42.52021-04-22 20:01:40,027 WARNING zen.ZenModeler: Unable to get data for 10.103.42.5 from either ipAddrTable or ipNetToMediaTable2021-04-22 20:01:40,028 INFO zen.ZenModeler: processing atldc.snmp.VeritivSensorMap for device 10.103.42.52021-04-22 20:01:40,028 INFO zen.ZenModeler: SensorTable: {'1': {'serialNumber': 'AB801F12ACA2A9C3', 'sensorName': 'POD-4'}}2021-04-22 20:01:40,028 INFO zen.ZenModeler: Object Map: <ObjectMap {'classname': '', 'compname': '', 'id': 'POD-4', 'modname': 'Zenpacks.atldc.Veritiv.VeritivSensor', 'sensorName': 'POD-4', 'serialNumber': 'AB801F12ACA2A9C3', 'snmpindex': '1'}>2021-04-22 20:02:04,082 INFO zen.ZenModeler: Counter eventCount, value 72021-04-22 20:02:34,082 INFO zen.ZenModeler: Counter eventCount, value 132021-04-22 20:03:04,082 INFO zen.ZenModeler: Counter eventCount, value 192021-04-22 20:03:34,082 INFO zen.ZenModeler: Counter eventCount, value 252021-04-22 20:04:04,082 INFO zen.ZenModeler: Counter eventCount, value 312021-04-22 20:04:34,082 INFO zen.ZenModeler: Counter eventCount, value 372021-04-22 20:04:34,422 WARNING zen.ZenModeler: Client 10.103.42.5 timeout2021-04-22 20:04:34,423 INFO zen.ZenModeler: Scan time: 180.18 seconds for collector loop #0012021-04-22 20:04:34,423 INFO zen.ZenModeler: Scanned 1 of 1 devices during collector loop #0012021-04-22 20:04:34,424 INFO zen.ZenModeler: Daemon ZenModeler shutting down​


------------------------------
Karl Lentz
Sr. Network Engineer
ATL Data Centers LLC
College Park GA
4049611009
------------------------------


< Previous
Modelling a list of devices 		  Next
ContextLoader error during zeneventserver start 		>