Something is wrong in your modeler code but we can't tell what from what you have posted.
zenmodeler run -v 10 -d <the name of a test device> --collect=<the name of your modeler> > /tmp/zenmodeler.out 2>&1
The bit on the end " > /tmp/zenmodeler.out 2>&1" sends the output to the file so you can then inspect the detail.
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-15 20:38
Thanks Jane.
And I use the command, the result is as follows. There's no ERROR in log, just always print the same message. Could you please help to find what's wrong? Thanks very much!
And before the log, my modeler plugin already excute successfully. Just always print "Counter eventCount".
2019-01-16 08:26:27,537 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627187.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:27,538 DEBUG zen.publisher: writing: {"timestamp":1547627187.53767,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:27,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627187.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:27,538 DEBUG zen.publisher: writing: {'timestamp': 1547627187.537672, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:27,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:27,538 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:27,538 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:28,484 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:28,484 DEBUG zen.publisher: flushing 1 metrics, current batch size 2
2019-01-16 08:26:28,485 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:28,485 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:28,488 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:28,489 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:28,490 DEBUG zen.publisher: connection closed
2019-01-16 08:26:28,490 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:32,483 DEBUG zen.ZenModeler: _checkZenHub: entry
2019-01-16 08:26:32,484 DEBUG zen.ZenModeler: ZenHub health check: perspective found. attempting remote ping call.
2019-01-16 08:26:32,485 DEBUG zen.ZenModeler: ZenHub health check: Got result pong
2019-01-16 08:26:32,485 DEBUG zen.ZenModeler: ZenHub health check: Success - received pong from ZenHub ping service.
2019-01-16 08:26:32,485 DEBUG zen.ZenModeler: _signalZenHubAnswering(True)
2019-01-16 08:26:32,486 DEBUG zen.ZenModeler: writing file at /opt/zenoss/var/zenhub_connected
2019-01-16 08:26:32,487 INFO zen.ZenModeler: Counter eventCount, value 6########
2019-01-16 08:26:32,487 DEBUG zen.MetricWriter: publishing metric modeledDevicesCount 0 1547627192.49 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:32,487 DEBUG zen.publisher: writing: {"timestamp":1547627192.48756,"metric":"modeledDevicesCount","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:32,487 DEBUG zen.MetricWriter: publishing metric modeledDevicesCount 0 1547627192.49 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:32,488 DEBUG zen.publisher: writing: {'timestamp': 1547627192.487562, 'metric': 'modeledDevicesCount', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:32,488 DEBUG zen.metricreporter: metric info: collectordaemon.eventTimer, <metrology.instruments.timer.Timer object at 0x822a650>
2019-01-16 08:26:32,488 DEBUG zen.metricreporter: metric info: zenmodeler.failures, <metrology.instruments.counter.Counter object at 0x8252a90>
2019-01-16 08:26:32,488 DEBUG zen.metricreporter: metric info: collectordaemon.discardedEvent, <metrology.instruments.meter.Meter object at 0x820abd0>
2019-01-16 08:26:32,489 DEBUG zen.metricreporter: metric info: collectordaemon.eventsSent, <metrology.instruments.meter.Meter object at 0x81fc850>
2019-01-16 08:26:32,489 DEBUG zen.metricreporter: metric info: zenmodeler.modeledDevices, <metrology.instruments.meter.Meter object at 0x820a910>
2019-01-16 08:26:32,489 DEBUG zen.metricreporter: metric info: collectordaemon.eventQueue, <Products.ZenHub.PBDaemon.EventQueueGauge object at 0x82524d0>
2019-01-16 08:26:32,489 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.count 2 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,489 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.count","value":2.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,489 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.count 2 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,489 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.count', 'value': 2.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,490 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.one_minute_rate 0.14430202054 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,490 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.one_minute_rate","value":0.1443,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,490 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.one_minute_rate 0.14430202054 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,490 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.one_minute_rate', 'value': 0.14430202053961189, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,490 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.five_minute_rate 0.18715337082 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,490 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.five_minute_rate","value":0.18715,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,490 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.five_minute_rate 0.18715337082 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,491 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.five_minute_rate', 'value': 0.1871533708196839, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,491 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.fifteen_minute_rate 0.195610611691 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,491 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.fifteen_minute_rate","value":0.19561,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,491 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.fifteen_minute_rate 0.195610611691 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,491 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.fifteen_minute_rate', 'value': 0.19561061169067304, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,491 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.mean_rate 0.0655984163807 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,491 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.mean_rate","value":0.0656,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,491 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.mean_rate 0.0655984163807 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,492 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.mean_rate', 'value': 0.0655984163806986, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,492 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.min 17 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,492 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.min","value":17.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,492 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.min 17 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,492 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.min', 'value': 17.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,492 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.max 26 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,492 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.max","value":26.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,493 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.max 26 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,493 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.max', 'value': 26.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,493 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.mean 21.5 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,493 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.mean","value":21.5,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,493 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.mean 21.5 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,493 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.mean', 'value': 21.5, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,493 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.stddev 6.32455532034 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,493 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.stddev","value":6.32456,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,494 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.stddev 6.32455532034 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,494 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.stddev', 'value': 6.324555320336759, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,494 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.median 21.5 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,494 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.median","value":21.5,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,494 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.median 21.5 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,494 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.median', 'value': 21.5, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,494 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.percentile_95th 26 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,494 DEBUG zen.publisher: writing: {"timestamp":1547627192.48843,"metric":"collectordaemon.eventTimer.percentile_95th","value":26.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,495 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventTimer.percentile_95th 26 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,495 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488428, 'metric': 'collectordaemon.eventTimer.percentile_95th', 'value': 26.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,495 DEBUG zen.MetricWriter: publishing metric zenmodeler.failures.count 0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,495 DEBUG zen.publisher: writing: {"timestamp":1547627192.4888,"metric":"zenmodeler.failures.count","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,495 DEBUG zen.MetricWriter: publishing metric zenmodeler.failures.count 0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,495 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488797, 'metric': 'zenmodeler.failures.count', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,495 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.count 0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,496 DEBUG zen.publisher: writing: {"timestamp":1547627192.4889,"metric":"collectordaemon.discardedEvent.count","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,496 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.count 0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,496 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488904, 'metric': 'collectordaemon.discardedEvent.count', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,496 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.one_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,496 DEBUG zen.publisher: writing: {"timestamp":1547627192.4889,"metric":"collectordaemon.discardedEvent.one_minute_rate","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,496 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.one_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,496 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488904, 'metric': 'collectordaemon.discardedEvent.one_minute_rate', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,497 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.five_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,497 DEBUG zen.publisher: writing: {"timestamp":1547627192.4889,"metric":"collectordaemon.discardedEvent.five_minute_rate","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,497 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.five_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,497 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488904, 'metric': 'collectordaemon.discardedEvent.five_minute_rate', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,497 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.fifteen_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,497 DEBUG zen.publisher: writing: {"timestamp":1547627192.4889,"metric":"collectordaemon.discardedEvent.fifteen_minute_rate","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,497 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.fifteen_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,497 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488904, 'metric': 'collectordaemon.discardedEvent.fifteen_minute_rate', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,498 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.mean_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,498 DEBUG zen.publisher: writing: {"timestamp":1547627192.4889,"metric":"collectordaemon.discardedEvent.mean_rate","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,498 DEBUG zen.MetricWriter: publishing metric collectordaemon.discardedEvent.mean_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,498 DEBUG zen.publisher: writing: {'timestamp': 1547627192.488904, 'metric': 'collectordaemon.discardedEvent.mean_rate', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,498 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.count 2 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,498 DEBUG zen.publisher: writing: {"timestamp":1547627192.48913,"metric":"collectordaemon.eventsSent.count","value":2.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,498 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.count 2 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,498 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489127, 'metric': 'collectordaemon.eventsSent.count', 'value': 2.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,499 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.one_minute_rate 0.14430202054 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,499 DEBUG zen.publisher: writing: {"timestamp":1547627192.48913,"metric":"collectordaemon.eventsSent.one_minute_rate","value":0.1443,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,499 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.one_minute_rate 0.14430202054 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,499 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489127, 'metric': 'collectordaemon.eventsSent.one_minute_rate', 'value': 0.14430202053961189, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,499 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.five_minute_rate 0.18715337082 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,500 DEBUG zen.publisher: writing: {"timestamp":1547627192.48913,"metric":"collectordaemon.eventsSent.five_minute_rate","value":0.18715,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,500 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.five_minute_rate 0.18715337082 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,500 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489127, 'metric': 'collectordaemon.eventsSent.five_minute_rate', 'value': 0.1871533708196839, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,500 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.fifteen_minute_rate 0.195610611691 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,500 DEBUG zen.publisher: writing: {"timestamp":1547627192.48913,"metric":"collectordaemon.eventsSent.fifteen_minute_rate","value":0.19561,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,500 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.fifteen_minute_rate 0.195610611691 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,500 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489127, 'metric': 'collectordaemon.eventsSent.fifteen_minute_rate', 'value': 0.19561061169067304, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,500 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.mean_rate 0.0655970067537 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,501 DEBUG zen.publisher: writing: {"timestamp":1547627192.48913,"metric":"collectordaemon.eventsSent.mean_rate","value":0.0656,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,501 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventsSent.mean_rate 0.0655970067537 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,501 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489127, 'metric': 'collectordaemon.eventsSent.mean_rate', 'value': 0.0655970067537108, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,501 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.count 0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,501 DEBUG zen.publisher: writing: {"timestamp":1547627192.4893,"metric":"zenmodeler.modeledDevices.count","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,501 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.count 0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,501 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489298, 'metric': 'zenmodeler.modeledDevices.count', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,502 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.one_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,502 DEBUG zen.publisher: writing: {"timestamp":1547627192.4893,"metric":"zenmodeler.modeledDevices.one_minute_rate","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,502 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.one_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,502 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489298, 'metric': 'zenmodeler.modeledDevices.one_minute_rate', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,502 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.five_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,502 DEBUG zen.publisher: writing: {"timestamp":1547627192.4893,"metric":"zenmodeler.modeledDevices.five_minute_rate","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,502 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.five_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,502 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489298, 'metric': 'zenmodeler.modeledDevices.five_minute_rate', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,503 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.fifteen_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,503 DEBUG zen.publisher: writing: {"timestamp":1547627192.4893,"metric":"zenmodeler.modeledDevices.fifteen_minute_rate","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,503 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.fifteen_minute_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,503 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489298, 'metric': 'zenmodeler.modeledDevices.fifteen_minute_rate', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,504 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.mean_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,504 DEBUG zen.publisher: writing: {"timestamp":1547627192.4893,"metric":"zenmodeler.modeledDevices.mean_rate","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,504 DEBUG zen.MetricWriter: publishing metric zenmodeler.modeledDevices.mean_rate 0.0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,504 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489298, 'metric': 'zenmodeler.modeledDevices.mean_rate', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,504 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventQueue.value 0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,504 DEBUG zen.publisher: writing: {"timestamp":1547627192.48948,"metric":"collectordaemon.eventQueue.value","value":0.0,"tags":{"zenoss_monitor":"localhost","zenoss_daemon":"zenmodeler"}}
2019-01-16 08:26:32,504 DEBUG zen.MetricWriter: publishing metric collectordaemon.eventQueue.value 0 1547627192.49 {'zenoss_monitor': 'localhost', 'internal': True, 'zenoss_daemon': 'zenmodeler'}
2019-01-16 08:26:32,505 DEBUG zen.publisher: writing: {'timestamp': 1547627192.489477, 'metric': 'collectordaemon.eventQueue.value', 'value': 0.0, 'tags': {'zenoss_monitor': 'localhost', 'zenoss_daemon': 'zenmodeler'}}
2019-01-16 08:26:32,537 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627192.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:32,538 DEBUG zen.publisher: writing: {"timestamp":1547627192.53783,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:32,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627192.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:32,538 DEBUG zen.publisher: writing: {'timestamp': 1547627192.537829, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:32,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:32,538 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:32,538 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:33,486 DEBUG zen.publisher: trying to publish 30 metrics
2019-01-16 08:26:33,487 DEBUG zen.publisher: flushing 2 metrics, current batch size 2
2019-01-16 08:26:33,487 DEBUG zen.publisher: trying to publish 30 metrics
2019-01-16 08:26:33,487 DEBUG zen.publisher: Posting 30 metrics
2019-01-16 08:26:33,491 DEBUG zen.publisher: published 30 metrics and received response: 200
2019-01-16 08:26:33,491 DEBUG zen.publisher: connection closed
2019-01-16 08:26:33,491 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:33,491 DEBUG zen.publisher: published 2 metrics to redis
2019-01-16 08:26:33,492 DEBUG zen.publisher: trying to publish 28 metrics
2019-01-16 08:26:33,492 DEBUG zen.publisher: flushing 4 metrics, current batch size 4
2019-01-16 08:26:33,494 DEBUG zen.publisher: published 4 metrics to redis
2019-01-16 08:26:33,494 DEBUG zen.publisher: trying to publish 24 metrics
2019-01-16 08:26:33,494 DEBUG zen.publisher: flushing 8 metrics, current batch size 8
2019-01-16 08:26:33,496 DEBUG zen.publisher: published 8 metrics to redis
2019-01-16 08:26:33,496 DEBUG zen.publisher: trying to publish 16 metrics
2019-01-16 08:26:33,496 DEBUG zen.publisher: flushing 16 metrics, current batch size 16
2019-01-16 08:26:33,498 DEBUG zen.publisher: published 16 metrics to redis
2019-01-16 08:26:37,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627197.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:37,538 DEBUG zen.publisher: writing: {"timestamp":1547627197.538,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:37,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627197.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:37,538 DEBUG zen.publisher: writing: {'timestamp': 1547627197.537999, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:37,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:37,538 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:37,539 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:38,491 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:38,491 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:38,494 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:38,494 DEBUG zen.publisher: connection closed
2019-01-16 08:26:38,495 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:38,496 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:38,496 DEBUG zen.publisher: flushing 1 metrics, current batch size 32
2019-01-16 08:26:38,498 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:42,537 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627202.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:42,538 DEBUG zen.publisher: writing: {"timestamp":1547627202.53792,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:42,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627202.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:42,538 DEBUG zen.publisher: writing: {'timestamp': 1547627202.537924, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:42,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:42,538 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:42,538 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:43,492 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:43,492 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:43,495 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:43,495 DEBUG zen.publisher: connection closed
2019-01-16 08:26:43,495 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:43,496 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:43,496 DEBUG zen.publisher: flushing 1 metrics, current batch size 32
2019-01-16 08:26:43,499 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:47,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627207.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:47,538 DEBUG zen.publisher: writing: {"timestamp":1547627207.53814,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:47,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627207.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:47,538 DEBUG zen.publisher: writing: {'timestamp': 1547627207.538135, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:47,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:47,539 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:47,539 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:48,493 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:48,493 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:48,496 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:48,496 DEBUG zen.publisher: connection closed
2019-01-16 08:26:48,496 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:48,497 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:48,497 DEBUG zen.publisher: flushing 1 metrics, current batch size 32
2019-01-16 08:26:48,499 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:52,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627212.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:52,538 DEBUG zen.publisher: writing: {"timestamp":1547627212.53813,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:52,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627212.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:52,538 DEBUG zen.publisher: writing: {'timestamp': 1547627212.538134, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:52,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:52,539 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:52,539 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:53,496 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:53,496 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:53,497 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:53,497 DEBUG zen.publisher: flushing 1 metrics, current batch size 32
2019-01-16 08:26:53,501 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:53,501 DEBUG zen.publisher: connection closed
2019-01-16 08:26:53,501 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:53,501 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:57,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627217.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:57,538 DEBUG zen.publisher: writing: {"timestamp":1547627217.53825,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:57,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627217.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:57,538 DEBUG zen.publisher: writing: {'timestamp': 1547627217.538255, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:57,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:57,539 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:57,539 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:58,498 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:58,498 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:58,499 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:58,499 DEBUG zen.publisher: flushing 1 metrics, current batch size 32
2019-01-16 08:26:58,501 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:58,502 DEBUG zen.publisher: connection closed
2019-01-16 08:26:58,502 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:58,502 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:27:02,483 DEBUG zen.ZenModeler: _checkZenHub: entry
2019-01-16 08:27:02,483 DEBUG zen.ZenModeler: ZenHub health check: perspective found. attempting remote ping call.
2019-01-16 08:27:02,485 DEBUG zen.ZenModeler: ZenHub health check: Got result pong
2019-01-16 08:27:02,485 DEBUG zen.ZenModeler: ZenHub health check: Success - received pong from ZenHub ping service.
2019-01-16 08:27:02,485 DEBUG zen.ZenModeler: _signalZenHubAnswering(True)
2019-01-16 08:27:02,485 DEBUG zen.ZenModeler: writing file at /opt/zenoss/var/zenhub_connected
2019-01-16 08:27:02,487 INFO zen.ZenModeler: Counter eventCount, value 12########
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Jane Curry
Posted: 2019-01-16 04:07
I can't spot anything in that lot that is actually running any custom modeler plugin. Go back to basics.
What version of Zenoss are you running?
Do you have separate collector Zenoss systems or a single Zenoss server?
What are you actually tring to model and how are you going about that?
Cheers,
Jane
------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-16 04:32
1.Zenoss version: 6.2.1
2.above log I just pick the log after plugin excute finished, and always run Counter eventCount.
3.zenoss run in a single vm (memory > 32G)
4. I am modeling huawei storage. use HTTP API.
more log:
2019-01-16 08:26:02,483 DEBUG zen.ZenModeler: _checkZenHub: entry
2019-01-16 08:26:02,483 DEBUG zen.ZenModeler: ZenHub health check: ZenHub may be down.
2019-01-16 08:26:02,483 DEBUG zen.ZenModeler: _signalZenHubAnswering(False)
2019-01-16 08:26:02,484 DEBUG zen.ZenModeler: removing file at /opt/zenoss/var/zenhub_connected
2019-01-16 08:26:02,484 DEBUG zen.ZenModeler: ignoring <type 'exceptions.OSError'> exception () removing file /opt/zenoss/var/zenhub_connected
2019-01-16 08:26:02,485 DEBUG zen.ZenModeler: Run in foreground, starting immediately.
2019-01-16 08:26:02,486 DEBUG zen.ZenModeler: Starting PBDaemon initialization
2019-01-16 08:26:02,486 INFO zen.ZenModeler: Connecting to localhost:8789
2019-01-16 08:26:02,486 DEBUG zen.pbclientfactory: Starting connection...
2019-01-16 08:26:02,486 DEBUG zen.ZenModeler: Logging in as admin
2019-01-16 08:26:02,487 DEBUG zen.ZenModeler: Starting Statistic posting
2019-01-16 08:26:02,521 DEBUG zen.pbclientfactory: clientConnectionMade
2019-01-16 08:26:02,521 DEBUG zen.pbclientfactory: Cancelling connect timeout
2019-01-16 08:26:02,521 DEBUG zen.pbclientfactory: _login called
2019-01-16 08:26:02,524 DEBUG zen.pbclientfactory: Cancelling connect timeout
2019-01-16 08:26:02,525 DEBUG zen.ZenModeler: Setting up initial services: EventService, ModelerService
2019-01-16 08:26:02,528 DEBUG zen.ZenModeler: Chaining getInitialServices with d2
2019-01-16 08:26:02,529 DEBUG zen.pbclientfactory: pinging perspective
2019-01-16 08:26:02,535 INFO zen.ZenModeler: Connected to the zenhub/0 instance
2019-01-16 08:26:02,535 DEBUG zen.ZenModeler: Loaded service EventService from zenhub
2019-01-16 08:26:02,535 DEBUG zen.ZenModeler: Loaded service ModelerService from zenhub
2019-01-16 08:26:02,536 DEBUG zen.ZenModeler: Queued event (total of 1) {'rcvtime': 1547627162.536054, 'severity': 0, 'component': 'zenmodeler', 'agent': 'zenmodeler', 'summary': 'started', 'manager': '6510372b2126', 'device': 'localhost', 'eventClass': '/App/Start', 'monitor': 'localhost'}
2019-01-16 08:26:02,536 DEBUG zen.ZenModeler: Sending 1 events, 0 perf events, 0 heartbeats
2019-01-16 08:26:02,537 DEBUG zen.ZenModeler: Calling connected.
2019-01-16 08:26:02,537 INFO zen.ZenModeler: Getting configuration from ZenHub...
2019-01-16 08:26:02,537 DEBUG zen.ZenModeler: fetching monitor properties
2019-01-16 08:26:02,537 DEBUG zen.pbclientfactory: perspective ponged
2019-01-16 08:26:02,537 DEBUG zen.pbclientfactory: Cancelling ping timeout
2019-01-16 08:26:02,550 DEBUG zen.ZenModeler: Getting threshold classes...
2019-01-16 08:26:02,563 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627162.56 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:02,563 DEBUG zen.publisher: writing: {"timestamp":1547627162.56322,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:02,563 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627162.56 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:02,563 DEBUG zen.publisher: writing: {'timestamp': 1547627162.563221, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:02,602 DEBUG zen.ZenModeler: Loading classes ['Products.ZenModel.MinMaxThreshold', 'Products.ZenModel.ValueChangeThreshold']
2019-01-16 08:26:02,605 DEBUG zen.ZenModeler: Getting collector thresholds...
2019-01-16 08:26:02,733 DEBUG zen.thresholds: Updating threshold ('high event queue', ('localhost collector', ''))
2019-01-16 08:26:02,734 DEBUG zen.thresholds: Updating threshold ('zenmodeler cycle time', ('localhost collector', ''))
2019-01-16 08:26:02,734 DEBUG zen.ZenModeler: Getting collector plugins for each DeviceClass
2019-01-16 08:26:03,005 INFO zen.ZenModeler: Starting modeling in 0 seconds.
2019-01-16 08:26:03,005 INFO zen.ZenModeler: Starting collector loop #001...
2019-01-16 08:26:03,005 INFO zen.ZenModeler: Collecting for device abc
2019-01-16 08:26:03,005 DEBUG zen.ZenModeler: getDeviceList returned 1 devices
2019-01-16 08:26:03,005 DEBUG zen.ZenModeler: getDeviceList returned ['abc'] devices
2019-01-16 08:26:03,005 INFO zen.ZenModeler: Got 1 devices to be scanned during collector loop #001
2019-01-16 08:26:03,564 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:03,564 DEBUG zen.publisher: flushing 1 metrics, current batch size 2
2019-01-16 08:26:03,564 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:03,564 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:03,568 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:03,569 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:03,570 DEBUG zen.publisher: connection closed
2019-01-16 08:26:03,570 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:07,537 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627167.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:07,538 DEBUG zen.publisher: writing: {"timestamp":1547627167.53763,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:07,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627167.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:07,538 DEBUG zen.publisher: writing: {'timestamp': 1547627167.537634, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:07,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:07,538 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:07,539 DEBUG zen.ZenModeler: Queued event (total of 1) {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'rcvtime': 1547627167.538911, 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:07,564 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:07,565 DEBUG zen.publisher: flushing 1 metrics, current batch size 2
2019-01-16 08:26:07,565 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:07,565 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:07,568 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:07,569 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:07,569 DEBUG zen.publisher: connection closed
2019-01-16 08:26:07,569 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:12,537 DEBUG zen.ZenModeler: Sending 1 events, 0 perf events, 0 heartbeats
2019-01-16 08:26:12,555 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627172.56 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:12,556 DEBUG zen.publisher: writing: {"timestamp":1547627172.5557,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:12,556 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627172.56 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:12,556 DEBUG zen.publisher: writing: {'timestamp': 1547627172.555699, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:12,556 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:12,556 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:12,557 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:12,565 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:12,565 DEBUG zen.publisher: flushing 1 metrics, current batch size 2
2019-01-16 08:26:12,566 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:12,566 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:12,569 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:12,570 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:12,570 DEBUG zen.publisher: connection closed
2019-01-16 08:26:12,570 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:17,537 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627177.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:17,538 DEBUG zen.publisher: writing: {"timestamp":1547627177.53778,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:17,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627177.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:17,538 DEBUG zen.publisher: writing: {'timestamp': 1547627177.537781, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:17,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:17,539 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:17,539 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:17,566 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:17,566 DEBUG zen.publisher: flushing 1 metrics, current batch size 2
2019-01-16 08:26:17,567 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:17,567 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:17,569 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:17,570 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:17,570 DEBUG zen.publisher: connection closed
2019-01-16 08:26:17,570 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:22,537 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627182.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:22,537 DEBUG zen.publisher: writing: {"timestamp":1547627182.5376,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:22,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627182.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:22,538 DEBUG zen.publisher: writing: {'timestamp': 1547627182.537602, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:22,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:22,538 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:22,538 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:22,567 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:22,567 DEBUG zen.publisher: flushing 1 metrics, current batch size 2
2019-01-16 08:26:22,568 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:22,568 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:22,570 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:22,571 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:22,571 DEBUG zen.publisher: connection closed
2019-01-16 08:26:22,571 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:23,211 INFO zen.ZenModeler: Filled collection slots for 1 of 1 devices during collector loop #001
2019-01-16 08:26:23,211 INFO zen.ZenModeler: skipping WMI-based collection, PySamba zenpack not installed
2019-01-16 08:26:23,211 INFO zen.ZenModeler: Collect on device abc for collector loop #001
2019-01-16 08:26:23,213 DEBUG zen.ZenModeler: Loaded plugin community.api.hwoceanstor
2019-01-16 08:26:23,213 DEBUG zen.ZenModeler: Using community.api.hwoceanstor on abc because of --collect flag
2019-01-16 08:26:23,214 INFO zen.ZenModeler: Python collection device abc
2019-01-16 08:26:23,214 INFO zen.ZenModeler: plugins: community.api.hwoceanstor
2019-01-16 08:26:23,214 DEBUG zen.PythonClient: Running collection for plugin community.api.hwoceanstor
2019-01-16 08:26:23,215 INFO zen.PythonClient: abc: collecting data
2019-01-16 08:26:23,215 INFO zen.OceanStor: Login array ips: ["x.x.x.x"].
2019-01-16 08:26:23,215 INFO zen.OceanStor: Login ip: x.x.x.x.
2019-01-16 08:26:23,216 INFO zen.OceanStor: Request url: https://x.x.x.x:8088/rest/xxxxx/sessions.
2019-01-16 08:26:24,415 INFO zen.OceanStor: Login success: x.x.x.x.
2019-01-16 08:26:24,415 INFO zen.OceanStor: Request url: https://x.x.x.x:8088/rest/xxx/system/.
2019-01-16 08:26:24,463 INFO zen.PythonClient: ##########: <RelationshipMap {'compname': '',
'objmaps': [<ObjectMap {'classname': '',
'compname': '',
'deviceId': u'123456789',
'deviceStatus': u'1',
'deviceTotalCapacity': u'1234',
'deviceType': 1,
'deviceUsedCapacity': u'123',
'deviceVersion': u'xxx',
'id': 'Huawei.Storage',
'modname': 'ZenPacks.community.OceanStor.OceanStorArray',
'product_id': u'xxxx',
'title': u'Storage'}>],
'parentId': '',
'relname': 'oceanStorArrays'}>
2019-01-16 08:26:27,537 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627187.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:27,538 DEBUG zen.publisher: writing: {"timestamp":1547627187.53767,"metric":"eventQueueLength","value":0.0,"tags":{"instance":"0","daemon":"zenmodeler","monitor":"localhost","metricType":"GAUGE","tenantId":"ad8cv42hf9205ex0pusq6lo5v"}}
2019-01-16 08:26:27,538 DEBUG zen.MetricWriter: publishing metric eventQueueLength 0 1547627187.54 {'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v', 'instance': '0', 'internal': True}
2019-01-16 08:26:27,538 DEBUG zen.publisher: writing: {'timestamp': 1547627187.537672, 'metric': 'eventQueueLength', 'value': 0.0, 'tags': {'instance': '0', 'daemon': 'zenmodeler', 'monitor': 'localhost', 'metricType': 'GAUGE', 'tenantId': 'ad8cv42hf9205ex0pusq6lo5v'}}
2019-01-16 08:26:27,538 DEBUG zen.thresholds: Checking value 0 on Daemons/localhost/zenmodeler_eventQueueLength
2019-01-16 08:26:27,538 DEBUG zen.MinMaxCheck: Checking zenmodeler_eventQueueLength 0 against min None and max 1000
2019-01-16 08:26:27,538 DEBUG zen.ZenModeler: allowduplicateclears dropping clear event {'zenoss.device.url': 'zport/dmd/Monitors/Performance/localhost/viewDaemonPerformance', 'zenoss.device.path': 'Monitors/Performance/localhost', 'severity': 0, 'min': None, 'max': 1000, 'component': '', 'agent': 'zenmodeler', 'summary': 'threshold of high event queue restored: current value 0.000000', 'current': 0, 'manager': '6510372b2126', 'eventKey': 'zenmodeler_eventQueueLength|high event queue', 'device': 'localhost collector', 'eventClass': '/Perf', 'monitor': 'localhost'}
2019-01-16 08:26:28,484 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:28,484 DEBUG zen.publisher: flushing 1 metrics, current batch size 2
2019-01-16 08:26:28,485 DEBUG zen.publisher: trying to publish 1 metrics
2019-01-16 08:26:28,485 DEBUG zen.publisher: Posting 1 metrics
2019-01-16 08:26:28,488 DEBUG zen.publisher: published 1 metrics to redis
2019-01-16 08:26:28,489 DEBUG zen.publisher: published 1 metrics and received response: 200
2019-01-16 08:26:28,490 DEBUG zen.publisher: connection closed
2019-01-16 08:26:28,490 DEBUG zen.publisher: response was: {"type":"OK","value":""}
2019-01-16 08:26:32,483 DEBUG zen.ZenModeler: _checkZenHub: entry
2019-01-16 08:26:32,484 DEBUG zen.ZenModeler: ZenHub health check: perspective found. attempting remote ping call.
2019-01-16 08:26:32,485 DEBUG zen.ZenModeler: ZenHub health check: Got result pong
2019-01-16 08:26:32,485 DEBUG zen.ZenModeler: ZenHub health check: Success - received pong from ZenHub ping service.
2019-01-16 08:26:32,485 DEBUG zen.ZenModeler: _signalZenHubAnswering(True)
2019-01-16 08:26:32,486 DEBUG zen.ZenModeler: writing file at /opt/zenoss/var/zenhub_connected
2019-01-16 08:26:32,487 INFO zen.ZenModeler: Counter eventCount, value 6########
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-16 04:25
Hi Jane,
Today I try to move the code（collect data） from collect to process function. It works!!!
I didn't delete collect function, but Model Device there's no collect function logs, maybe doesn't entry collect. I don't know why.
Here's my collect function. Look forward to your reply. Thanks!
--------------------------------------------------
def collect(self, device, log):
client = Client(controllers, user, password)
response = None
try:
response = client.get_array()
log.info("##########: %s", response)
except Exception, e:
log.error('%s: %s', device.id, e)
returnValue(None)
returnValue(response)
--------------------------------------------------
PS
1.I didn't use getPage, and use requests.
2.I didn't use @inlineCallbacks, because I met a problem as follows and don't know how to return a generator value.
ERROR zen.ZenModeler: Unable to fill collection slots: [Failure instance: Traceback (failure with no frames): <type 'exceptions.TypeError'>: inlineCallbacks requires <function collect at 0x9dca938> to produce a generator; insteadcaught returnValue being used in a non-generator
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Jane Curry
Posted: 2019-01-16 05:02
So it is working now and you are creating your new component?
This is the part of the log to look for (and it looks good as far as I can tell):
2019-01-16 08:26:23,213 DEBUG zen.ZenModeler: Loaded plugin community.api.hwoceanstor
2019-01-16 08:26:23,213 DEBUG zen.ZenModeler: Using community.api.hwoceanstor on abc because of --collect flag
2019-01-16 08:26:23,214 INFO zen.ZenModeler: Python collection device abc
2019-01-16 08:26:23,214 INFO zen.ZenModeler: plugins: community.api.hwoceanstor
2019-01-16 08:26:23,214 DEBUG zen.PythonClient: Running collection for plugin community.api.hwoceanstor
2019-01-16 08:26:23,215 INFO zen.PythonClient: abc: collecting data
2019-01-16 08:26:23,215 INFO zen.OceanStor: Login array ips: ["x.x.x.x"].
2019-01-16 08:26:23,215 INFO zen.OceanStor: Login ip: x.x.x.x.
2019-01-16 08:26:23,216 INFO zen.OceanStor: Request url: https://x.x.x.x:8088/rest/xxxxx/sessions.
2019-01-16 08:26:24,415 INFO zen.OceanStor: Login success: x.x.x.x.
2019-01-16 08:26:24,415 INFO zen.OceanStor: Request url: https://x.x.x.x:8088/rest/xxx/system/.
2019-01-16 08:26:24,463 INFO zen.PythonClient: ##########: <RelationshipMap {'compname': '',
'objmaps': [<ObjectMap {'classname': '',
'compname': '',
'deviceId': u'123456789',
'deviceStatus': u'1',
'deviceTotalCapacity': u'1234',
'deviceType': 1,
'deviceUsedCapacity': u'123',
'deviceVersion': u'xxx',
'id': 'Huawei.Storage',
'modname': 'ZenPacks.community.OceanStor.OceanStorArray',
'product_id': u'xxxx',
'title': u'Storage'}>],
'parentId': '',
'relname': 'oceanStorArrays'}>
I assume you have substituted the x.x.x.x s to keep confidentiality?
Cheers,
Jane
------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-16 05:08
yes.
but I also want use collect and process function do seperate things. do you know what's the problem?
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-16 05:10
If I use requests to collect data in collect function. then will met 'always Counter eventCount' problem.
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Jane Curry
Posted: 2019-01-16 08:19
Can Isuggest you look at the ZenPack Developers' Guide that you can get from here -
https://github.com/ZenossDevGuide/DevGuide - there is a detailed section on developing modelers and the different elements like collect and process.
Modeling is typically a periodic process - it is controlled from ADVANCED -> Collectors and select localhost - the default Modeler Cycle is 720 minutes (12 hours). With Zenoss 6 you also get the option to set when th modeler cycle runs in addition to how frequently.
Component graphs are definitely supported by Zenoss Core as well as Resource Manager.
Cheers,
Jane
------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-16 22:22
Got it, thank you very much!
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-23 06:39
Hi Jane,
Now I was developing component graph using datasource plugin. And the component graph also has not data.
And I use "zenpython run -v10 --device=xxx" to debug, got the following message:
2019-01-23 18:27:22,350 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from QUEUED to RUNNING
2019-01-23 18:27:22,350 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from RUNNING to BLOCKING
2019-01-23 18:27:30,145 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from BLOCKING to RUNNING
2019-01-23 18:27:30,145 WARNING zen.python: Task x.x.x.x 300 sample-Statistics blocked for 7.80 seconds in collect
2019-01-23 18:27:30,145 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from RUNNING to BLOCKING
2019-01-23 18:27:30,146 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from BLOCKING to RUNNING
2019-01-23 18:27:30,146 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from RUNNING to BLOCKING
2019-01-23 18:27:30,146 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from BLOCKING to RUNNING
2019-01-23 18:27:30,146 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from RUNNING to BLOCKING
2019-01-23 18:27:30,146 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from BLOCKING to RUNNING
2019-01-23 18:27:30,146 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from RUNNING to SEND_EVENTS
2019-01-23 18:27:30,147 DEBUG zen.zenpython: Queued event (total of 5) {'rcvtime': 1548268050.14713, 'severity': 0, 'eventClassKey': 'zenpython-timeout', 'component': '', 'monitor': 'localhost', 'agent': 'zenpython', 'summary': 'timeout collecting statistics datasource for multiple components', 'manager': 'f8738fddd418', 'eventKey': 'zenpython-timeout|x.x.x.x 300 sample-Statistics', 'components': 'Disk0.0,Disk0.1,Disk0.10,Disk0.2,Disk0.3,Disk0.4,Disk0.5,Disk0.6,Disk0.7,Disk0.8,Disk0.9', 'device': 'x.x.x.x', 'device_guid': 'a56ec813-2f56-4d5f-b51e-102a83b04c46', 'datasources': 'statistics'}
2019-01-23 18:27:30,147 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from SEND_EVENTS to STORE_PERF_DATA
2019-01-23 18:27:30,147 DEBUG zen.MetricWriter: publishing metric x.x.x.x/statistics_iops 2.0 1548268050 {'device': 'x.x.x.x', 'contextUUID': '99c324bf-4d1e-4a9f-a1e5-1f0e08b8b2c8', 'key': 'Devices/x.x.x.x/Disks/Disk0.0'}
2019-01-23 18:27:30,148 DEBUG zen.publisher: writing: {"timestamp":1548268050,"metric":"x.x.x.x\/statistics_iops","value":2.0,"tags":{"device":"x.x.x.x","contextUUID":"99c324bf-4d1e-4a9f-a1e5-1f0e08b8b2c8","key":"Devices\/x.x.x.x\/Disks\/Disk0.0"}}
2019-01-23 18:27:30,148 DEBUG zen.MetricWriter: publishing metric x.x.x.x/statistics_iops 2.0 1548268050 {'device': 'x.x.x.x', 'contextUUID': 'bcd5ffb1-4931-4d27-a542-151b2aba0959', 'key': 'Devices/x.x.x.x/Disks/Disk0.1'}
2019-01-23 18:27:30,148 DEBUG zen.publisher: writing: {"timestamp":1548268050,"metric":"x.x.x.x\/statistics_iops","value":2.0,"tags":{"device":"x.x.x.x","contextUUID":"bcd5ffb1-4931-4d27-a542-151b2aba0959","key":"Devices\/x.x.x.x\/Disks\/Disk0.1"}}
2019-01-23 18:27:30,149 DEBUG zen.MetricWriter: publishing metric x.x.x.x/statistics_iops 2.0 1548268050 {'device': 'x.x.x.x', 'contextUUID': '2de84a65-f4bf-437d-a5ab-99ed242a8c89', 'key': 'Devices/x.x.x.x/Disks/Disk0.2'}
......
2019-01-23 18:27:30,153 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics finished, result: None
2019-01-23 18:27:30,153 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from STORE_PERF_DATA to IDLE
2019-01-23 18:27:30,153 DEBUG zen.publisher: trying to publish 11 metrics
Zenoss version : zenoss core 6.2.0
Could you help to find out what's the problem? Thanks!
------------------------------
Andy Hu
------------------------------
2019-01-15 14:39:31,735 INFO zen.ZenModeler: Counter eventCount, value 6
2019-01-15 14:40:01,735 INFO zen.ZenModeler: Counter eventCount, value 12
2019-01-15 14:40:31,735 INFO zen.ZenModeler: Counter eventCount, value 18
2019-01-15 14:41:01,735 INFO zen.ZenModeler: Counter eventCount, value 24
2019-01-15 14:41:31,735 INFO zen.ZenModeler: Counter eventCount, value 30
2019-01-15 14:42:01,735 INFO zen.ZenModeler: Counter eventCount, value 36
2019-01-15 14:42:31,735 INFO zen.ZenModeler: Counter eventCount, value 42
2019-01-15 14:43:01,735 INFO zen.ZenModeler: Counter eventCount, value 48
2019-01-15 14:43:31,735 INFO zen.ZenModeler: Counter eventCount, value 54
2019-01-15 14:44:01,735 INFO zen.ZenModeler: Counter eventCount, value 60
2019-01-15 14:44:31,735 INFO zen.ZenModeler: Counter eventCount, value 66
2019-01-15 14:45:01,735 INFO zen.ZenModeler: Counter eventCount, value 72
2019-01-15 14:45:31,735 INFO zen.ZenModeler: Counter eventCount, value 78
2019-01-15 14:46:01,735 INFO zen.ZenModeler: Counter eventCount, value 84
2019-01-15 14:46:31,735 INFO zen.ZenModeler: Counter eventCount, value 90
2019-01-15 14:47:01,735 INFO zen.ZenModeler: Counter eventCount, value 96
2019-01-15 14:47:31,735 INFO zen.ZenModeler: Counter eventCount, value 102
2019-01-15 14:48:01,735 INFO zen.ZenModeler: Counter eventCount, value 108
2019-01-15 14:48:31,735 INFO zen.ZenModeler: Counter eventCount, value 114
2019-01-15 14:49:01,735 INFO zen.ZenModeler: Counter eventCount, value 120
2019-01-15 14:49:31,735 INFO zen.ZenModeler: Counter eventCount, value 126
2019-01-15 14:50:01,735 INFO zen.ZenModeler: Counter eventCount, value 132
2019-01-15 14:50:31,735 INFO zen.ZenModeler: Counter eventCount, value 138
2019-01-15 14:51:01,735 INFO zen.ZenModeler: Counter eventCount, value 144
2019-01-15 14:51:31,735 INFO zen.ZenModeler: Counter eventCount, value 150
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Jane Curry
Posted: 2019-01-23 06:50
Looks like you are actually getting data for your disk:
2019-01-23 18:27:30,147 DEBUG zen.collector.scheduler: Task x.x.x.x 300 sample-Statistics changing state from SEND_EVENTS to STORE_PERF_DATA
2019-01-23 18:27:30,147 DEBUG zen.MetricWriter: publishing metric x.x.x.x/statistics_iops 2.0 1548268050 {'device': 'x.x.x.x', 'contextUUID': '99c324bf-4d1e-4a9f-a1e5-1f0e08b8b2c8', 'key': 'Devices/x.x.x.x/Disks/Disk0.0'}
2019-01-23 18:27:30,148 DEBUG zen.publisher: writing: {"timestamp":1548268050,"metric":"x.x.x.x\/statistics_iops","value":2.0,"tags":{"device":"x.x.x.x","contextUUID":"99c324bf-4d1e-4a9f-a1e5-1f0e08b8b2c8","key":"Devices\/x.x.x.x\/Disks\/Disk0.0"}}
Do you have a datasource called something like sample-Statistics and a datpoint of statistics_iops?
Cheers,
Jane
------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-23 07:02
yes, i have component collect the data.
my zenpack.yaml:
SampleDisk:
description: SampleDisk Template.
targetPythonClass: ZenPacks.community.sample.SampleDisk
datasources:
statistics:
enabled: True
type: Python
plugin_classname: ZenPacks.community.sample.dsplugins.Statistics
cycletime: "300"
datapoints:
iops: GAUGE
graphs:
IOPS:
units: operation
graphpoints:
IOPS:
dpName: statistics_iops
format: "%5.0lf iops"
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-16 07:42
Hi Jane,
Two more question about zenoss.
1. Model Device is timing task or just one excute. Because I didn't find where to set the interval time.
2. Component graph, does zenoss core support it? or zenoss service dynamics.
Thanks very much!
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-27 20:52
Hi Jane,
One more question about zenpython "blockingtimeout".
When I use zenpython to collet device statistics, because the component and statistics is large. So it's timeout above 30 sec where "blockingtimeout" defined. Also I try to update the zenpython.conf, add blockingtimeout 900, but it seems not work.
Do you have some good ways to solve it? Thanks very much!
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Jane Curry
Posted: 2019-01-28 07:34
I know it used to be the case that you could set blockingtimeout to 0 which prevented this feature entirely. Looking around, it would appear that references to this might have been removed. You certainly need to be very acreful in setting it to 0 as it may have nasty performance implications. There is an excellent explanation of the theory of this from Chet at
Quirky disabling behaviour with PythonCollector ZenPack 1.7.3 .
Cheers,
Jane
------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-29 06:13
OK, Thanks!
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-01-29 07:22
It seems does not work when set blockingtimeout to 0. And it still timeout(30s).
the zenpython.conf file is:
#
# Configuration file for zenpython
#
# To enable a particular option, uncomment the desired entry.
#
# Parameter Setting
# --------- -------
#
blockingtimeout 0
------------------------------
Andy Hu
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Ryan Matte
Posted: 2019-02-21 11:19
blockingtimeout is for a specific use case where a plugin isn't doing what it's supposed to be and ends up blocking for a long period of time which halts all data collection for zenpython. The blockingtimeout specifies how long a plugin should remain in that blocked state before zenpython disables the plugin to allow itself to continue collection with the other plugins.
I've reviewed the output you provided and I can't see any indication that you're hitting a timeout at all. The only line that references a timeout is...
2019-01-23 18:27:30,147 DEBUG zen.zenpython: Queued event (total of 5) {'rcvtime': 1548268050.14713, 'severity': 0, 'eventClassKey': 'zenpython-timeout', 'component': '', 'monitor': 'localhost', 'agent': 'zenpython', 'summary': 'timeout collecting statistics datasource for multiple components', 'manager': 'f8738fddd418', 'eventKey': 'zenpython-timeout|x.x.x.x 300 sample-Statistics', 'components':
...and that has a severity of 0 meaning it's a clear event (which means it's saying that a timeout did not happen).
I do however see that your plugin appears to be returning no data:
Which would indicate a potential logic issue in your code.
serviced service attach collectorname/zenpython
If it does show up in there you'd need to edit the file and remove it then restart that zenpython instance to re-enable it. I generally do not recommend setting blockingtimeout to 0 as that's a very bad idea (that can cause zenpython to get in to a state where it locks up and never recovers). If you think that's an issue you should increase that from 30 to 600.
If the plugin still blocks with it set that high then there's a problem with the plugin code that needs to be fixed. zenpython plugins are supposed to defer their work using twisted (@inlineCallbacks in the plugin code) so that they don't tie up zenpython while running. If your zenpython code isn't written properly that could cause it to block.
The example plugin code here demonstrates how the plugin should work: https://zenpack-sdk.zenoss.com/en/latest/tutorial-http-api/datasource-plugin-events.html#using-pythoncollector
------------------------------
Ryan Matte
------------------------------
Subject: RE: zen.ZenModeler doesn't finish
Author: Andy Hu
Posted: 2019-02-21 20:08
Thanks Ryan!
The explain is very clear. Now I just set blocktimeout to 180 not 0, and it works fine.
------------------------------
Andy Hu
------------------------------
blockingtimeout is for a specific use case where a plugin isn't doing what it's supposed to be and ends up blocking for a long period of time which halts all data collection for zenpython. The blockingtimeout specifies how long a plugin should remain in that blocked state before zenpython disables the plugin to allow itself to continue collection with the other plugins.
I've reviewed the output you provided and I can't see any indication that you're hitting a timeout at all. The only line that references a timeout is...
2019-01-23 18:27:30,147 DEBUG zen.zenpython: Queued event (total of 5) {'rcvtime': 1548268050.14713, 'severity': 0, 'eventClassKey': 'zenpython-timeout', 'component': '', 'monitor': 'localhost', 'agent': 'zenpython', 'summary': 'timeout collecting statistics datasource for multiple components', 'manager': 'f8738fddd418', 'eventKey': 'zenpython-timeout|x.x.x.x 300 sample-Statistics', 'components':
...and that has a severity of 0 meaning it's a clear event (which means it's saying that a timeout did not happen).
I do however see that your plugin appears to be returning no data:
Which would indicate a potential logic issue in your code.
serviced service attach collectorname/zenpython
If it does show up in there you'd need to edit the file and remove it then restart that zenpython instance to re-enable it. I generally do not recommend setting blockingtimeout to 0 as that's a very bad idea (that can cause zenpython to get in to a state where it locks up and never recovers). If you think that's an issue you should increase that from 30 to 600.
If the plugin still blocks with it set that high then there's a problem with the plugin code that needs to be fixed. zenpython plugins are supposed to defer their work using twisted (@inlineCallbacks in the plugin code) so that they don't tie up zenpython while running. If your zenpython code isn't written properly that could cause it to block.
The example plugin code here demonstrates how the plugin should work: https://zenpack-sdk.zenoss.com/en/latest/tutorial-http-api/datasource-plugin-events.html#using-pythoncollector
------------------------------
Ryan Matte
------------------------------
2019-01-15 14:39:31,735 INFO zen.ZenModeler: Counter eventCount, value 6
2019-01-15 14:40:01,735 INFO zen.ZenModeler: Counter eventCount, value 12
2019-01-15 14:40:31,735 INFO zen.ZenModeler: Counter eventCount, value 18
2019-01-15 14:41:01,735 INFO zen.ZenModeler: Counter eventCount, value 24
2019-01-15 14:41:31,735 INFO zen.ZenModeler: Counter eventCount, value 30
2019-01-15 14:42:01,735 INFO zen.ZenModeler: Counter eventCount, value 36
2019-01-15 14:42:31,735 INFO zen.ZenModeler: Counter eventCount, value 42
2019-01-15 14:43:01,735 INFO zen.ZenModeler: Counter eventCount, value 48
2019-01-15 14:43:31,735 INFO zen.ZenModeler: Counter eventCount, value 54
2019-01-15 14:44:01,735 INFO zen.ZenModeler: Counter eventCount, value 60
2019-01-15 14:44:31,735 INFO zen.ZenModeler: Counter eventCount, value 66
2019-01-15 14:45:01,735 INFO zen.ZenModeler: Counter eventCount, value 72
2019-01-15 14:45:31,735 INFO zen.ZenModeler: Counter eventCount, value 78
2019-01-15 14:46:01,735 INFO zen.ZenModeler: Counter eventCount, value 84
2019-01-15 14:46:31,735 INFO zen.ZenModeler: Counter eventCount, value 90
2019-01-15 14:47:01,735 INFO zen.ZenModeler: Counter eventCount, value 96
2019-01-15 14:47:31,735 INFO zen.ZenModeler: Counter eventCount, value 102
2019-01-15 14:48:01,735 INFO zen.ZenModeler: Counter eventCount, value 108
2019-01-15 14:48:31,735 INFO zen.ZenModeler: Counter eventCount, value 114
2019-01-15 14:49:01,735 INFO zen.ZenModeler: Counter eventCount, value 120
2019-01-15 14:49:31,735 INFO zen.ZenModeler: Counter eventCount, value 126
2019-01-15 14:50:01,735 INFO zen.ZenModeler: Counter eventCount, value 132
2019-01-15 14:50:31,735 INFO zen.ZenModeler: Counter eventCount, value 138
2019-01-15 14:51:01,735 INFO zen.ZenModeler: Counter eventCount, value 144
2019-01-15 14:51:31,735 INFO zen.ZenModeler: Counter eventCount, value 150
------------------------------
Andy Hu
------------------------------
