Zenoss ZenTech Community

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Laurent Hemeryck
Posted:	2020-04-03 04:29

Hello Larry,

Your issue is quite difficult to troubleshoot without any idea of what the ZP does, or without the source code of those datasources.

It's perfectly possible for a ZP to break Zenoss or other ZenPacks.

Did you setup the config_key method correctly in order not to shoot more datasource plugins than necessary ?

Cheers,

Laurent

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Larry
Posted:	2020-04-07 17:26

Hi Laurent,
Thanks for your response. It looks like it it is configured correctly, but we might be missing something. The code is attached. Can you take a look at it?

Thanks,

------------------------------
Larry
------------------------------

Attachments:

Qlik_Sense_-_Datasource.txt

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Laurent Hemeryck
Posted:	2020-04-08 03:32

Hello Larry,

It's quite hard to troubleshoot with the given info. I have no clue of what you are trying to do.

Can you please add a static value, e.g. "MyPlugin" in the output of the config_key function ?
Also, it depends on what you are polling, but I would test removing "context.id" from the ouput of the same function.

Cheers,

Laurent

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Larry
Posted:	2020-04-08 14:58

Hi Laurent,
Thanks for the reply. We will test your suggestions.

Best regards,

------------------------------
Larry
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Larry
Posted:	2020-04-08 23:21

Hi Laurent,
Unfortunately, removing "context.id" didn't fix the issue. And this cause another issue with the display. So we're putting it back.

We're going to try adding "MyPlugin" at the end for the next test.

BTW, I noticed that a lot of the classes have a trailing comma (comma after last element). Is this an issue, or can it be ignored? I'm not a developer, so I don't know the answer. It should seems odd to me based on my old programming experience.

One other thing: This ZP *appears* to be working correcting on Zenoss 4.2.5 (not able to confirm). We upgraded Python to 2.7.10 on that install though. We're running Python 2.7.5 (default) on Zenoss 6.2.1. Do you think it's worthwhile to upgrade Python to 2.7.10 here?

Thanks in advance,

------------------------------
Larry
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Laurent Hemeryck
Posted:	2020-04-09 04:05

Hello Larry,

I'm not 100% sure of which trailing comma you are reffering to. If you mean a comma after the last element in a dictionary, it has no impact. I think it's even considered as Pythonic for some people.

I haven't seen many ZP that were working on 4.2.5 and not on 6.x. Anyway, the Python that your ZP is using is the one that's included in the Docker image. This is still a Python 2.7.5. So, if you upgrade the python on the OS, it will have no impact. It's possible to upgrade the Python embedded in the Docker image, but I wouldn't recommend it.

Again, a better insight of the context and more details about the code would help troubleshooting. I have no idea whether you are polling single or multiple sources for a single or multiple components.

Cheers,

Laurent

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Larry
Posted:	2020-04-09 12:15

Hi Laurent,
I saw some trailing comma examples after my last post, so I realized that it's not an issue: https://help.zenoss.com/dev/zenpack-sdk/tutorials/monitoring-an-http-api/datasource-plugin-data-points/create-conditions-data-source-plugin

I would be upgrading Python inside Docker. We've done it before in testing, and there are some additional steps needed (e.g. upgrading other packages). I want to avoid doing this if possible.

We tried adding "MyPlugin" as you recommended, but it's still not working correctly.

Here are some details about how it was developed and what our developer used to build the custom ZenPack:

ZenPacklib (The ZenPack was built using ZenPacklib. Here's the documentation followed to build the datasources, datapoints, and plugins: https://help.zenoss.com/dev/ZenPack-sdk/tutorials/monitoring-an-http-api/datasource-plugin-data-points/create-conditions-data-source-plugin)

Datasource File - Collect Method: Method executed for getting the reload task values. For each component (reload task) will call this method informing the Reload task ID, with the ID it will make a call to the Qlik API and get the values referent to that task. Then all the values are formatted to their respective Datapoints for graphs and also some maps (values showed on the main table: Status, Start Time, Stop Time…)

To not cause the blocking issues, I used a function called deferToThread. This function belongs to the Twisted library, and it was used as non-blocking code to make the requests to the Qlik API service. The method shown in the documentation uses another method called getPage, but it only works for HTTP and not HTTPS.

response = yield threads.deferToThread(self.getResult)

Thanks,

------------------------------
Larry
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Laurent Hemeryck
Posted:	2020-04-09 14:13

Hello Larry,

Again, I would avoid touching the Docker image.

I'm a bit surprised reading that getPage only works with HTTP and not HTTPS. I'm pretty sure I have used this function with HTTPS. However, it's not compatible with HTTP/1.1 and higher, from what I remember.

I think that using defertoThread makes your life more complex. And I'm not sure how Zenoss/twisted handles those extra threads.
Please have a look at this repo, it's similar to the code used for the HTTP API tutorial. However, it doesn't make use of getPage, but rather of twisted.web.client.Agent.
https://github.com/Hemie143/ZenPacks.training.NWS

I guess that your ZP should have a similar structure (or better ;) ).

Cheers,

Laurent

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Larry
Posted:	2020-04-10 00:44

Thanks Laurent! This is awesome! We will give this a shot. You are correct----you used https in the ZP.

Happy Easter!

------------------------------
Larry
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Laurent Hemeryck
Posted:	2020-04-17 03:30

Hello Larry,

How are you doing ? Were you able to successfully collect data the way you expected ?

Cheers,

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Larry
Posted:	2020-04-20 23:22

Hi Laurent,
Thanks for your message. We found that the version of Twisted was too old. We had 11.0.0, and it needs to be at least 13.10.0.

We were able to upgrade Twisted to 15.5.0, and our developer will be testing it tomorrow. I'll let you know the results.

Thanks,

------------------------------
Larry
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Laurent Hemeryck
Posted:	2020-04-21 02:58

Hello Larry,

How does it come that you're not using the default embedded install of Twisted ? I would never touch any of the Zenoss components (Twisted, Zope, RabbitMQ, ..) within the docker images they are providing. It's just asking for troubles. However, I would like to be able to upgrade the Python engine or Twisted that come within Zenoss. The components are a bit old now, however, those are the ones that Zenoss is supporting.
I'm running 6.4.1 and it's using Twisted 16.1.1. So, you can probably even further upgrade your Twisted library.

Cheers,

Laurent

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Larry
Posted:	2020-04-21 16:11

Hi Laurent,
Sorry for the confusion. Our ZenPack server is running 4.2.5. I tried to upgrade to Twisted 16.1.1, but it was complaining about missing a plugin. We are looking to migrate to another server for ZP development after this is done.

Do you know if there's anything published on how to make a 4.2.5 server to develop ZPs? The only thing I found is this, and it's for 5.X+ (possibly only .X): https://help.zenoss.com/dev/zenpack-sdk/development-environment

Are you serious about upgrading the Zenoss Python? I thought you were opposed to it. Maybe I understood you wrong. The notes from my upgrade are below. I think I had to create some directories under /opt/zenoss as they were all linked previously.

Let me know if you try this and run into any issues.

Note: Upgrading Python higher than 2.7.10 required updating some other packages.

Thanks,
Larry

sudo yum groupinstall -y "Development tools"
sudo yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel
sudo su - zenoss
wget https://www.python.org/ftp/python/2.7.10/Python-2.7.10.tgz
tar -zxf Python-2.7.10.tgz
cd Python-2.7.10
./configure --prefix=/opt/zenoss --enable-unicode=ucs4

make
make install

------------------------------
Larry
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Laurent Hemeryck
Posted:	2020-04-22 03:43

Hello Larry,

You don't need anything specific for a Development Zenoss 4.x. Just a Core edition, running on a single-hsot should be sufficient. The specs of the server don't need to be very high. Just follow the standard install procedure. After that, I have some recommandations:
- install just the ZenPacks you need
- remove the ZenPacks you don't need (be careful about this one, don't forget any dependency)
- Simply add the monitoring of the Zenoss server, just one device
- Run the zentoolbox checks and fix any possible error
- take a full backup or snapshot of the instance when it's running cleanly and correctly. So you can revert to this state whenever you have finished the development of a ZenPack or when you have messed up during the development. With Zenoss 5.X or 6.x, this is very easy, because you have a snapshot command within serviced.
- never monitor a ton of devices

Otherwise, in my environment, I'm not developing directly on the server. I'm using PyCharm on my laptop. I can push my code in a git repo and my code is also automatically synchronized with the code on the server, through SSH. This works, of course, with Zenoss 4.x and higher.

However, I would say that if you can upgrade to Zenoss 6.x, I would do it. All the new ZenPacks are only supported on 6.x. However, the migration effort is very high.

I would like to upgrade Python to Python 3.7 or 3.8, but I won't do it. Same for Twisted. I think it's more important to have a stable and supported environment rather than the latest release. My concerns with Python 2.7 is that there won't be any new patch now (final release 2.7.18 on 20/04/2020). So, what will happen with possible security issues ? How will they be fixed ? Secondly, it's a bit annoying to still code in Python 2.x when you're also used to Python 3.x.

Cheers,

Laurent

------------------------------
Laurent Hemeryck
Monitoring Engineer
FedNot
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Jane Curry
Posted:	2020-04-22 03:53

Hi Larry,
Do you know about the ZenPack Developers' Guide that i wrote?https://github.com/ZenossDevGuide/DevGuide .

I find development in a post-4.2.5 environment just too tedious and hard. I have successfully developed several ZenPacks in 4.2.5 and then just deployed them on a dockerised version. Obviously it needs testing there but the developmemnt and debugging is SO much easier on Zenoss 4.

Cheers,
Jane

------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------

Subject:	RE: Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Ryan Matte
Posted:	2020-06-05 17:35

Hi Larry,

Zenoss installs are heavily dependent on the version of twisted that comes packaged with them. Any time we upgrade twisted to a newer version it requires extensive testing and often code changes to accommodate that. For instance for the upcoming 6.5.0 release we upgraded twisted to a newer version and it required significant effort to make sure the product didn't break from that. As other people in this thread mentioned, upgrading the Zenoss twisted library is just an all around bad idea, you can and will break Zenoss. If you're upgrading that library outside of the Zenoss environment where Zenoss won't be directly using it and then selectively importing it in to your own things that could work depending on how you're making use of it. I would advise extreme caution with that.

------------------------------
Ryan Matte
------------------------------

Subject:	Custom ZenPack causing erratic data collection for other ZenPacks
Author:	Larry
Posted:	2020-03-31 00:37

Custom ZenPack causing erratic data collection for other ZenPacks