TECHZEN Zenoss User Community ARCHIVE  

Python Collector Datasource with SSH

Subject: Python Collector Datasource with SSH
Author: Peter
Posted: 2020-08-20 10:54

Hello

I am looking to create some Python Collector datasources (PythonDataSourcePlugin and friends) that uses SSH in the collection method.

All the examples I have seen in the documentation usually show hitting an API. I can't find anything that attempts SSH.

The ZenPack I am creating is intended to replace some COMMAND checks that require more processing and control than can be found with some Bash wizardry.

I know that the SshClient class exists in Products/DataCollector/SshClient, but it's not super obvious how to best use it. Further, I am not sure if I should be using the @inlineCallbacks decorator for the collect method when using this class. The class itself makes use of Twisted's SSH library but I don't know if that matters in the context of a datasource.

Any hints/help or pointers to a working example would be fantastic, as my Google-Fu is letting me down.

If it helps, the ZenPack is being targetted towards Zenoss 6+ (and Zenoss Cloud specifically, though will be developed against a Zenoss Core 6+ instance).

EDIT: I would like to add that I asked about this in the #zenoss IRC channel a few weeks ago, and it was there that I was pointed to the SshClient class. The person who responded said that this has been used by some ZenPacks, but I don't recall which one(s) they were referring to. So, if they exist and anyone knows which ones they are, please share!


------------------------------
Peter
------------------------------


Subject: RE: Python Collector Datasource with SSH
Author: Michael Rogers
Posted: 2020-08-20 16:38

Hi, Peter!

I managed to find the IRC logs and it looks like Josh Wilmes was pointing toward the OpenStack ZenPack (https://help.zenoss.com/in/zenpack-catalog/open-source/openstack-provider-view).  It looks like there are a few good examples of how the SSHClient is used.  While that may not translate perfectly to your use case, it should get you started.

As for the inlineCallbacks, I'm not sure either, but the ZenPack contains a few examples of them as well.

I hope that helps!

------------------------------
Michael J. Rogers
Senior Instructor - Zenoss
Austin TX
------------------------------


Subject: RE: Python Collector Datasource with SSH
Author: Peter
Posted: 2020-08-21 06:12

Hi Michael

I took a look at the OpenStack ZenPack, but it looks like a different SSH client is being leveraged. Looking on GitHub I can see that there is an ssh.py, which in turn loads an SSH client from an sshclient module. The only one I could find was in the lib directory, but that's empty. So it's not clear what it actually uses. It makes mention of the "platform" SSH, but I have no idea what that is or where it would be on a Zenoss install...

I also notice they import SSHClient and not SshClient. The signature is also different when creating a new instance:

OpenStack version:

client = SSHClient({ 'hostname': manageIp, 'port': device.zCommandPort, 'user': device.zCommandUsername, 'password': device.zCommandPassword, 'identities': [device.zKeyPath], 'buffersize': 32768})​

Whereas the Products.DataCollector.SshClient has separate keyword arguments (and doesn't accept a dict):

class SshClient(CollectorClient.CollectorClient):
    """
    SSH Collector class to connect to a particular device
    """
    def __init__( self, hostname, ip, port=22, plugins=[], options=None, device=None, datacollector=None, isLoseConnection=False, )​


If you can find a ZenPack that uses SshClient that would be useful. I think there's enough to go on to give things a try, but was hoping to see a working example and save myself some effort!


------------------------------
Peter Spain
------------------------------


Subject: RE: Python Collector Datasource with SSH
Author: Michael Ellwood
Posted: 2020-08-26 16:40

It's using txsshclient from https://github.com/zenoss/txsshclient
It's located in the src directory in the root of the OpenStackInfrastructure zenpack repo. If you look at the GNUmakefile, the build command will "build" it from src and copy it to that lib directory. The ssh client will be located in the lib directory after installing it.

------------------------------
Michael Ellwood
------------------------------


Subject: RE: Python Collector Datasource with SSH
Author: Peter
Posted: 2020-08-27 05:19

Hi Michael

I found that txsshclient the other day. I haven't had much time this week to look at this, but a bit of testing this morning and it works like a charm.

I didn't bother with the build step - it's not obvious from the Makefile that it does anything other than build an egg. I just lifted and shifted the module to my ZPs lib directory.

In any case, thank you for your help!​

------------------------------
Peter Spain
------------------------------


Subject: RE: Python Collector Datasource with SSH
Author: Jane Curry
Posted: 2020-08-25 05:23

Hi Peter,
Get my ZenPack Developers' Guide from here -  ZenPack Developers' Guide
There is an example that uses ssh to collect both modeling data and performance data using the Python Collector.

Cheers,
Jane

------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------


Subject: RE: Python Collector Datasource with SSH
Author: Peter
Posted: 2020-08-27 05:20

Hi Jane

I didn't notice that in the Dev Guide before (I refer to your guide a lot at times). 

I have solved the issue by using the txsshclient module from Zenoss. However, I will bear the overall approach you took in mind for the future.

Thanks!

------------------------------
Peter Spain
------------------------------


Subject: RE: Python Collector Datasource with SSH
Author: Jane Curry
Posted: 2020-08-27 06:57

Thanks Peter - glad you have a solution. Also delighted that the
Developer's Guide is useful to you! I also use it a lot :)

Cheers,
Jane

On 27/08/20 10:22, Peter Spain via Zenoss wrote:
> Hi Jane I didn't notice that in the Dev Guide before (I refer to your
> guide a lot at times). I have solved the issue by using the
> txsshclient...
> Zenoss
>
>
> ZenPacks, JSON API, and Integrations
>
>
>
> Post New Message
>
> Re: Python Collector Datasource with SSH
>
>
> Reply to Group
>
> Reply to Sender
>
>
>
> Peter
>
>
> Aug 27, 2020 4:20 AM
> Peter
>
>
>
> Hi Jane
>
> I didn't notice that in the Dev Guide before (I refer to your guide a
> lot at times).
>
> I have solved the issue by using the txsshclient module from Zenoss.
> However, I will bear the overall approach you took in mind for the future.
>
> Thanks!
>
> ------------------------------
> Peter Spain
> ------------------------------
>
> *Reply to Group Online
> *
> *View Thread
> *
> *Recommend
> *
> *Forward
> *
> *Flag as Inappropriate
> *
>
> -------------------------------------------
> Original Message:
> Sent: 08-25-2020 04:23 AM
> From: Jane Curry
> Subject: Python Collector Datasource with SSH
>
> Hi Peter,
> Get my ZenPack Developers' Guide from here - ZenPack Developers'
> Guide
> There is an example that uses ssh to collect both modeling data and
> performance data using the Python Collector.
>
> Cheers,
> Jane
>
> ------------------------------
> Jane Curry
> Skills 1st United Kingdom
> jane.curry@skills-1st.co.uk
>
>
>
>
> You are receiving this notification because you followed the 'Python
> Collector Datasource with SSH' message thread. If you do not wish to
> follow this, please click here
> .
>
>
> Update your email preferences
> to
> choose the types of email you receive
>
> Unsubscribe from all participation emails
>
>

--
Zenoss Master
Skills 1st Limited, 2 Cedar Chase, Taplow, Bucks, SL6 0EU, UK.
Registered in England & Wales, Company No. 3458854.
Tel: +44 (0)1628 782565 Skype: jane_curry_uk
Email: jane.curry@skills-1st.co.uk Web: http://www.skills-1st.co.uk
Copyright (c) 2020 Jane Curry < jane.curry@skills-1st.co.uk >. All rights reserved.


< Previous
zenpacklib yaml file - defining minProcessCount and maxprocessCount for a proces ...
  Next
how can i monitoring the iostat for a linux device for each hard disk
>