TECHZEN Zenoss User Community ARCHIVE  
FORUMS / ZenPacks, JSON API, and Integrations / DISCUSSION

Parsing output from custom Powershell command

Subject: Parsing output from custom Powershell command
Author: Daniel
Posted: 2017-10-23 10:11

Hi Zenoss Community,

I'm running Zenoss 4.2.5 with the latest Microsoft Windows and Python Collector ZenPacks.

I'm having trouble parsing the output from a custom Powershell command and generating an event based on the output. I'm trying to do a simple ping test on the remote server to check internet connection. My Windows Shell script is this:

powershell -noprofile -command "Test-Connection www.google.com -quiet"

Data Source strategy is "Custom Command" and Parser is set to "Auto"

I'm using this line to test from the command line:

winrs single -r http://Server-Name:5985 -u 'Zenoss-User' -a kerberos --command 'powershell -noprofile -command "Test-Connection www.google.com"'

A successful ping returns the following message:

{'exit_code': 0, 'stderr': [], 'stdout': [u'True']}

A failed ping returns this:

{'exit_code': 0, 'stderr': [], 'stdout': [u'False']}

The powershell command completes fine in both instances so this is probably why the exit code is 0. Is there a way to generate an alert when standard output is 'False'?

Thanks!

Subject: RE: Parsing output from custom Powershell command
Author: Jane Curry
Posted: 2017-10-23 12:57

Have you looked at the "Custom Commands" section of the Windows ZenPack readme - https://www.zenoss.com/product/zenpacks/microsoft-windows#custom-commands   ?

The auto parser typically means output is expected from the script in "nagios format".  This is a string that will appear as the summary of any event and then, if you want them, datapoint values in varname=value pairs where the string is separated from the variables by the pipe (vertical bar) symbol. The exit status of the script can determine whether you get an event or not. 

You may not want datapoint values but, for completeness here is a valid little shellscript (sorry - not good on Windows and Powershell) that delivers an exit status, a string and 3 values:

jane@zenny:~/Documents/zenoss> cat /home/vmware/rwshare/test1.sh
#!/bin/sh

# Nagios return codes
STATE_OK=0
STATE_WARNING=1
STATE_CRITICAL=2
STATE_UNKNOWN=3
#
exitstatus=$STATE_OK

# Nagios format echos information and status followed by pipe |
# followed by <var name>=<value> tuples
# Note that Zenoss datapoints must match these var names exactly
#
echo "This is a test - status OK | timevar=1s sizevar=2B percentvar=10% countervar=123c"
exit $exitstatus


Cheers,
Jane

------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------

Subject: RE: Parsing output from custom Powershell command
Author: Daniel
Posted: 2017-10-25 11:57

Thanks for the quick reply and your feedback, Jane. I've read several of your papers, including Using Nagios NRPE with Zenoss and was able to write several bash shell scripts which are working great. 

The problem I'm having is with Nagios return codes and Powershell. Based on the "Custom Commands" section on the official Microsoft Windows ZenPack page, I put together the following script:

$$STATE_OK = 0;
$$STATE_WARNING = 1;

$$exitstatus = $$STATE_OK;

If (Test-Connection 'www.google.com' -quiet)
{$$exitstatus = $$STATE_WARNING };
return $$exitstatus;

The command Test-Connection 'www.google.com' -quiet runs fine on the server so I reversed the logic to intentionally generate an event in Zenoss, but I'm not getting any events.

Has anyone had any success with custom Powershell scripts and Nagios return codes? I would greatly appreciate your feedback.

Below is DEBUG output from zenpython:

2017-10-23 16:59:53,251 DEBUG zen.zenpython: Tasks for config MY-TESTING-SERVER: {'MY-TESTING-SERVER 300 Windows Perfmon': <__main__.PythonCollectionTask object at 0x66d3ed0>, 'MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER': <__main__.PythonCollectionTask object at 0x66d8310>, 'MY-TESTING-SERVER 60 ZenPacks.zenoss.Microsoft.Windows.datasources.ServiceDataSource.ServicePlugin': <__main__.PythonCollectionTask object at 0x66d83d0>, 'MY-TESTING-SERVER 300 ZenPacks.zenoss.Microsoft.Windows.datasources.ServiceDataSource.ServicePlugin': <__main__.PythonCollectionTask object at 0x66d8450>}
2017-10-23 16:59:53,252 DEBUG zen.collector.scheduler: add task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER, <__main__.PythonCollectionTask object at 0x66d8310> using 120 second interval
2017-10-23 16:59:53,256 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER starting (waited 0 seconds) on 120 second intervals
2017-10-23 16:59:53,256 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from IDLE to QUEUED
2017-10-23 16:59:53,260 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from QUEUED to RUNNING
2017-10-23 16:59:53,260 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from RUNNING to BLOCKING
2017-10-23 16:59:53,261 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from BLOCKING to RUNNING
2017-10-23 16:59:56,808 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from RUNNING to BLOCKING
2017-10-23 16:59:56,808 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from BLOCKING to RUNNING
2017-10-23 16:59:56,808 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from RUNNING to BLOCKING
2017-10-23 16:59:56,810 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from BLOCKING to RUNNING
2017-10-23 16:59:56,810 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from RUNNING to BLOCKING
2017-10-23 16:59:56,810 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from BLOCKING to RUNNING
2017-10-23 16:59:56,810 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from RUNNING to SEND_EVENTS
2017-10-23 16:59:56,812 DEBUG zen.zenpython: Queued event (total of 15) {'rcvtime': 1508795996.812056, 'severity': 0, 'eventClassKey': 'winrsCollectionError', 'agent': 'zenpython', 'summary': 'Monitoring ok', 'manager': 'my-zenoss-server', 'eventKey': 'datasourceWarning_TestInternetConnection', 'device': 'MY-TESTING-SERVER', 'device_guid': '92c9ef63-99e4-4b83-9604-8b0fbfa34d48', 'monitor': 'localhost'}
2017-10-23 16:59:56,813 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER finished, result: None
2017-10-23 16:59:56,813 DEBUG zen.collector.scheduler: Task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER changing state from SEND_EVENTS to IDLE
2017-10-23 16:59:57,378 DEBUG zen.collector.scheduler: Stopping running task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER
2017-10-23 16:59:57,378 DEBUG zen.collector.scheduler: call finished LoopingCall<120>(CallableTask: MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER, *(), **{}) : LoopingCall<120>(CallableTask: MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER, *(), **{})
2017-10-23 16:59:57,378 DEBUG zen.collector.scheduler: Removing task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER
2017-10-23 16:59:57,379 DEBUG zen.collector.scheduler: Cleanup on task MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER <__main__.PythonCollectionTask object at 0x66d8310>
2017-10-23 16:59:57,379 DEBUG zen.collector.scheduler: Scheduler._cleanupTaskComplete: result=None task.name=MY-TESTING-SERVER 120 Custom Command TestInternetConnection MY-TESTING-SERVER

------------------------------
Daniel
------------------------------

Subject: RE: Parsing output from custom Powershell command
Author: Jane Curry
Posted: 2017-10-26 03:57

For a start, try changing your script so that it runs a bad exitstatus - whatever the result.  At least we can then see if the event mechanism is working.  The event you got in your trace was basically a clear sev event suggesting that things worked.
Cheers,
Jane

------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------

Subject: RE: Parsing output from custom Powershell command
Author: Daniel
Posted: 2017-10-26 14:06

I got it to work with the snippet below. When the Test-Connection is true, the command runs fine. When I make it false by putting in some arbitrary host that won't ping, the event "CRITICAL: No connection" gets generated, but there is a second event with the Summary:

No output from script for TestInternetConnection on MY-TESTING-SERVER

I've been modifying the script and trying different things but cannot figure out why the second event gets generated. I've tried single quotes, double quotes, and parentheses for the Write-Host commands but it doesn't make any difference. Any ideas?

$$STATE_OK = 0;
$$STATE_CRITICAL = 1;

$$LASTEXITCODE = $$STATE_OK;

If (Test-Connection www.google.com -quiet)
{
     Write-Host 'Connection OK';
}
else
{
     Write-Host 'CRITICAL: No connection';
     $$LASTEXITCODE = $$STATE_CRITICAL;
}

exit $$LASTEXITCODE;

------------------------------
Daniel
------------------------------

Subject: RE: Parsing output from custom Powershell command
Author: Jane Curry
Posted: 2017-10-27 05:57

The "No output from script for TestInternetConnection on MY-TESTING-SERVER" event is generated directly from the ShellDataSource.py code in the datasources directory of the Windows ZenPack - if the script doesn't deliver any output!  You can't really stop that.  What you could do is write a wee transform for the event class that gets generated, checking for the string "No output from script" and then drop that event, so something like:

if ("No output from script" in evt.summary):
   evt._action = "drop"

Cheers,
Jane

------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------

Subject: RE: Parsing output from custom Powershell command
Author: Daniel
Posted: 2017-10-27 10:23

Thanks for your help, Jane! The transform is working and Zenoss is generating alerts for my custom command.

Cheers!

------------------------------
Daniel
------------------------------


< Previous
Fix for Serviced command not found 		  Next
Updating ZenPack catalog 		>