TECHZEN Zenoss User Community ARCHIVE  

Error message, FileSystem template (SSH Linux)

Subject: Error message, FileSystem template (SSH Linux)
Author: Jonas Stoltz
Posted: 2018-06-12 05:56

When running or testing the template we get an error message;
Preparing Command...
Executing command /bin/sh [args omitted] against 172.26.1.48
exception while executing command
type: <class 'twisted.internet.error.ReactorNotRestartable'> value:
DONE in 0 seconds

Command run;
/bin/sh -c 'export PATH=$$PATH:/bin:/sbin:/usr/bin:/usr/sbin; if command -v timeout > /dev/null 2>&1; then timeout 30 /usr/bin/env df -kP; else /usr/bin/env df -kP; fi'

(Zenoss 6.1.0 r46)
------------------------------
Jonas Stoltz

------------------------------


Subject: RE: Error message, FileSystem template (SSH Linux)
Author: Luke Lofgren
Posted: 2018-06-13 08:29

Does it work with just the simple DF command? I've not gotten to the point I can troubleshoot that complex command very well as well, but I've been able to work around it by using a simpler command line.

------------------------------
Luke Lofgren
Infrastructure Architect
Acxiom Corporation (home based associate)
Waterford PA
------------------------------

Subject: RE: Error message, FileSystem template (SSH Linux)
Author: Jane Curry
Posted: 2018-06-14 06:26

Excellent suggestion to test first with a very simple command like df to check that the authentication is setup correctly.

Then I would try running zencommand standalone in debug and checking the output.  You don't say whether you are on Zenoss 4.x or earlier, or on 5 or 6 (the dockerized versions).  If on Zenoss 4 or less, from a command line, for  a device called fred.class.example.org, use:

zencommand run -v 10 -d fred.class.example.org --showfullcommand > /tmp/fred 2>&1

then check /tmp/fred.  The --showfullcommand should reveal the exact command sent, complete with parameter substitution.  It's almost certainly a glitch with parameter substitution, quoting or special characters in the bash command, once you have eliminated basic authentication issues.

If you are on a Zenoss 5 or 6, you will need to attach to the zencommand container first with:

serviced service attach zencommand su - zenoss

Cheers,
Jane

------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------


Subject: RE: Error message, FileSystem template (SSH Linux)
Author: Jonas Stoltz
Posted: 2018-06-18 06:08

Hi,

Thanks for the suggestion on running just "df -kP" which I already have (did again just to be sure), but same error.
I did post version in my post (Zenoss 6.1.0 r46) and running the debug did not get me any closer to the problem.

I'm not sure what to look for in that file (contains a LOT), but searching for "df" did not work (noting found except some other words containing df).

------------------------------
Jonas Stoltz
------------------------------
Preparing Command...
Executing command /bin/sh [args omitted] against 172.26.1.48
exception while executing command
type: <class 'twisted.internet.error.ReactorNotRestartable'> value:
DONE in 0 seconds

Command run;
/bin/sh -c 'export PATH=$$PATH:/bin:/sbin:/usr/bin:/usr/sbin; if command -v timeout > /dev/null 2>&1; then timeout 30 /usr/bin/env df -kP; else /usr/bin/env df -kP; fi'

(Zenoss 6.1.0 r46)
------------------------------
Jonas Stoltz

------------------------------


Subject: RE: Error message, FileSystem template (SSH Linux)
Author: Jay Stanley
Posted: 2018-06-29 21:03

Try using the full path for df

Example:
/bin/df

Also, using timeout before df is a very good idea. Especially if you have network drives.

I have seen network drives hang from NAS issues and if you try running df, it just hangs. This is bad when Zenoss is sending this command every 5 minutes.

------------------------------
jstanley
------------------------------


Subject: RE: Error message, FileSystem template (SSH Linux)
Author: Jonas Stoltz
Posted: 2018-07-04 07:11

Jay,

Thanks for your suggestion but no luck.

I did on the other hand restart everything and then it worked with the defaults, but just ONCE.
Restarting all services does the trick, but cannot work like that.

Any suggestions?

------------------------------
Jonas Stoltz
------------------------------


Subject: RE: Error message, FileSystem template (SSH Linux)
Author: Jay Stanley
Posted: 2018-07-09 08:58

type: <class 'twisted.internet.error.ReactorNotRestartable'> is the error? If that is the case, something is stopping the twisted reactor. Working after a restart would make sense, since everything gets restarted and the reactor will be running again, executing commands.

  • You said it works once, how do you know it works once?
  • Can you post the output (scrubbed) of a zencommand run -d<deviceId> --showfullcommand
  • Are you running tests from the datasource? If so, can you restart zencommand, then not run any tests and see if things start graphing



------------------------------
jstanley
------------------------------


Subject: RE: Error message, FileSystem template (SSH Linux)
Author: Jonas Stoltz
Posted: 2018-08-06 08:03

Hi Jay,

Thanks for the assistance and suggestions here; much appreciated.

  • You said it works once, how do you know it works once?
I did "test" and saw it working, did another test which failed.

  • Can you post the output (scrubbed) of a zencommand run -d<deviceId> --showfullcommand
Attached.
  • Are you running tests from the datasource? If so, can you restart zencommand, then not run any tests and see if things start graphing
Restarted, but no data is coming in (checked graphs for diks util and inode util).
As far as I can see from the output, it does not run the "FileSystem" commands?

------------------------------
Jonas Stoltz
------------------------------


Subject: RE: Error message, FileSystem template (SSH Linux)
Author: Jonas Stoltz
Posted: 2018-08-09 03:49

Hi,

It seems to be working now after just restarting, and not testing the setup.
The only thing left now is that it comes an error message now and then from *all* servers;
Unable to process COMMAND datasource(s) for device XYZ -- skipping

It is connected to this, because if I disable the command for file system (disk/idisk) the message is not coming anymore, but also no data of course.

------------------------------
Jonas Stoltz
------------------------------


< Previous
Monitoring loss on Ethernet SNMP thermometer
  Next
Monitor for Presence of a file
>