TECHZEN Zenoss User Community ARCHIVE  

Zenoss 6 zendmd adding device takes 40 seconds

Subject: Zenoss 6 zendmd adding device takes 40 seconds
Author: Jad Baz
Posted: 2018-11-19 11:16

Hello

I'm issuing the following script in Zenoss 6.2.1 zendmd inside Zope container:

dmd.DeviceLoader.loadDevice(
    deviceName="device1",
    devicePath="Server",
    tag="",
    serialNumber="123",
    zSnmpCommunity=None,
    zSnmpPort=None,
    zSnmpVer=None,
    rackSlot="type1",
    productionState="prod",
    comments="",
    hwManufacturer="",
    hwProductName="",
    osManufacturer="",
    osProductName="",
    locationPath="location1",
    groupPaths=[],
    systemPaths=[],
    performanceMonitor="localhost",
    manageIp="",
    discoverProto="none",
    priority=3,
    title=None,
    REQUEST=None
)


This runs fine except that it takes 40 seconds to finish.
I've gone through the python classes and identified exactly why this is happening:

1st instance
file: /opt/zenoss/Products/ZenModel/ZDeviceLoader.py
function: load_device

if discoverProto == 'none' and not manageIp:
    try:
        manageIp = getHostByName(deviceName)
    except socket.error:
        pass


2nd instance
file: /opt/zenoss/Products/ZenModel/Device.py
function: manage_createDevice > setManageIp

if not ip:
    try:
        ip = getHostByName(ipunwrap(self.id))
    except socket.error:
        ip = ''
        if origip:
            message = ("%s is an invalid IP address, "
                "and no appropriate IP could"
                " be found via DNS for %s") % (origip, self.id)
            log.warn(message)
        else:
            message = "DNS lookup of '%s' failed to return an IP" % \
                        self.id


In both instances, when manageIp="", it tries to resolve its IP from its device name using getHostByName()
When I run getHostByName it takes exactly 20 seconds (default os timeout I think). I am now sure that this is the issue.

I tried setting the IP to "127.0.0.1" and this goes from 40s (20x2 + epsilon) all the way down to 52ms (epsilon).

I now get a warning that the ip is already in use.
I realize this is not standard. My Zenoss workflow is agent-based (custom agent that calls zendmd) and I don't let Zenoss access the devices by IP.
As such, I'd like to set empty IPs.

Any way I can achieve this without loadDevice taking 40 seconds?
Is there a global Zenoss configuration that disables this behavior?
If I am destined to use an IP, are there any implications to setting all my devices to 127.0.0.1? (except for a warning that the ip is already in use)?

Thank you


------------------------------
Jad
------------------------------


Subject: RE: Zenoss 6 zendmd adding device takes 40 seconds
Author: Jane Curry
Posted: 2018-11-20 04:36

Can you adjust the DNS timeout on your Zenoss server?  You might have a look at this article - https://unix.stackexchange.com/questions/78041/configuring-dns-resolver-failover-timeout  .

Cheers,
Jane

------------------------------
Jane Curry
Skills 1st United Kingdom
jane.curry@skills-1st.co.uk
------------------------------


Subject: RE: Zenoss 6 zendmd adding device takes 40 seconds
Author: Jad Baz
Posted: 2018-11-20 07:44

Thanks Jane,

But this doesn't seem to be working.
Even if I do get it to work though, I'd prefer not to change Linux configuration only for Zenoss device creation.
Might there be a flag for this in Zenoss somewhere?
Otherwise, do you see any potential downsides to setting the same IP, 127.0.0.1, for all devices?

------------------------------
Jad
------------------------------
dmd.DeviceLoader.loadDevice(
    deviceName="device1",
    devicePath="Server",
    tag="",
    serialNumber="123",
    zSnmpCommunity=None,
    zSnmpPort=None,
    zSnmpVer=None,
    rackSlot="type1",
    productionState="prod",
    comments="",
    hwManufacturer="",
    hwProductName="",
    osManufacturer="",
    osProductName="",
    locationPath="location1",
    groupPaths=[],
    systemPaths=[],
    performanceMonitor="localhost",
    manageIp="",
    discoverProto="none",
    priority=3,
    title=None,
    REQUEST=None
)


This runs fine except that it takes 40 seconds to finish.
I've gone through the python classes and identified exactly why this is happening:

1st instance
file: /opt/zenoss/Products/ZenModel/ZDeviceLoader.py
function: load_device

if discoverProto == 'none' and not manageIp:
    try:
        manageIp = getHostByName(deviceName)
    except socket.error:
        pass


2nd instance
file: /opt/zenoss/Products/ZenModel/Device.py
function: manage_createDevice > setManageIp

if not ip:
    try:
        ip = getHostByName(ipunwrap(self.id))
    except socket.error:
        ip = ''
        if origip:
            message = ("%s is an invalid IP address, "
                "and no appropriate IP could"
                " be found via DNS for %s") % (origip, self.id)
            log.warn(message)
        else:
            message = "DNS lookup of '%s' failed to return an IP" % \
                        self.id


In both instances, when manageIp="", it tries to resolve its IP from its device name using getHostByName()
When I run getHostByName it takes exactly 20 seconds (default os timeout I think). I am now sure that this is the issue.

I tried setting the IP to "127.0.0.1" and this goes from 40s (20x2 + epsilon) all the way down to 52ms (epsilon).

I now get a warning that the ip is already in use.
I realize this is not standard. My Zenoss workflow is agent-based (custom agent that calls zendmd) and I don't let Zenoss access the devices by IP.
As such, I'd like to set empty IPs.

Any way I can achieve this without loadDevice taking 40 seconds?
Is there a global Zenoss configuration that disables this behavior?
If I am destined to use an IP, are there any implications to setting all my devices to 127.0.0.1? (except for a warning that the ip is already in use)?

Thank you


------------------------------
Jad
------------------------------


< Previous
Zenoss Shows Everything Down
  Next
Reindexing zodb in Zenoss 6
>