Did you follow the steps described in this article (Permanent or Semi-Permanent)?
https://support.zenoss.com/hc/en-us/articles/207610516-How-to-add-tools-or-scripts-into-a-Resource-Manager-5-x-Docker-Container
Subject: |
RE: serviced snapshot commit |
Author: |
Joana Sianard |
Posted: |
2019-03-07 17:27 |
hello
I have the same problem when I install a zenpack or I restore !
INFO:zen.ZenPackCmd:Successfully restored zenpacks!
FATA[0339] Unable to commit container containerid=b8236d11c88d6ec302d0c9b2881f053a0b0fccdf2f05a2c34232f2eafea9cb80 containername=4jpVZOxxuVDkEB9egl44Dw error=API error (500): {"message":"max depth exceeded"}
location=shell.go:264 logger=cli.api
------------------------------
JOANA SIANARD
SYSADMIN
GLOBECAST
PARIS FRANCE
Zenoss 5.3.3 r419
Control Center ControlCenter 1.4.2
------------------------------
Subject: |
RE: serviced snapshot commit |
Author: |
Arthur |
Posted: |
2019-03-08 16:35 |
Can you post the output of the following commands:
# docker info && serviced volume status
# img=$(docker images |grep core |grep latest | awk {'print $1'}) && docker history $img
------------------------------
Arthur
------------------------------
Subject: |
RE: serviced snapshot commit |
Author: |
Joana Sianard |
Posted: |
2019-03-08 18:05 |
Hello Arthur,
the files.
Regards,
JOANA
------------------------------
JOANA SIANARD
GLOBECAST
------------------------------
Attachments:
docker-history.txt
docker-info.txt
volume-status.txt
Subject: |
RE: serviced snapshot commit |
Author: |
Arthur |
Posted: |
2019-03-09 03:01 |
Hello Joana
It looks like your image has too many layers (126). As I haven't seen this message so far this is onl my assumption and I can't give you a solution yet. You can google for docker AND "message":"max depth exceeded" for example to find some help how to solve it.
Anyhow I would be interested about your solution/findings. If I have some free time I'll try to reproduce it, but I can't give you any date so far.
------------------------------
Arthur
------------------------------
Subject: |
RE: serviced snapshot commit |
Author: |
Andrew Kirch |
Posted: |
2019-03-13 14:35 |
This happens when your docker container has a lot of layers. Have you added a lot of things to this before? (Zenpacks etc). I would suggest flattening it if so. Note that you may lose the ability to roll back.
https://tuhrig.de/flatten-a-docker-container-or-image/
------------------------------
Andrew Kirch
Senior Solutions Engineer
GoVanguard
------------------------------
Has anyone found a workable solution to this issue? I'm running into the same problem on Zenoss Core 6.3.2.
Subject: |
RE: serviced snapshot commit |
Author: |
Michael Rogers |
Posted: |
2021-11-05 18:55 |
Duane,
The following steps should take care of things.
0) Note - There is a restart of the Zenoss application in this process under step 10
1) First we need to find out what our Zenoss Core image is by running:
[root@core ~]# ZCORE="$(docker images | grep core_ | awk '{print $1}' | head -1)"
[root@core ~]# echo $ZCORE
localhost:5000/5ipo4ged9ffy4gze3zqkq2tdh/core_6.1
2) Check current image layer count:
[root@core ~]# docker history $ZCORE | wc -l
37
3) We need the sha256 hash of the latest layer from the docker history once we have that we run:
[root@core ~]# IMAGE_ID="$(docker history --no-trunc -q $ZCORE | head -1)"
[root@core ~]# echo $IMAGE_ID
sha256:bcc1c00a7c9fd08c19566fd7e1ccbffe61477801279aac8b0356076716005834
4) Create a backup of the current image:
[root@core ~]# docker tag $ZCORE:latest $ZCORE:latest-backup
[root@core ~]# docker images | grep backup
localhost:5000/5ipo4ged9ffy4gze3zqkq2tdh/core_6.1 latest-backup bcc1c00a7c9f 8 weeks ago 2.22GB
5) Create a new container based on the latest layer:
[root@core ~]# docker run --name "corecontainer" $IMAGE_ID /bin/bash
[root@core ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
09efecb488be bcc1c00a7c9f "/bin/bash" 7 seconds ago Exited (0) 5 seconds ago corecontainer
* If serviced is running you will also see other containers including each isvcs
6) Use this manually started container to flatten the current image by performing an export/import (this will take a few minutes):
[root@core ~]# docker export corecontainer | docker import - flat-corecontainer
sha256:d99d9c93c6e65bfe905079382bbc9f081b58efaf420b678b1a93c9ebbdfb5874
* The sha256 line will show when complete
7) Override the current image with the new, flattened image:
[root@core ~]# serviced docker override $ZCORE:latest flat-corecontainer:latest
[root@core ~]# docker images | grep latest | head -n3
flat-corecontainer latest d99d9c93c6e6 2 minutes ago 1.96GB
localhost:5000/5ipo4ged9ffy4gze3zqkq2tdh/core_6.1 latest d99d9c93c6e6 2 minutes ago 1.96GB
localhost:5000/5ipo4ged9ffy4gze3zqkq2tdh/core_6.1 latest-backup bcc1c00a7c9f 8 weeks ago 2.22GB
8) Verify your image has been flattened:
[root@core ~]# docker history $ZCORE | wc -l
2
9) Asynchronously push all images from the serviced Docker registry index into the Docker registry (may take a minute to complete):
[root@core ~]# serviced docker sync
* Command produces no output
10) Restart Zenoss Core to pull the latest image for each container:
serviced service restart Zenoss.core
* Restarting immediately isn't necessarily required, but plan to perform a restart before performing any further image edits.
If the process fails, your original container will still be backed up with the latest-backup tag from Step 4.
Let us know if this helps?
------------------------------
Michael J. Rogers
Senior Instructor - Zenoss
Austin TX
------------------------------
Subject: |
RE: serviced snapshot commit |
Author: |
Duane Wylie |
Posted: |
2021-11-05 20:25 |
Thanks, Michael.
I've run through this process without issue through step 9. (I'll restart a bit later).
On the off chance that this detonates in my face, I imagine the rollback would look something like this?
[root@core ~]# serviced docker override $ZCORE:latest $ZCORE:latest-backup
[root@core ~]# serviced docker sync
serviced service restart Zenoss.core
------------------------------
Duane Wylie
------------------------------
Subject: |
RE: serviced snapshot commit |
Author: |
Michael Rogers |
Posted: |
2021-11-05 21:48 |
Duane,
I should have included the instructions for rollback in my last post, but that looks perfect.
------------------------------
Michael J. Rogers
Senior Instructor - Zenoss
Austin TX
------------------------------
Subject: |
RE: serviced snapshot commit |
Author: |
Duane Wylie |
Posted: |
2021-11-05 23:56 |
After restarting with the flattened image, stats from zenperfsnmp weren't being graphed. (but stats from zenprocess were.)
I tried reverting to the $ZCORE:latest-backup, but this didn't resolve my issue.
I'm restoring the system from backup (hopefully I haven't completely lost the system).
I was seeing a lot of errors like this in various container logs:
error="Identity token signature cannot be verified"
------------------------------
Duane Wylie
------------------------------