TECHZEN Zenoss User Community ARCHIVE  

troubleshooting backup/restore problems by creating much smaller backups

Subject: troubleshooting backup/restore problems by creating much smaller backups
Author: Luke Lofgren
Posted: 2018-06-25 11:37

I'm troubleshooting problems with "out of space" errors both backing up and restoring things. They all happen "at the end" of a large full backup (almost 1 TB) for both the restore and and the backup. 

To speed things up I went ahead and used this command line to do a backup: 

serviced backup --exclude hbase-* --exclude etl-analytics --exclude mariadb-events --exclude mariadb-model --exclude zencatalogservice /opt/serviced/var/backups

I am an enterprise customer, but I'm wanting to share more with the community. The above backup dramatically reduces what is being backed up (to the point its not really a backup anymore), but it allowed me to recreate the backup error quickly. Now I'm going to proceed to see if I can use it to recreate my error with a restore. 

Version for the above: Control Center 1.5.1



------------------------------
Luke Lofgren
Infrastructure Architect
Acxiom Corporation (home based associate)
Waterford PA
------------------------------


Subject: RE: troubleshooting backup/restore problems by creating much smaller backups
Author: Luke Lofgren
Posted: 2018-06-25 12:12

The same command worked on CC 1.4.2, however I got 22 compressed GB which was more than what I was expecting -- however that's much better than the 350 GB files that are the result of a normal full backup.

------------------------------
Luke Lofgren
Infrastructure Architect
Acxiom Corporation (home based associate)
Waterford PA
------------------------------


Subject: RE: troubleshooting backup/restore problems by creating much smaller backups
Author: Luke Lofgren
Posted: 2018-06-25 13:55

This helped me self-diagnose that we were having problem with filling /var with docker images as part of backup/restore. 

While a link for a "tmp" folder is standard install instructions, this actually seems better:

update /etc/sysconfig/docker to add a NEW line to the file which reads:
DOCKER_TMPDIR=/opt/serviced/var/backups/tmp

Also, doing backups that exclude hbase and other things make for more concise ways to do upgrade validation in non-PROD environments. 



------------------------------
Luke Lofgren
Infrastructure Architect
Acxiom Corporation (home based associate)
Waterford PA
------------------------------


< Previous
Getting an error when select Devices under infrastructure menu
  Next
How to get Dynamic View to work in Zenoss RM 4.2.5 Enterprise
>