Zenoss ZenTech Community

I'm trying to create a backup of my Zenoss 5 in Control Center, and get the following error.

Error 500 unable to run cmd:[btrfs send /opt/serviced/var/volumes/4m0n6mbqijlljmcwssqfvki41_20150409-190449 -f /opt/serviced/var/backups/backup-2015-04-09-190445/snapshots/4m0n6mbqijlljmcwssqfvki41/4m0n6mbqijlljmcwssqfvki41_20150409-190449.0] output:At subvol /opt/serviced/var/volumes/4m0n6mbqijlljmcwssqfvki41_20150409-190449 ERROR: open opt_serviced_var_volumes/4m0n6mbqijlljmcwssqfvki41_20150409-190449 failed. No such file or directory error:exit status 1

I understand that it can't find that path, but could someone explain what is missing then Or why Here is my output of df -h

[root@zenoss tmp]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda7 67G 5.1G 62G 8% /
devtmpfs 7.8G 0 7.8G 0% /dev
tmpfs 7.8G 0 7.8G 0% /dev/shm
tmpfs 7.8G 1.8M 7.8G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/sda3 40G 3.4G 36G 9% /opt/serviced/var
/dev/sda2 147G 9.4G 136G 7% /var/lib/docker
/dev/sda2 147G 9.4G 136G 7% /opt/serviced/var/volumes
/dev/sda1 497M 90M 408M 18% /boot
/dev/sda5 40G 512K 38G 1% /opt/serviced/var/backups

Subject:	anyone?
Author:	[Not Specified]
Posted:	2015-04-13 10:15

anyone

Subject:	Check https://jira.zenoss.com
Author:	Jan Garaj
Posted:	2015-04-13 11:28

Check https://jira.zenoss.com/, there is a few reported problems "backup 500".

Devops Monitoring Expert advice: Dockerize/automate/monitor all the things.

DevOps stack: Docker / Kubernetes / Mesos / Zabbix / Zenoss / Grafana / Puppet / Ansible / Vagrant / Terraform / Elasticsearch

Subject:	Thanks Jan,
Author:	[Not Specified]
Posted:	2015-04-13 12:19

Thanks Jan,

I did see an issue but they can't replicate it.. I think this is to do with when I originally installed this CentOS7 I partitioned the /opt/serviced/var/backups as xfs that's what the install guide showed. When I went to run the autodeploy script it wouldn't even fire up because it was complaining about /opt/serviced/var/backups not being btrfs. So I tried to manually change it over. I show it as mounted. But its caused issues even on reboot, not liking that line in the /etc/fstab.
/dev/sda5 on /opt/serviced/var/backups type btrfs (rw,relatime,space_cache)

This was the original line in /etc/fstab. I had to comment it out just for the kernel to load, hence its not xfs anymore.
UUID=ee53b5d5-cf0b-4a3d-b4b5-cca6e1f8d6ae /opt/serviced/var/backups xfs defaults 1 2

Subject:	Good point:
Author:	Jan Garaj
Posted:	2015-04-13 13:13

Good point:
blame the author of autodeploy script :-D - /opt/serviced/var/backups should be xfs, not btrfs - fixed.

So did you format your /dev/sda5 as xfs (mkfs -t xfs -f /dev/sda5) and is it mounted as xfs (df -Th)

Subject:	Ah, that explains that. so it
Author:	[Not Specified]
Posted:	2015-04-13 13:30

Ah, that explains that. so it really should be xfs. okay, I have most likely goofed this entire mount up then and probably need your help manually remaking it from scratch. Here is my current output of df -Th

[root@zenoss ~]# df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda7 xfs 67G 5.9G 62G 9% /
devtmpfs devtmpfs 8.8G 0 8.8G 0% /dev
tmpfs tmpfs 8.8G 0 8.8G 0% /dev/shm
tmpfs tmpfs 8.8G 1.5M 8.8G 1% /run
tmpfs tmpfs 8.8G 0 8.8G 0% /sys/fs/cgroup
/dev/sda1 xfs 497M 89M 408M 18% /boot
/dev/sda3 xfs 40G 4.9G 35G 13% /opt/serviced/var
/dev/sda2 btrfs 147G 8.6G 137G 6% /var/lib/docker
/dev/sda2 btrfs 147G 8.6G 137G 6% /opt/serviced/var/volumes
/dev/sda5 btrfs 40G 512K 38G 1% /opt/serviced/var/backups

Subject:	You don't need to reinstall
Author:	Jan Garaj
Posted:	2015-04-13 13:45

You don't need to reinstall your Zenoss.

Unmount, xfs format, update fstab and mount:
umount /dev/sda5
mkfs -t xfs -f /dev/sda5
sed -i -e "\|^/dev/sda5|d" /etc/fstab
echo "/dev/sda5 /opt/serviced/var/backups xfs defaults 0 0" > /etc/fstab
mount /opt/serviced/var/backups

Subject:	I did the mkfs and remounted
Author:	[Not Specified]
Posted:	2015-04-13 13:56

I did the mkfs and remounted it as an xfs. That is all squared away, it correctly mounts now on a reboot etc. I've tried doing a backup again and same error.

[root@zenoss ~]# df -Th
Filesystem Type Size Used Avail Use% Mounted on
/dev/sda7 xfs 67G 5.9G 62G 9% /
devtmpfs devtmpfs 8.8G 0 8.8G 0% /dev
tmpfs tmpfs 8.8G 0 8.8G 0% /dev/shm
tmpfs tmpfs 8.8G 652K 8.8G 1% /run
tmpfs tmpfs 8.8G 0 8.8G 0% /sys/fs/cgroup
/dev/sda1 xfs 497M 89M 408M 18% /boot
/dev/sda3 xfs 40G 4.9G 35G 13% /opt/serviced/var
/dev/sda2 btrfs 147G 8.6G 137G 6% /var/lib/docker
/dev/sda2 btrfs 147G 8.6G 137G 6% /opt/serviced/var/volumes
/dev/sda5 xfs 40G 33M 40G 1% /opt/serviced/var/backups

Subject:	Not sure if this will help,
Author:	[Not Specified]
Posted:	2015-04-13 13:58

Not sure if this will help, but here is my /etc/fstab

/etc/fstab
# Created by anaconda on Wed Apr 1 04:05:58 2015
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=fcafde62-031e-4684-b29a-b816dd22268d / xfs defaults 1 1
UUID=74e99368-3e9a-40db-8412-c1e00c3ae2ab /boot xfs defaults 1 2
UUID=af3bdafd-448a-4e5e-97ca-ec77989945f7 /opt/serviced/var xfs defaults 1 2
UUID=b5e8ddaf-0bf5-4a55-b80e-cd1c50ac49cb /opt/serviced/var/backups xfs defaults 1 2
UUID=db4f03d0-560d-4286-b695-1a6b4c63900e /opt/serviced/var/volumes btrfs subvol=opt_serviced_var_volumes 1 2
UUID=db4f03d0-560d-4286-b695-1a6b4c63900e /var/lib/docker btrfs subvol=var_lib_docker 1 2
UUID=1d42ead0-fb92-40ef-9666-aa55621333eb swap swap defaults 0 0

Subject:	/opt/serviced/var/backups is
Author:	Jan Garaj
Posted:	2015-04-13 18:00

/opt/serviced/var/backups is not a problem

"subvol=" -> you are using subvolumes - maybe problem
Also you are not mounting with required btrf mount options "rw,noatime,nodatacow" - maybe problem

Subject:	Not sure what to do about the
Author:	[Not Specified]
Posted:	2015-04-14 10:09

Not sure what to do about the subvol=

Do I need to add those options in the fstab

Subject:	Nope - I think that btrfs
Author:	Jan Garaj
Posted:	2015-04-14 15:08

Nope - I think that btrfs subvolumes should not be used. Maybe they are your problems.

My fstab:
/dev/sdb1 /var/lib/docker btrfs rw,noatime,nodatacow 0 0
/dev/sdb2 /opt/serviced/var xfs defaults 0 0
/dev/sdc2 /opt/serviced/var/backups xfs defaults 0 0
/dev/sdc1 /opt/serviced/var/volumes btrfs rw,noatime,nodatacow 0 0

Subject:	How do I get rid of the
Author:	[Not Specified]
Posted:	2015-04-15 08:41

How do I get rid of the subvolume without blowing the box away

Subject:	I'm assuming I have to
Author:	[Not Specified]
Posted:	2015-04-20 16:29

I'm assuming I have to recreate the BTRFS partitions to eliminate the sub-volumes Which would mean having to recreate the entire Zenoss setup.

Subject:	Control Center Backup/ Restore Error 500
Author:	[Not Specified]
Posted:	2015-04-09 14:12

Control Center Backup/ Restore Error 500