TECHZEN Zenoss User Community ARCHIVE  

Docker volume running out of space

Subject: Docker volume running out of space
Author: Martin
Posted: 2016-09-28 01:08

Good Morning,

unfortunately it looks like our Docker volume is going to run out of disk space. I'm wondering what's the best and safest way to increase disk space for this volume without the risk of loosing data. Can someone please point me into the right direction

Filesystem                                                                Size  Used Avail Use% Mounted on
/dev/sda2                                                                 101G  9.1G   87G  10% /
devtmpfs                                                                   63G     0   63G   0% /dev
tmpfs                                                                      63G     0   63G   0% /dev/shm
tmpfs                                                                      63G  1.9M   63G   1% /run
tmpfs                                                                      63G     0   63G   0% /sys/fs/cgroup
/dev/sdb2                                                                 365G   10G  355G   3% /opt/serviced/var/isvcs
/dev/sda1                                                                 4.5G  240M  4.0G   6% /boot
:/mnt//data/infra_nfs/  1.0T  308G  717G  31% /opt/serviced/var/backups
/dev/mapper/docker-8:2-2359915-7w0Di9g1K7V2SGp0InCAIf                      99G   82G   12G  88% /opt/serviced/var/volumes/cykew3tdcdk9oxtqesrzheygp
tmpfs                                                                      13G     0   13G   0% /run/user/0
      Name                Flags             Part Type        FS Type                    [Label]                 Size (MB)
--------------------------------------------------------------------------------------------------------------------------------
Unusable                                                1.05        *
sda1                Boot               Primary         ext4                                                 4999.62      	 *
sda2                                   Primary         ext4                                               109999.82      	 *
sda3                                   Primary         swap                                                31999.40      	 *
sda4                                   Primary         LVM2_member                                       1052638.19      	 *

Thanks, Martin



Subject: can you post the outputs of
Author: [Not Specified]
Posted: 2016-09-28 20:19

can you post the outputs of below commands.

lvs,lsblk and what version of cc are you running



Subject: lvs
Author: Martin
Posted: 2016-09-29 00:28

lvs
File descriptor 3 (pipe:[61190209]) leaked on lvs invocation. Parent PID 10158: -bash
LV            VG       Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
docker-pool   docker   twi-aot--- 341.65g             6.08   7.88
serviced-pool serviced twi-aot--- 881.43g             19.44  8.81
lsblk
NAME                                                                                        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                                                                                           8:0    0   1.1T  0 disk
sda1                                                                                        8:1    0   4.7G  0 part /boot
sda2                                                                                        8:2    0 102.5G  0 part /
sda3                                                                                        8:3    0  29.8G  0 part [SWAP]
sda4                                                                                        8:4    0 980.4G  0 part
serviced-serviced--pool_tmeta                                                           253:2    0  1004M  0 lvm
 serviced-serviced--pool                                                               253:5    0 881.4G  0 lvm
   docker-8:2-2359915-7w0Di9g1K7V2SGp0InCAIf                                           253:13   0   100G  0 dm   /exports/serv
serviced-serviced--pool_tdata                                                           253:3    0 881.4G  0 lvm
serviced-serviced--pool                                                               253:5    0 881.4G  0 lvm
docker-8:2-2359915-7w0Di9g1K7V2SGp0InCAIf                                           253:13   0   100G  0 dm   /exports/serv
sdb                                                                                           8:16   0 744.6G  0 disk
sdb1                                                                                        8:17   0   380G  0 part
 docker-docker--pool_tmeta                                                               253:0    0   392M  0 lvm
  docker-docker--pool                                                                   253:4    0 341.7G  0 lvm
    docker-8:2-6030882-822380afef93cc1c8ab45002b3cac759332841b1f2a9e647dc7c7851860d2468 253:6    0   100G  0 dm
    docker-8:2-6030882-44a5ecc6e501c65ee895e7c1612a4d45927d590271647073d100727edd26fd14 253:7    0   100G  0 dm
    docker-8:2-6030882-3fb5d06ef78ac7690a3c1dededeee11ea08eab0e6390b745bd401e15eb4cff52 253:8    0   100G  0 dm
    docker-8:2-6030882-e655e90707f09071634b3319ad81800d131f20201bb26064f18c24c1c8d46d3d 253:9    0   100G  0 dm
    docker-8:2-6030882-d8e0e785ad61010c14df35bd6a5b551d20582e1dea03722f7acbf63128305127 253:10   0   100G  0 dm
    docker-8:2-6030882-f6e64ccbc85cac0b5a1634c4ae56bc16fa4bd2945da0bd0875339ad8bd00090c 253:11   0   100G  0 dm
    docker-8:2-6030882-51473709dfade95dc7360c21204ced3d728772ae92db1f1c4d8dfe7484440448 253:12   0   100G  0 dm
    docker-8:2-6030882-fbbb10b09b474f115705f5ca58b8fc13af9b1253b9092218e917471ebddab25c 253:14   0   100G  0 dm
    docker-8:2-6030882-7e3af7827bb319268cb538d7e939a318e06bbe032a36fb3cd47232f6d2d2011d 253:15   0   100G  0 dm
    docker-8:2-6030882-e517c93ca0f584031f5634c048021c1714d48aeb32c7254b55636a36daf31608 253:16   0   100G  0 dm
    docker-8:2-6030882-d8cbbc1946f4fb0d1c78098bdb2f213046300af1850d860fa0b47a888bd16b0e 253:17   0   100G  0 dm
    docker-8:2-6030882-e07ab9e3390463154e0276cc84bbe575bcfa27e3b3b1c9013f06cdef7051d262 253:18   0   100G  0 dm
    docker-8:2-6030882-0e7ea12be1749119948180a97cf47c2a12c1cd3608d2d0237a373a748f28d040 253:19   0   100G  0 dm
    docker-8:2-6030882-f9651113f06daa6b2b624b3c7368ec111e48d052c666ee735ab9cdd37d14c86e 253:20   0   100G  0 dm
    docker-8:2-6030882-876ed1713954d344126d9280d0f078173906af733781c20170a8cb4dd307f17b 253:21   0   100G  0 dm
    docker-8:2-6030882-f63aa6ba6e9abf29c9183445cf0f47657e7cd42dc64d064befbf805d3e497753 253:22   0   100G  0 dm
    docker-8:2-6030882-b2f45ada9db16614cbaf270ff2fdac736409276ed675f4789c53f72d0298bf75 253:23   0   100G  0 dm
    docker-8:2-6030882-c009200bfb5db805bc191df6eb936d6978558d2b1646ef36f7df955584ad7818 253:24   0   100G  0 dm
    docker-8:2-6030882-0be88865a762500b50e5c8f580eb163ea5c85636d5c35a228dbc956c8c01f76c 253:25   0   100G  0 dm
    docker-8:2-6030882-cdc90aca33d3fa3b9d96c9c7aa12fa59b2cd0e15cc01bf0e77b7c09233a44c42 253:26   0   100G  0 dm
    docker-8:2-6030882-4e93123cb419df1674faa290d2286f2907d0a9ff50937a653299bfdd247a2d7d 253:27   0   100G  0 dm
    docker-8:2-6030882-2424d9819270102fd45f0fae1c04f771c8f282a7a89e44151a698b0ca1475c0f 253:28   0   100G  0 dm
    docker-8:2-6030882-bf43311eddf8ed486f97d94cf5411363d145273faf2ede4805f7a85410f18e13 253:29   0   100G  0 dm
    docker-8:2-6030882-515eea68e9ae3717bda2b147e828880ef7aeaf73d32c655ba11111a05dc6df1d 253:30   0   100G  0 dm
    docker-8:2-6030882-abe3cf8a5dcf366ee9239024119f2f31c23e6766377ffe49befc562b2532e07f 253:31   0   100G  0 dm
    docker-8:2-6030882-e5391f11e1237419d68ae842c1d9d63a4c7611da242f3d09d4ceabb97436940d 253:32   0   100G  0 dm
    docker-8:2-6030882-29fae6c5d855dd639427fe0fba59f979d133ad899898f13c6613e3b3f9d07d3c 253:33   0   100G  0 dm
    docker-8:2-6030882-10a259a3dd80306258db73985820f84a9ced63e93118f02a957b2f28e605d63f 253:34   0   100G  0 dm
    docker-8:2-6030882-38faeb9dcd17f37b3cf402a846094007d4f50c636a1c3a0cfeef075013ac5201 253:35   0   100G  0 dm
    docker-8:2-6030882-eaa27d49e47c3876ab6783bf754be853e25d57730f7d5b545c0e34716109ca63 253:36   0   100G  0 dm
    docker-8:2-6030882-8bd896b269b43432a8fadef22e5ba561ded8b4db9240681fce89feb274ee5369 253:37   0   100G  0 dm
    docker-8:2-6030882-31f4eadbb0d2247f27eaab6fcebd4f269dcd1ec6a2aad624dc2fc3b88431ae69 253:38   0   100G  0 dm
    docker-8:2-6030882-efbdbd5de0ee6001c1a2b621637a7701f7c2e74c630ec757aa9009c0b8ebf750 253:39   0   100G  0 dm
    docker-8:2-6030882-67059e6555a8ec3b01814d0b416b19661a5e6964a9584d28d2745c3c3382365b 253:40   0   100G  0 dm
    docker-8:2-6030882-c22da2e1a2c0830600a411e6b4828b60fffbda7da1de6c188ba5c1cf37cc5538 253:41   0   100G  0 dm
    docker-8:2-6030882-e517aef1639ec16ad6b1f8513a19730eb6594fb3d72414777fb4d8c71fbcc86b 253:42   0   100G  0 dm
    docker-8:2-6030882-8ad3c5ea6715ea241f70165a5c0c164e18d4e64d78acaa7c07e5e99cc9720e36 253:43   0   100G  0 dm
    docker-8:2-6030882-1422e97972ab9e29f879ae568bca5335f402986becd51037a488ad61ec69d27a 253:44   0   100G  0 dm
 docker-docker--pool_tdata                                                               253:1    0 341.7G  0 lvm
   docker-docker--pool                                                                   253:4    0 341.7G  0 lvm
     docker-8:2-6030882-822380afef93cc1c8ab45002b3cac759332841b1f2a9e647dc7c7851860d2468 253:6    0   100G  0 dm
     docker-8:2-6030882-44a5ecc6e501c65ee895e7c1612a4d45927d590271647073d100727edd26fd14 253:7    0   100G  0 dm
     docker-8:2-6030882-3fb5d06ef78ac7690a3c1dededeee11ea08eab0e6390b745bd401e15eb4cff52 253:8    0   100G  0 dm
     docker-8:2-6030882-e655e90707f09071634b3319ad81800d131f20201bb26064f18c24c1c8d46d3d 253:9    0   100G  0 dm
     docker-8:2-6030882-d8e0e785ad61010c14df35bd6a5b551d20582e1dea03722f7acbf63128305127 253:10   0   100G  0 dm
     docker-8:2-6030882-f6e64ccbc85cac0b5a1634c4ae56bc16fa4bd2945da0bd0875339ad8bd00090c 253:11   0   100G  0 dm
     docker-8:2-6030882-51473709dfade95dc7360c21204ced3d728772ae92db1f1c4d8dfe7484440448 253:12   0   100G  0 dm
     docker-8:2-6030882-fbbb10b09b474f115705f5ca58b8fc13af9b1253b9092218e917471ebddab25c 253:14   0   100G  0 dm
     docker-8:2-6030882-7e3af7827bb319268cb538d7e939a318e06bbe032a36fb3cd47232f6d2d2011d 253:15   0   100G  0 dm
     docker-8:2-6030882-e517c93ca0f584031f5634c048021c1714d48aeb32c7254b55636a36daf31608 253:16   0   100G  0 dm
     docker-8:2-6030882-d8cbbc1946f4fb0d1c78098bdb2f213046300af1850d860fa0b47a888bd16b0e 253:17   0   100G  0 dm
     docker-8:2-6030882-e07ab9e3390463154e0276cc84bbe575bcfa27e3b3b1c9013f06cdef7051d262 253:18   0   100G  0 dm
     docker-8:2-6030882-0e7ea12be1749119948180a97cf47c2a12c1cd3608d2d0237a373a748f28d040 253:19   0   100G  0 dm
     docker-8:2-6030882-f9651113f06daa6b2b624b3c7368ec111e48d052c666ee735ab9cdd37d14c86e 253:20   0   100G  0 dm
     docker-8:2-6030882-876ed1713954d344126d9280d0f078173906af733781c20170a8cb4dd307f17b 253:21   0   100G  0 dm
     docker-8:2-6030882-f63aa6ba6e9abf29c9183445cf0f47657e7cd42dc64d064befbf805d3e497753 253:22   0   100G  0 dm
     docker-8:2-6030882-b2f45ada9db16614cbaf270ff2fdac736409276ed675f4789c53f72d0298bf75 253:23   0   100G  0 dm
     docker-8:2-6030882-c009200bfb5db805bc191df6eb936d6978558d2b1646ef36f7df955584ad7818 253:24   0   100G  0 dm
     docker-8:2-6030882-0be88865a762500b50e5c8f580eb163ea5c85636d5c35a228dbc956c8c01f76c 253:25   0   100G  0 dm
     docker-8:2-6030882-cdc90aca33d3fa3b9d96c9c7aa12fa59b2cd0e15cc01bf0e77b7c09233a44c42 253:26   0   100G  0 dm
     docker-8:2-6030882-4e93123cb419df1674faa290d2286f2907d0a9ff50937a653299bfdd247a2d7d 253:27   0   100G  0 dm
     docker-8:2-6030882-2424d9819270102fd45f0fae1c04f771c8f282a7a89e44151a698b0ca1475c0f 253:28   0   100G  0 dm
     docker-8:2-6030882-bf43311eddf8ed486f97d94cf5411363d145273faf2ede4805f7a85410f18e13 253:29   0   100G  0 dm
     docker-8:2-6030882-515eea68e9ae3717bda2b147e828880ef7aeaf73d32c655ba11111a05dc6df1d 253:30   0   100G  0 dm
     docker-8:2-6030882-abe3cf8a5dcf366ee9239024119f2f31c23e6766377ffe49befc562b2532e07f 253:31   0   100G  0 dm
     docker-8:2-6030882-e5391f11e1237419d68ae842c1d9d63a4c7611da242f3d09d4ceabb97436940d 253:32   0   100G  0 dm
     docker-8:2-6030882-29fae6c5d855dd639427fe0fba59f979d133ad899898f13c6613e3b3f9d07d3c 253:33   0   100G  0 dm
     docker-8:2-6030882-10a259a3dd80306258db73985820f84a9ced63e93118f02a957b2f28e605d63f 253:34   0   100G  0 dm
     docker-8:2-6030882-38faeb9dcd17f37b3cf402a846094007d4f50c636a1c3a0cfeef075013ac5201 253:35   0   100G  0 dm
     docker-8:2-6030882-eaa27d49e47c3876ab6783bf754be853e25d57730f7d5b545c0e34716109ca63 253:36   0   100G  0 dm
     docker-8:2-6030882-8bd896b269b43432a8fadef22e5ba561ded8b4db9240681fce89feb274ee5369 253:37   0   100G  0 dm
     docker-8:2-6030882-31f4eadbb0d2247f27eaab6fcebd4f269dcd1ec6a2aad624dc2fc3b88431ae69 253:38   0   100G  0 dm
     docker-8:2-6030882-efbdbd5de0ee6001c1a2b621637a7701f7c2e74c630ec757aa9009c0b8ebf750 253:39   0   100G  0 dm
     docker-8:2-6030882-67059e6555a8ec3b01814d0b416b19661a5e6964a9584d28d2745c3c3382365b 253:40   0   100G  0 dm
     docker-8:2-6030882-c22da2e1a2c0830600a411e6b4828b60fffbda7da1de6c188ba5c1cf37cc5538 253:41   0   100G  0 dm
     docker-8:2-6030882-e517aef1639ec16ad6b1f8513a19730eb6594fb3d72414777fb4d8c71fbcc86b 253:42   0   100G  0 dm
     docker-8:2-6030882-8ad3c5ea6715ea241f70165a5c0c164e18d4e64d78acaa7c07e5e99cc9720e36 253:43   0   100G  0 dm
     docker-8:2-6030882-1422e97972ab9e29f879ae568bca5335f402986becd51037a488ad61ec69d27a 253:44   0   100G  0 dm
sdb2                                                                                        8:18   0 364.6G  0 part /opt/serviced
cc -v
Using built-in specs.
COLLECT_GCC=cc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC)

Thanks, Martin



Subject: Please do below with root
Author: [Not Specified]
Posted: 2016-09-29 22:45

Please do below with root user or user with sudo permissions

You can safely do upto 400G. So your storage shouls be 400G

serviced-storage resize -d /opt/serviced/var/volumes -o dm.thinpooldev=/dev/mapper/serviced-serviced--pool

Thanks,

krishna



Subject: Works like charm :)
Author: Martin
Posted: 2016-09-30 02:04

Works like charm :)

Thanks, Martin



< Previous
ZenOSS logfile monitoring
  Next
Linux Monitor Package
>