OpenStack Glance: disable cache management while using Ceph RBD
The OpenStack documentation often recommends to enable the Glance cache while using the default store
file, with the Ceph RBD backend things are slightly different.
Depending on how you are consuming your Cloud platform, using the
keystone+cachemanagement in Glance can be extremely harmful.
Enabling this cache will result in images getting cached in your OpenStack controllers under
/var/lib/glance/image-cache/ everytime an image goes into
Implying that you booted a virtual machine, just think about a 50 GB big Windows image, now think about 10 of them.
This will potentially lead to a full system sooner or later unless your entire
/var/ is mounted on a big volume.
Everytime a virtual machine is booted often, this image will go into the cache.
As mentioned, the cache gets activated when an image goes into
/var/lib/nova/instances/_base, this can happen during multiple circumstances:
- You are using Juno but you are using QCOW2 images (for Ceph we want RAW images)
- You are using an OpenStack version prior to Juno and didn’t apply the patch that provides support for COW clones
This means that People only using RAW images and the COW clones in Nova will not be affected since nothing goes into
Everything is happening at the Ceph level (image snapshot and clone creation).
Now assuming your cache was enabled, let’s check the images present in the cache:
$ glance-cache-manage list-cached
Delete the cached images:
$ glance-cache-manage -f delete-all-cached-images
Now it is time to disable it, edit your
[paste_deploy] flavor = keystone
Then restart Glance:
$ sudo glance-control all restart
Ceph doc impact has already been submitted and merged.