Ceph Jewel: configure BlueStore with multiple devices
As presented in my preview of BlueStore, this new store has the ability tp be configured with multiple devices.
ceph-disk utility does not support configuring multiple devices, OSD must be configured manually.
Let’s see how we can configure this.
As a reminder, BlueStore supports 3 ways to store several kind of data and metadata. Such as:
- Data: stored on a RAW block device, typically a HDD
- Metadata: stored on a RAW block device, can be on a SSD
- RocksDB write-ahead log: stored on a RAW block device, can be on a Non-volatile Memory (NVM) or Non-volatile random-access memory (NVRAM) or persistent memory
It is quite simple to configure those on separate devices, for this you have two options:
- Register an OSD section in your
- Use symlinks that point to the device partition
Register OSD IDs like so:
[osd.0] host = ceph-01 osd data = /var/lib/ceph/osd/ceph-0/ bluestore block path = /dev/disk/by-partlabel/osd-device-0-block bluestore block db path = /dev/disk/by-partlabel/osd-device-0-db bluestore block wal path = /dev/disk/by-partlabel/osd-device-0-wal
Simply create the following symlinks:
block: pointing to the device where data will be stored
block.db: pointing to the device that store RocksDB metada
block.wal: pointing to the device that stores RocksDB write-ahead journal
So this will give us:
$ sudo ln -sf /dev/disk/by-partlabel/osd-device-0-block /var/lib/ceph/osd/ceph-0/block
The support from
ceph-diskshould likely land in the next release of Jewel. Yet another reminder regarding BlueStore. Don’t get to excited, BlueStore is still early ages and should not be deployed on production environment with sensitive data. So make sure you always perform backup because the risk of losing data exists.