Bootstrap two Ceph and configure RBD mirror using Ceph Ansible
Since Jewel is out everyone wants to try the new RBD-mirroring feature.
This development environment assumes that you have two virtual machines, with two block devices attached.
I ran my tests on two CentOS 7.2 virtual machines.
SUBNET=$(ip r | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/[0-9]\{1,2\}' | head -1)
~/ceph-ansible$ cp site.yml.sample site.yml ~/ceph-ansible$ mkdir cluster-{primary,secondary} ~/ceph-ansible$ echo cluster-{primary,secondary} | xargs -n 1 cp -a group_vars ~/ceph-ansible$ cp cluster-primary/group_vars/all.sample cluster-primary/group_vars/all ~/ceph-ansible$ cp cluster-primary/group_vars/osds.sample cluster-primary/group_vars/osds ~/ceph-ansible$ cp cluster-secondary/group_vars/osds.sample cluster-secondary/group_vars/osds ~/ceph-ansible$ cp cluster-secondary/group_vars/all.sample cluster-secondary/group_vars/all ~/ceph-ansible$ sed -i "s/#osd_auto_discovery: false/osd_auto_discovery: true/" cluster-{primary,secondary}/group_vars/osds ~/ceph-ansible$ sed -i "s/#journal_collocation: false/journal_collocation: true/" cluster-{primary,secondary}/group_vars/osds ~/ceph-ansible$ sed -i "s/#monitor_address: 0.0.0.0/monitor_address: 192.168.0.50/" cluster-primary/group_vars/all ~/ceph-ansible$ sed -i "s/#monitor_address: 0.0.0.0/monitor_address: 192.168.0.51/" cluster-secondary/group_vars/all ~/ceph-ansible$ sed -i "s|#public_network: 0.0.0.0\/0|public_network: ${SUBNET}|" cluster-{primary,secondary}/group_vars/all ~/ceph-ansible$ sed -i "s/#journal_size: 0/journal_size: 100/" cluster-{primary,secondary}/group_vars/all ~/ceph-ansible$ sed -i "s/#common_single_host_mode: true/common_single_host_mode: true/" cluster-{primary,secondary}/group_vars/all ~/ceph-ansible$ sed -i "s/#pool_default_size: 3/pool_default_size: 2/" cluster-{primary,secondary}/group_vars/all ~/ceph-ansible$ sed -i "s/#ceph_dev: false/ceph_dev: true/" cluster-{primary,secondary}/group_vars/all ~/ceph-ansible$ sed -i "s|#ceph_dev_branch: master|ceph_dev_branch: jewel|" cluster-{primary,secondary}/group_vars/all ~/ceph-ansible$ sed -i "s|#cluster: ceph |cluster: primary |" cluster-primary/group_vars/all # the space after 'primary' is important ~/ceph-ansible$ sed -i "s|#cluster: ceph |cluster: secondary |" cluster-secondary/group_vars/all # the space after 'secondary' is important
In the end I have the following configuration:
~/ceph-ansible$ egrep -v '^#|^$' cluster-{primary,secondary}/group_vars/all cluster-primary/group_vars/all:--- cluster-primary/group_vars/all:dummy: cluster-primary/group_vars/all:cluster: primary # cluster name cluster-primary/group_vars/all:ceph_dev: true# use ceph development branch cluster-primary/group_vars/all:ceph_dev_branch: jewel # development branch you would like to use e.g: master, wip-hack cluster-primary/group_vars/all:monitor_address: 192.168.0.50 cluster-primary/group_vars/all:journal_size: 100 cluster-primary/group_vars/all:pool_default_size: 2 cluster-primary/group_vars/all:public_network: 192.168.0.0/24 cluster-primary/group_vars/all:common_single_host_mode: true cluster-secondary/group_vars/all:--- cluster-secondary/group_vars/all:dummy: cluster-secondary/group_vars/all:cluster: secondary # cluster name cluster-secondary/group_vars/all:ceph_dev: true# use ceph development branch cluster-secondary/group_vars/all:ceph_dev_branch: jewel # development branch you would like to use e.g: master, wip-hack cluster-secondary/group_vars/all:monitor_address: 192.168.0.51 cluster-secondary/group_vars/all:journal_size: 100 cluster-secondary/group_vars/all:pool_default_size: 2 cluster-secondary/group_vars/all:public_network: 192.168.0.0/24 cluster-secondary/group_vars/all:common_single_host_mode: true
Comments