I have a Ceph cluster, and created rules as well as Pools. After words I assigned these pools to rules created for this. What I want to know is how can I be sure that it actually works as advertised?
Primarily I want to know that data is truly being written to SAS drives and not SSD, except when I want to. Is there anyway to get beyond the pool definition and look at the block device itself to see where data was written.
Reason I am asking this is because the performance is exceptionally fast and about the same on both pools. This is possible (20G connection, small file), but want to be 100% sure.
The ceph osd map command shows which OSD is used when storing an object in a pool. For instance
$ ceph osd map rbd OBJECT
osdmap e11 pool 'rbd' (0) object 'OBJECT' ->
pg 0.383caa09 (0.1) -> up ([2,0,1], p2) acting ([2,0,1], p2)
tells you that [2,0,1] is the list of OSD id used to store OBJECT and its replicas, in the rbd pool. With the ceph osd tree command you can then verify which device is used by a given OSD id:
$ ceph osd tree
ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY
-1 3.00000 root default
-2 3.00000 host fold
0 1.00000 osd.0 up 1.00000 1.00000
1 1.00000 osd.1 up 1.00000 1.00000
2 1.00000 osd.2 up 1.00000 1.00000
In this example we learn that osd.0, osd.1 and osd.2 are on the host fold. In the case of a separation between SAS disks and SATA disks, the ceph osd tree command will provide you with a similar information.