If Ceph replicates a file in whole in multiple places can I pull one of those drives and insert it into a dock and read the file on another computer.
I think the answer to the way you intend the question, is No. Ceph isn’t just a really fancy version of MergerFS, Unraid or such. Files are split as objects across the OSDs in a flat namespace with the metadata like the POSIX file name stored separately, and the filestore used by the OSDs is currently by default a raw file system called BlueStore. I think in the past it was XFS (and may be a configuration option), but even then you need the CephFS metadata to reconstruct the file.
See
https://docs.ceph.com/en/latest/architecture/#storing-data
https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/
https://docs.ceph.com/en/latest/cephfs/disaster-recovery/
Of course these are just 1s and 0s, so in a forensic data recovery situation with the right tools and knowledge it’s possible that files or parts of files could be recovered and reconstructed, but I don’t think that is what you are referring to.
No, this is not possible.
As @DigitalGarden said ceph does not act like a typical RAID/Filesystem.
In ceph, all your data is broken into objects and these objects are what are distributed across the clusters OSDs so every 4k of data is a new object that ceph makes.
The metadata to these objects are also spread across the cluster’s OSDs so simply taking 1 HDD out of the cluster will not give you a full copy of data and metadata for you to use elsewhere.
It does however mean that if you have a 3-node cluster(using host failure) and 2 nodes die you still have 1 full copy intact of both the data and metadata on that last standing host meaning you can still retrieve the data from that 1 system, but again this data would be across all OSDs in that host not a single drive.
1 Like