I am newer to the Ceph world than I am to the Proxmox VE world and want to maximize the use of my fairly new 3-node Ceph cluster.
Currently I am running Ceph using the Proxmox VE tooling on 3 old QNAP whitebox NAS boxes with 4 OSDs per node and 2 SSDs for the OS, and while this has been working really well for it purpose of providing shared storage for my Docker Swarm cluster and for ISO storage for my main Proxmox cluster of 7-nodes.
I looking for the best way to also allow for NFS or CIFS shares to work using the pool as well.
I typically use TrueNAS for my NFS and CIFS shares but I am not sure how I would mount the CephFS share on it and dont want to go down the method of creating virtual disks for use with TrueNAS.
Is this Cluster a Proxmox Hyper converged ceph cluster?
What we typically do for this would be to spin up a VM “gateway” server which would mount the cephfs filesystem then reshare it out with SMB or NFS
So all you would need is to mount cephfs and then setup a local SMB/SMB share on that mount point
I have both, a 7 node proxmox cluster that is hosting both VMs and ceph and i have a separate 3 node cluster that was setup using Proxmox but is not hosting anything. Though i do plan at some point to move it to Debian and setup ceph without proxmox.
If that is the case do you have cephfs deployed on the cluster using the proxmox tools?
If you do you can mount that to a VM “gateway” which will then reshare out the cephfs mount as SMB or NFS depending on what you want to use it for.
Yes i have both CephFS and RDB using the proxmox tooling. I will look into creating a gateway VM.
OK so depending on your use case if your want the cephfs to be accessible to Linux clients you can just mount cephfs natively from the Ceph cluster
If you need SMB or NFS then you can mount it to a Vm "gateway"system and setup a share on that cephfs mountpoint
Forgive the possibly silly followup question but I am assuming that in order to setup a gateway it would be creating a VM using a Linux distro of choice (mine is Debian) and then configuring it to mount the CephFS share and then installing and configuring a CIFS/SMB and NFS services?
Yes correct, that way your end clients can see the SMB/NFS shares but its still going to the cephfs filesystem on the cluster for the HA