I’ll probably only fill the HL15 with 6 or 8 drives (16 or 18TB Exos) and maybe 2 SATA SSD’s to start with, this will see me through the next few years after which I could add another bunch of drives for a second pool.
Mainly I will be storing media files which are written once and afterwards only rarely read. Anything that needs performance will go on an NVME drive anyway. If I would have to pick I’d pick read speed over write speed. Writing would also use RAM and maybe an ZiL drive so I wouldn’t notice any performance drops on the HDDs probably.
Anyway I was contemplating about how to setup the pool. RAIDZ or RAIDZ2 would give me max capacity with some redundancy, a striped mirror (RAID10) would half my space but give some performance benefits. But I don’t think I need that performance.
One of the SATA SSDs could be used as a caching drive, although I plan to have 48 or 64GB of RAM as well.
Would striped RAIDZ be an option? Effectively creating a RAID50. Or should I just go with RAIDZ or RAIDZ1? What about dRAID?
The server will have a 10GE NIC so going beyond at least 2,5 or 5GE would be nice.
Without asking more questions about your backup strategy, high availability needs and the devices that are writing and consuming the media, I’d suggest RAIDZ2. With only 6-8 typical 3.5" NAS drives (assuming not dual actuator, etc), you’re probably not going to be saturating a 10GB connection on read or write.
as DigitalGarden said without asking too many questions about what you’re trying to do, in Truenas scale striped mirrors have the benefit of being expandable, quicker to rebuild and better IOPS, but just generally raidz2 sounds like a reliable fine bet. I think an ideal for raidz2 is considered 6 drives, raidz1 isn’t recommended because of failures during rebuild especially with very large drives that take a while to rebuild
This is what I’ve often heard as well. RAIDz2 would get you two disks of parity and your total usable storage uses the formula:
n-2 * sTB = Usable
So taking the 16TB drives for example and using 6 of them.
6 - 2 = 4 drives * 16TB = 64TB usable
I think the 6 drives rule is because just adding two more drives DOUBLES your usable storage with RAIDz2. The large disadvantage for RAIDz1 is that it’s similar to RAID 5, resilvering takes forever at that size, and you’re now under more stress during the rebuild… with only 1 drive failure away from complete data loss.
Usecase: Just storing media files will be 90% of my use. I’ll run some containers and Plex. All on TrueNAS Scale.
Backups: The media files can be redownloaded so these datasets won’t be backupped. Anything remotely important gets synced to OneDrive (only like 20 GB worth of data).
HA: Not important
While I would like the benefits op striped mirrors it would eat into my usable space a lot. I could add more drives but this whole build is over €3000 already so I’d rather not
So RAIDZ2 over Striped RAIDZ (2x 3-4) drive (RAID50)? I have never used the latter so no experience with that. IOPS I don’t care too much about. No VMs will run on the HDD pool anyway. I would like to be able to write large files from my desktop to the server at some speeds above 1 Gbit. At least 2.5 Gbit but preferably more.
About the 10G. Am I wrong for banking on RAM (ARC) and ZiL on SATA or NVME SSD to improve my write speeds? I don’t need to reach 10G, but half would be nice.
To be pedantic here, only because it will be helpful for this thread, what is your definition and understanding of HA? Traditionally it’s via multiple systems in a cluster. In storage, you may be thinking redundancy. If you mean redundancy, you do want to reconsider that as losing a single disk too many means you’ve lost your whole pool and will have to start from scratch. ZFS is not a flat file system under the covers like BTRFS.
In regards to your last points, I have a simple mirror setup, not even RAIDz#. I have plenty fast upload/download speeds to my TrueNAS Core box and have 2.5GbE between them. I would honestly ignore the ZiL and SLOG stuff unless you REALLY know you need it and what you’re doing, as it does have potential pitfalls. If you’re coming from something like unRAID, you may be thinking it similar to the cache pool which helps speed up file transfers. This is not the case with ZFS, as it spins ALL the disks in the vdev where BTRFS will always be limited to one disk’s speed as it consumes files from each disk individually.
As far as the last half regarding the calculator, it has been a bit since I’ve looked up performance differences. I’ll have to dig a little myself to give you a better answer. Hopefully someone with better ZFS understanding can chime in if not.
Cool cheers. About the HA, I am running a RAID6 so I can lose 2 disks but I’ve never really lost a disk. I replaced one when it was failing but thats about it.
I am still interested in striping a RAIDZ, effectively creating a RAID50. This seems (on paper at least) a good compromise between RAIDZ2 and striped mirrors? I dont see many people doing it online though so maybe I am missing something.
I run a 4 disk striped NVME mirror on one of my production machines which is great. Just wondering how things would work with 8 disks and how I would set it up. 4 vdevs of 2 disks or 2 vdevs of 4 disks etc and the practical difference between the 2.
