I’m looking for a fully built rack mounted homelab to replace my current setup, and the HL15 seems to be, I think, the perfect match.
I’m currently running many containers out of portainer (nextcloud, immich, jellyfin, *arrs, jellyseer, vikunga, gitea, authentik, traefik, outline, memos, home assistant, homepage, and many others) on a 4 disk QNAP NAS and want to upgrade to something more powerful and future proof. It’s for home use for me and my family. I would then use the QNAP for backups.
Hardware is not my area of expertise and that is why I would like something pre-built. My understanding is that ideally you would want:
A hypervisor with e.g. proxmox for the VMs
A NAS? for data storage with ZFS
Would an HL15 be sufficient to act as both or are 2 servers still recommended? If sufficient, initially I saw feedback that the HL-15 didn’t have enough threads (Xeon Bronze 3204) for proxmox, but now with the Xeon Silver 4216 offered, would that be sufficient? Also how much RAM would be needed. My understanding is ZFS is RAM heavy.
Also how complicated from a software point of view would this be? I can setup containers with docker compose in portainer, but this would be new to me. Would this be Proxmox as the main OS and then something like TrueNas for the data part? Is Houston more of a management interface, or a replacement for TrueNas?
To answer your last question first, my understanding is Houston is more geared towards the storage (ZFS) aspects of a NAS and it’s modules for support of VMs and containers are only beginning to be introduced. You would likely be looking at Proxmox and/or Truenas OS; since you have expressed more concern over all the container apps than data storage and redundancy, probably Proxmox. Based on your own assessment of your level, I might suggest shying away from a combined Proxmox/Truenas solution and just choosing one or the other. Adding that layer of complexity is going to require some level of geek and not be for someone who mainly wants something to work.
What QNAP do you have (model, CPU, RAM, amount of data, RAID level) and what problem(s) are you having with it that you are hoping to solve with the HL-15? Even the Xeon 3204 may be equivalent to what is in your QNAP depending on how high-end it is. Are you running into high CPU utilization on the QNAP? If you aren’t looking to add huge amounts of space and can wait a few weeks or a month or two, maybe the new HL8 would be a better fit for you. I didn’t see Plex on your list, but one downside of the Xeons for some here is that they don’t have the Intel QuickSync feature that aids transcoding.
You shouldn’t need two servers.
More RAM will always help ZFS, but it really depends on your workload of reading and writing. How many people will access the system concurrently? How much data do you have and how much is it changing? It sounds like you are mainly consuming media and so your RAM requirements shouldn’t be that high.
If you go with Truenas, it has preconfigured installable apps for nextcloud, immich, jellyfin, *arrs, jellyseer, gitea, home assistant, and homepage. You would probably load the others as custom docker images unless they are QNAP specific, in which case you would need to find an alternative.
Let’s start with the hardware side of your question. I think the more you look at self hosting, the more two different servers will make sense. At the end of the day, there’s only so much you pump into one system before something becomes a bottleneck. However, you seem to be doing fine your current QNAP setup. I have a hard time imaging the HL15 won’t be able to do better out of the box.
Looking then at hyper-converge setup running everything on an HL15, I think the CPU upgrade would serve you well. The 32 threads of a 4216 will definitely net you more but also takes you to the other end of the spectrum. What do you have now on your QNAP? A 4215 with 16 threads might be a considerable upgrade and could save you a little bit of money - maybe to put towards extra RAM.
Speaking of RAM, ZFS will almost always be more performant with more RAM. That’s different than being “RAM heavy” and definitely doesn’t mean you’ll have a negative experience with less RAM. It really comes down to your individual workloads, the number of individual workloads running at the same time, and if those workloads will fit into the ZFS Arc which is what’s using the RAM. The other nice thing about ZFS is you have several options to get a level of performance you’ll find satisfactory by using special VDEV’s. You can always upgrade RAM later as well. Unless you max it out from 45Drives, your HL15 should have some open RAM slots.
Anyway, as a starting point for RAM, you can figure 1GB of RAM for every 1TB of disk space in your ZFS pool and then add onto that amount the RAM needed for your containers and VM’s.
In have a QNAP TS-453D. It has an Intel Celeron J4125 4-core/4-thread processor and I have 20 GB (4 + 16 GB) RAM and 4 disks running raid 5. The reason for the upgrade is because it is starting to lag behind, some services are getting slow, I often have difficulty connecting and I have no backup system in place. Storage is also running low.
It started as a plex media server (I then moved to jellyfin) and that is why no backups, was not really worried about losing any of the media. But since then it has grown and now I have a lot of data too (images with immich, wiki with outline, and other data in nextcloud), backups are important so also need more storage.
It also keeps growing, I’m running more services, moving more data from other sources, and at some point would like to also run maybe run an LLM (maybe not on this machine)
Good point about Intel quicksync. I’m guessing there is enough room to install a graphics card, since my understanding is it would be needed to transcode media.
An HL8 or HL15 would be good upgrades that should be able to do everything you need. A lot depends on how much money you have to spend and how much of a leap you are looking to take and headroom to have for future-proofing. Although I have two HL15s (custom build) and an old Q30 (from the enterprise side), with the HL8 coming out I, personally, have a bit of a hard time justifying the relative price/performance of the fully built HL15 unless you really need the additional drive bays, PCI slots, greater than 64GB RAM or other server features of the Supermicro motherboard. (Which you will if you are going to install GPUs and run LLMs on it).
So, given your concern is lag, you probably would want at least the 4210 upgrade if you go with the HL15. Where will this live, though? In an office/living space or a conditioned closet? If the HL15 isn’t going to be stashed away in a sound damped area, you will also want the Noctua fan upgrade. So, all in you are now at ($2099 + $759 + $150+ + $231 =) $3239 for an HL15 with a Xeon 4216 and at least 64 GB RAM. Quite a sum compared to what seems to be ($1399 + $150 =) $1549 for what is in many ways a similarly powerul HL8. Of course, there may be other reasons as mentioned above to get an HL15, but are they $1690 worth of reasons? They may be for you, but for me it’s definitely a consideration.
You can install many GPUs but not the longest 3-fan ones, the max length is around 216mm including some space for the power connectors if they come out the back. You don’t need a GPU or QS to transcode, but it takes a lot of load off the CPU if you are doing other multitasking at the same time and/or watching multiple streams.
Yes, that setup would work. Note that that video is 2.5 years old now and the features of these OSs change over time. Also note, and @rymandle05 can speak to this more than me because I use TrueNAS mostly, there seems to be quite a delay in rolling out Rocky Linux upgrades for Houston and the HL15, so I’m not sure how that impacts what version of Proxmox you can run hyper-converged. I could be entirely wrong, but if Houston requires Rocky 8 and can’t easily be upgraded to Rocky 9 that may also peg you to a release of Proxmox from a few years ago? This might be another reason to have separate storage and compute machines if those are your desired OSs, at least for the way he has the two platforms running together in that video.
Specifically on Proxmox and Houston, I was able to successfully setup the repo’s from 45Drives and install their cockpit modules on Proxmox 8.2 which is still latest the major/minor release. I just had one extra step to get it to work.
It’s just needs to be known that it’s not officially supported on Proxmox or Debian Bookworm. I’m still hoping that will change with the next version of Houston that 45Drives has been actively developing.
Also, Proxmox w/ Houston is not my daily driver on my HL15 so I can’t say my validations were super robust or all encompassing. My HL15 is on TrueNAS Scale Dragonfish. I do have Rocky 8 and Houston on the original NVME drive in case I ever wanted to check it out again.
I agree with @DigitalGarden on considering the HL8. It’ll be an upgrade at less cost and still supports 45HomeLab. The HL8 won’t be rack mount like you wanted but should still be a solid chassis for upgrades in the future. It also won’t have some of the enterprise features like IPMI and will be even more limited in space for a GPU.
All that said, I can’t deny that, if you have the money, the HL15 is a nice system. It all comes down to tradeoffs.
Oh, I missed that in the original post, sorry. I assumed coming from a 4-bay NAS appliance we weren’t wanting rack mount and were going to use the HL15 as a tower. Sure @nfdfgd, if you own or are buying a rack and have considered where you’re going to put it re noise, temperature, etc, then yeah buy an HL15. I’d start with at least the 4210 and 64GB of RAM, but whether you need the 4216 and how much more RAM you need really depends on the specifics of your workloads and if you hyperconverge. Without knowing that, what I’d probably say is if you decide to do storage and compute on the same box then get the highest spec CPU and amount of RAM you can afford. Remember in that scenario the RAM needs to support both ZFS cache and all the running containers and/or VMs. If you decide to do them on separate boxes, then the 3204 is sufficient for a NAS, and spend the money on the additional cores and threads for the CPU in the compute box.
Look at this! 45Drives just dropped a video on HoustonUI module updates and support for Rocky 9 and Ubuntu 22.04 coming in October.
@nfdfgd With these updates, you may want to consider using HoustonUI which ships with the HL15. The Cockpit modules from 45Drives combined with the standard ones would cover everything we talked about here: Disk Management, NAS Sharing, Virtual Machines via KVM, and containers via Podman.
@DigitalGarden and @rymandle05 , thank you both! I have a lot to think about
One idea floating in my head is using the HL15 temporarily as hyperconverge, with TrueNas and portainer on it. Once I have a hypervisor, move the compute part over and hl15 would be just data.
While money isn’t a huge issue, still don’t want to throw it away so need to think well about this.
