Quick question for @Vikram-45HomeLabs or @Ashley-45Drives, I think I’m going to get #3 out of the original orders on Tuesday after the holidays (this is awesome, thanks so much for the communications and patience throughout the order process).
If I ordered another two barebones HL15s what’s the approximate lead time at this juncture?
Posting here just so if anyone else from the community is thinking “How long might it take?”, they can get some insight into how stacked up 45Drives is with orders at this moment in time!
When Springer decides it’s no longer 45Homelabs HL15, but instead 4 to 5 HL15’s I think we all start to get curious! So, whatcha running, if you don’t mind sharing.
Three are destined to be my ‘online’ storage - 3x (15x20TB SATA; 4-6x 6.4TB NVMe, have 4 today and may expand). Works out to ~650TB usable in SATA and ~50TB usable in NVMe and is replacing a server with a HP 60-bay JBOD today which uses a mixture of 6TB, 10TB and 12TB SATA yielding closer to 400TB usable (and no NVMe arrays). I’m around 300TB used, so wanted something which would take me through 2024 - 2026 in capacity terms
If I buy two more:
Another would be my ‘backup’ storage and ZFS snapshots for a subset of the aforementioned.
One’s for mucking around, I’m going to do an Ampere One ARM64 build as soon as I can get an ATX-sized mainboard which’ll take one of those CPUs.
I’m super pleased with how the HL15s turned out. I haven’t had any issues with PCIe cards unlike some other members of the community. Great value for money, IMO.
If I pick up two more then I’ll end up with just 2x Supermicro 1019D-16C-RAN13TP+ and 5x HL15. Should result in me retiring a couple R630s, a couple DL360 Gen9, and a HP c3000 with a handful of BL460c Gen9 plus a HPE D6000 enclosure. Much simpler and more modern!
I took a bit of a break from all the forums as I got busy with a work project. What OS/Distro are you using for yours? I’ve bounced between TrueNAS and Rocky a few times and never feel quite at home with either unfortunately.
Man, I’m jealous! I really want to spec out an Ampere server so badly! Unless I’m looking at the wrong hardware, it seems like it’s still rather expensive!
I said in another comment but I made a change to add Lenovo SA120’s to each to get me to having 27 drives per building block and currently 81 drives total between the three HL15, that’s avoided me needing two more units soon so I didn’t order #4 and #5.
When I do need a few extra drives I’m hoping to ask 45Drives what a chassis-only price looks like for the 30-drive or 45-drive enterprise units. That’s preferable to me versus having shelves added on to the HL15 and so if I can get 30-drive for reasonable cost, I’ll likely do that whenever I’d like to go from 81 to ~141 drives (both the usage of the SA120 and the options to get to 141 within my existing rack mean I’m 100% set for the next years!).
I’ve still got a HPE Apollo 4520 which takes 2x23-drive (and it’s fine for just “storing bits”), it’s currently not populated or powered on. Everything above was to get after my goal of having converged Compute and Storage, inclusive of lots of NVME-based storage alongside GPUs and TPUs which is where the older equipment like the Apollo isn’t great! Probably this becomes “Snapshot Central” for backups of the converged system.
I’ve also still got a HPE c3000 enclosure with all the trimmings and a D6000 disk enclosure (mentioned previously) — I am trying to get rid of those but they’re too heavy to ship and nobody I know in PNW seems interested in taking them, so they will likely go to the electronics recycling place soon.
@geerlingguy did a review of what looked like an Ampere Quicksilver CPU in a HL15.
After that there’s Mystique (adds more cores) and Siryn, a.k.a. Ampere One, but unfortunately I haven’t come across anyone making a board which would fit a HL15 for the newest stuff.
Still want to build one of these and it’d be a better use of an HL15 for me because I don’t think I’d need >15 drives in the ARM system.
And yeah, cost is still pretty high on Ampere. Doesn’t seem likely there’ll be “IT recyclers” handling it anytime soon which is typically what drives the secondary markets.
I run Debian (testing) on basically everything. I’ve been using Debian for about 25 years and so my choices are not necessarily great advice for everyone to follow.
Hmmm, good to know! I’ve primarily been an Ubuntu or CentOS guy when it comes to Server distro. The latter only because it was the chosen and deployed distro for our customer-deployed platform, well before RedHat killed it.
Debian has lately caught my attention as I know Apt the best and really do not like snaps. Have you managed to get Cockpit/Houston running on it, or do you drive it primarily via CLI? If you’re using ZFS, how has your experience been in Debian?
I’m largely a CLI-driven person, I don’t run Cockpit or Houston, although for WebUI-based options it’s worth noting that TrueNAS Scale is built on Debian, as is Proxmox, so there are UI-based options should you want to go that route, and if you SSH in you’re still getting Debian to some degree.
How much you should customize Proxmox or TrueNAS is debatable, it’s often best to think of these as appliances, and a ton of customization is likely to cause you problems when doing upgrades.
ZFS has a pretty good Just Works ™ factor on Debian, in my experience, but I don’t tend to use any of the more edgy features like deduplication, native encryption, etc.
How much you should customize Proxmox or TrueNAS is debatable, it’s often best to think of these as appliances, and a ton of customization is likely to cause you problems when doing upgrades.
This is really my biggest concern with those two. While it’s Debian under the hood, making changes via CLI is often met with disapproval and in some cases disaster. I prefer to be CLI as often as I can as well, which is what drew me to Houston as it had just the right amount of UI for a quick glance at health and what not. I could stop being lazy and finish my CheckMK, Grafana, and Graylog project that I started, but boy did I bite off a bit too much.
Also, Jeff is a wizard and may very well have an addiction to ARM. If you haven’t watched them yet, his recent PCIE shenanigans on the RPI 5 are worth a watch.
You could pretty trivially do something like Proxmox VE on the physical hardware, a PCIe passthrough of the HBA into a TrueNAS virtual machine, and then create another Debian VM onto which to run all your CLI-driven customizations without violating the it’s an appliance, leave it alone goal state.
I do a decent amount of virtualization atop my HL15s, and the PCIe passthrough is fantastic in 2024, I’m specifically using that to push SR-IOV VFs from either a Mellanox ConnectX-5 100GbE NIC or a Intel E810 100GbE NIC into the virtual machines, so the hypervisor doesn’t slow down networking at all (which even if you’re using VirtIO and then a bridged interface will happen a little at high speeds).
Coming from ESXi, Proxmox has never really clicked for me. The virtual networking is frustrating and I had issues with linked clones eating up my boot pool for some reason.
I’m glad it exists, and I’ll likely give it another shot. But my VMUG is still paid for and lets me spend less time learning on the virtualization side and more time learning other things like Ansible and Terraform.
I have plenty of compute nodes, so really no want/need for another hypervisor handling my storage. Which is why the HL15 full build was perfect for me. It’s essentially a big smart JBOD.
so I didn’t order 
