Received HL15 and having various issues

General
1

Note: I sent a similar message to info@45homelab.com, but hoped someone on the forums might have some suggestions also.

I received my HL15 and on the plus side, it had the 1000w PSU and the slot covers were painted red like the red of the unit and they aren’t break off anymore (can unscrew them and screw them in like most pc’s now which is good.).

I am having issues with the system.

  1. I’m getting an alarm from the unit after running it for about 20 minutes. The CPU (Xeon Silver 4210) is getting an overheating alarm and is at 90c according to the sensor readings from IPMI. The room isn’t particularly hot.

  2. After putting in five 24tb drives and installing TrueNas Scale onto the NVME drive, the installer told me to reboot and remove the USB drive that had the installer image. Upon doing so, the system rebooted but didn’t boot up from the NVME drive. Upon going into the bios, the boot options seemed to let me pick any of the 24TB drives but didn’t give an option for the NVME drive.

Any suggestions?

2

Made some progress I think. I had to go with Legacy/Grub bootup option. Couldn’t get it to boot up off NVME drive using UEFI. Not sure if I’ll have to manually go to boot loader every time I reboot and select TrueNas in grub or not. Would be nice to have it default to it at least.

As for the overheating, I removed the top and adjusted the fan speed to max which didn’t seem to push that much more air, but the CPU temp is currently at 41c. I’ll keep monitoring it and try with lid back on. I’m setting up my TrueNas datasets/etc. atm.

Update: I’ve been replicating from my other TrueNas system and the max temp has been 46c since adjusting fan speed which seems ok.

3

Not sure about the NVME. What make/model and in which slot? Was it new or had it been previously partitioned by some, possibly different, OS?

4

I believe it is a Kingston NV2 1TB M.2. It is in the only NVME slot on the Supermicro X11SPH-NCTPF and it’s device name is nvme0n1.

During the TrueNas install, it allows setting up boot using UEFI (for newer hardware with NVME drives) or using Bios (for Legacy Hardware). When I use the UEFI approach, it doesn’t find a boot device and seems like it is trying to boot across network devices? If I take the Bios approach, I can hit a key during startup to go to a boot menu (grub) and it will properly start up TrueNas. I’m assuming I have to hit a key during startup each time to have it go to TrueNas which isn’t great.

5

It sounds like the cpu or cooler has loosened or something else is not right. This can affect the use of pcie lanes and create the heat issue you are having. Have you attempted to reseat the cpu?

6

I haven’t yet. I just got a Noctua NH-U12S DX-3647 Premium CPU Cooler and Xeon Gold 6230R sitting around. I’m not sure the cooler will fit yet, but if it does, I’ll likely install the new processor, make sure it is seated well, and apply a proper amount of Noctua NT-H2 paste. The cooler is supposedly 158mm which is very close to the 160mm that was stated as the max size for the cooler but I’ve seen conflicting reports about it fitting anyway.

7

I have an arctic 4U-M. It is 156mm. You will be close but should be OK.

8

Ok. I don’t have an X11SPH, but there’s usually a BIOS setting for legacy vs UEFI boot, maybe that got changed. I do run Truenas on a few different systems and don’t remember encountering that message.

9

I suspect there is a BIOS setting or two that needs changed. Do you have a spare SSD lying around that you can try installing TrueNAS Scale onto in UEFI mode? That would help rule out if it’s really just NVME or if all devices have problems booting via UEFI.

I think this is pretty common these days in modern UEFI BIOS where the BIOS checks drives to see if it thinks it can boot a drive and only displays those options. Not many but I’ve had a few instances where a borked or atypical install of an OS didn’t show up in the BIOS and confused the heck out of me. I think I prefer the older ways of just listing me the devices and trying whatever I tell it to. :slight_smile:

10

In particular, check the following under “PCIe/PCI/PnP Configuration”:

  • NVMe Firmware Source
  • Onboard NVMe1 ROM
  • Onboard NVMe2 ROM

Probably worthwhile to also check on Secure Boot settings in the BIOS while you’re in there.

More info found in the motherboard manual - MNL-1949.pdf

11

I did find a setting that would dual boot and boot up with UEFI if it failed. I think if I’d played with the boot settings before giving up and using grub, things would have worked out better. With the change to the bios, it is booting into TrueNas without my intervention.

So where things are now:

  1. Boots fine into TrueNas

  2. I have the ARC 310 video card doing transcoding in Plex for me like a champ (transcoding without it was choppy).

  3. I have 2 SFP+ connections to my switch and the LACP works great.

  4. I updated the fan profile to be max speed on fans at 1900RPM with the Noctua’s and the noise level isn’t that bad. The temps have been great. I haven’t seen above 45c even when pushing the server and while idle it is 30-34c range. I probably could back off a bit on cooling to lower the noise a bit.

  5. So everything is working great now and I have the Noctua NH-U12S DX-3647 and Xeon Gold 6230R that I could upgrade to. I’m a bit chicken to jump into that since everything works so well, but I’ll probably dive into that soon.

Kinda loving this machine right now.

2 Likes