Anyone got a picture of how/where to mount the official fan shroud for the HL15 2.0? The one for cooling the PCIe slots with all PCIe slots in use?
Only place I can think of is on the back side of the internal fan wall (facing the PCIe slots) and use the fan mounting holes but then it will be in conflict with full height and full length x16 cards.
I was hoping to use the fan shroud to add air flow over the motherboard m.2 slots which is placed under the PCIe slots. But I guess I’ll have to find another way to cool the HL15.
Does the cantilever fan bracket offer a better choice? Little room to move air around so close to the lid I guess? Where does even that one install? Will an exhaust fan over the IO shield help in any way for that fan so to not just sirculate hot air inside the HL15?
The cantilever fan attaches through the back grille;
https://forum.45homelab.com/t/hba-fan-bracket/511
45Drives calims to use it on their enterprise gear, but I have the same doubts as you as to actual effectiveness.
I think the Fan Shroud is supposed to attach to the back of the 120mm fan on the fan wall that blows over the PCI slots. How well that lines up with PCIE cards that are taller than full height, or long, I don’t know. I think these parts were originally designed to cool hot HBAs and NICs, not hideously oversized GPUs.
The fan shroud is available for 3D printing if you think you could modify it to fit your PCIE setup;
https://forum.45homelab.com/t/fan-shroud-design/2428
It is possible with zip ties to mount two 80mm fans over the IO shield, whether that would help with your cooling issues, I’m not sure;
https://forum.45homelab.com/t/6bay-ssd-bracket-kit/3512/4
You might need to adjust your fan profile to have higher RPMs on the front and mid fans serving the PCIE slots, or add a 40mm fan and a duct to push air umder the GPU or whatever and over the SSD(s).
1 Like
This is exactly my thought. These add-on’s are intended to help cool server parts that require a high rate of air flow over their heatsinks. The HL15 doesn’t have the high speed “sounds like a jet taking off” fans you typically see in enterprise servers that also produce the necessary airflow for these cards.
I tend to this the cantilever fan is the better choice. As @DigitalGarden mentioned, the fan shroud can be 3D printed if you have access to a 3D printer and want to try that first.
Where is the M.2 in relation to the case and fans? A picture or diagram could help here.
Thanks for the links and suggestions.
No GPU in this build, and the cards are mainly enterprise gear but I’ve got adapter cards for EDSFF E3.S drives that don’t fit with the fan shroud.
I already have the fan shroud. But no 3D printer to test other variants. Moving the cards around didn’t help either, limited options too.
Yes the fan profiles are at max already. The problem is mainly about directing air flow to the right places. Cooling seems like an afterthought in the HL15 2.0
It try this too:
I’ll just order the cantilever to test. I got the fan shroud already, but no way to get it to not interfere with the EDSFF E3.S adapter cards.
The M.2 NVMe are located on the motherboard under the PCIe 5.0 x8 slots as per the image in the link below. This creates two problems. Height constraints to not obstruct both the cards in the x8 and x16 slots and directing enough air flow to cool the drives as there’s no space for heatsink on them.
The motherboard is the H14SSL-NT as shown here: Sorry, as a new user I’m not allowed to post links or upload images. But you can find the picture of the motherboard by searching for the Supermicro H14SSL-NT hope that helps.
It’s all good. I was able to find the motherboard no problem. I was going to ask about heatsinks knowing that the other cards were probably the limiting the factor. How much performance degradation are you experiencing at this point due to M.2 thermal throttling? I imagine the lower M.2 slot is more problematic than the upper one. Trying to get a sense on how far off the mark your setup is with case fans already at 100%.
I have the noctua fans version of the HL15 for another setup in the first place but that was a mistake for this setup, so I might replace those. Though I think directing air flow is the main concern. Both over the PCIe cards, M.2 drives and out of the HL15.
I’m using PM9A3 drives and they throttle fast with this setup. Degradation I will say is 50% but speed is not my main problem (latency is), I’m more concerned about longevity.
Now, I must admit that they don’t throttle too often as the drives are OS drives only, but it is a big enough problem that I feel I need to fix it. Replacing dead drives caused by overheating with todays prices feels stupid.
Added two noctua 40mm fans over the io shield as exhaust, can’t say it did too much for the M.2 drives. But at least now I’m not just recycling stale hot air.
Tried with some cardboard cutouts as shroud for the PCIe and M.2 space, but not enough preassure from the Noctua fans as expected.
What I did instead was reducing the speed of the big Noctua fans and put one 40mm fan in front of each M.2. Speed didn’t do too much difference to the M.2 temperature.
And I’ll need to keep an eye on the PCIe cards temps and increase the 40mm fan speed to keep air flowing so it’s not just sitting stale in that area.
And a way to mount those 40mm fans so they stay put.
Ordered “Pure Copper Thin Heatsink Thermal Pad for M.2 NGFF 22110” from AliExpress, they’re 2mm thick and with grooves. And 0.5mm thick 120x20mm Minus Pad extreme 2 from Thermal Grizzly (hoping they’re good enough). Hoping there might be room for this on top of the drives (hard to tell from just looking at the spacing between all the installed hardware). It will take a few weeks to arrive, so I’ll update later.
Would love to have a shroud leading the hot air from the PCIe lanes out. Now it has to go sideways and compete with the CPU cooler exhaust to get pushed out by the two 40mm fans over the IO shield.
I have to admit I’m struggling with why your setup is so hot. I have the full build HL15 with ROMED8-2T motherboard with a quite a few cards installed - dual slot Intel Arc A750, 9400-16i HBA, Startech Quad M.2 card, and a Intel X550 Dual Port NIC. I also have two M.2 boot drives installed on the motherboard. None of the six M.2 drives have breached 50C and average 40C or less. I don’t have the Noctua fans, but its still that lower fan pair that’s doing all the cooling of these enterprise cards in my setup.
If you’re thermal throttling then that’s greater than 70C for your M.2 drives. Are you all your slots full of EDSFF E3.S adapter cards and drives? Is there something else about the setup that’s restricting or blocking airflow? How are your temps without the adapter cards?
2 Likes
Looking at your motherboard I don’t think your M.2 on the motherboard are blocked by most of your PCIe cards like they are on the H14SSL-NT? Which drives do you have and are you using thermal pad and/or a heatsink?
The PM9A3 is known for running hot if the net is to be believed, I might replace them if the heatsink doesn’t fix it. And just to confirm, I’m currently running them without thermal pads or any heatsink due to space restrictions. Also, I’m attributing the high temps to the stale air in that area. Adding a cantilever fan might help, if I can pull air away and not push air down onto the PCIe slots (if that’s even a good idea).
Something I’ve forgot to test, newer firmware on the PM9A3, I found GC7802Q. After reading all the changelogs I found that at some point they figured out a mismatch between SMART and SMB values for temperatures. Don’t know if that could cause a false trigger of throttling. I’m away from the system the next week I’ll find time to flash firmware after that. And a week later or so the heatsinks are in.
E3 drives have heatsinks and I’ve not had any trouble with them. Regarding before/after temperatures, I don’t have too much info since there was no errors. If I have to guess I think it’s mostly due to stale hot air in the area.
There’s 15 HDDs in the HL15, mostly Toshiba MG10 20TB and Seagate Exos X18 18TB and a few others. They have temperatures at 30-35 celsius but all that heat is blowing directly into the case and forwarded to my problem area.
I’m open to that it might be something wrong with either the m.2 or the temperature reporting causing them to throttle while not actually beeing too hot. So I’ve ordered some Micron 7450 Pro to test.
Solved
My initial questions about the shroud and cantilever fan are answered, thanks.
Got someone to replace one of the PM9A3 m.2 and can confirm something must have been wrong with them. Will replace the second one shortly. I would say things are normal now so should’ve just started with replacing drives instead of trying to add more cooling.
I still think there’s room for better cooling of enterprise PCIe hardware.
3 Likes