As others have done, I replaced the stock fans with Noctua NF-P12 redux 1700’s. Based on what others here had experienced I was expecting to have the ‘throttle lower’ issue and was prepared with the appropriate impitool command; which worked great! Shortly after that however, I noticed that my drive temps were getting higher than expected. Digging in, I found that I have an issue where the BMC only sets the Noctua fans to 2 speeds: low (~300RPM) and full (1700RPM).
I’ve changed the FAN Mode(s) and Unit Reset the BMC multiple times but, other than Full Speed, the only fan it changes is the CPU fan (SM active cooler). Both the Sensor Readings in the BMC and ‘ipmitool sdr | grep FAN’ confirm that the Noctua’s only run at ~300RPM. The one caveat to this is when the System/BMC boots: the fans spin at ~900RPM, then slow to ~300RPM again.
Using ipmitool, I AM able to correctly set them using raw commands - e.g. I set zones 0 and 1 to 50% and they’re running at ~900RPM. This of course doesn’t survive a reboot, Fan Mode change or BMC reset so it’s not ideal.
I read about someone else experiencing this on another forum but they weren’t the same fans. His hypothesis was that the BMC was setting the fans to an initial speed of ~300RPM, too slow for his fans, and trying to ramp up from there. Indeed, this is slower than the Min rotation speed (~450RPM) of the redux 1700.
Are other people running these fans experiencing anything like this? Did you need to do anything to get them to run at >300RPM? I’m hoping I just missed something simple.
Testing further: it seems that if I stress the CPU all the fans DO increase speed to compensate for the temperature rise. Unfortunately, when the temp falls back down to normal, any manual speed settings are lost (spins back down to ~300RPM).
Would someone else running these fans, that has access to ipmitool or the like, be willing to post the output of:
ipmitool sensor list all | grep FAN
I’m interested in seeing the RPMs in either Standard or Optimal Fan Mode(s). I’m wondering if the much smaller (and higher RPM) CPU fan is confusing things.
So my MB has 7 fan headers: FAN1-5 and FANA, B. Zone 0 is Fans 1-5 (run by the CPU temp) and then the “peripheral” fans A and B - which adhere to different thresholds based on the ‘profile’ set in the BMC.
My first configuration was the CPU fan on FAN4 and the Noctua’s on FAN’s 1-3 and 5 plus A and B. I tried moving things around in multiple different configs but I always ended up with the same issue: Noctua’s set at ~300RPM and the SM CPU cooler was at 2500-4100RPM. This did NOT vary whether set at “optimal” or “standard” modes in the BMC.
Admittedly I did NOT try things without the CPU cooler involved. Nonetheless, the NF-P12’s were, when controlled by the BMC, set at an ineffective speed and drive temps rose to unacceptable levels (>40 degrees without taking action).
Since I seem to be the only one experiencing this ‘phenomenon’, I broke down and purchased 2x3-way PWM splitter cables and am now running the 6 chassis fans off the A and B zone//headers with the BMC set to “Heavy IO”. This is keeping things at a baseline of ~2500RPM (CPU) and ~900RPM (peripheral/Noctua). If the system workload rises, the fans spin up, then return to this baseline upon completion. Temps are where I’d expect them to be now.
While I wish I had a better understanding as to why I 'm “different”, this setup is working for me - for now.
My 2¢ is I don’t think you’re different. None of the temperature sensors on a typical motherboard are representative of the drive temperatures, so if you’re not going to/able to control the case fans from their temperature reporting then you need some other proxy. “Heavy IO” would seem like a reasonable proxy.
Although I have not done it, I believe from within linux there are ways to install and configure software that will control the fans based on drive temps as input.
Unless you’re hammering away on spinning rust 24x7, you should be able to keep the drives cool with RPMs around 1500-1800 RPM (and provide sufficient airflow for any passively cooled NIC/HBA). A nice thing about the 45drives pods vs denser designs is the large gaps around the drives allow for quite a lot of air to move by the drives. HDDs are going to be giving off a baseline of heat just from spinning even if they are not reading/writing. The only way to lower that would be to spin down the drives after a period of inactivity. I don’t know what your acoustic requirements are.
I assume you don’t have mesh covering the fan intakes. What is the ambient temperature where the server is?.
I’m personally using this script to manage fan speed on supermicro boards. It works great on my H12, it should be fine on 11, gen too which I believe is what the HL15 ships with.
You can customize it to use whatever probe you want on the motherboard.
Thanks for the reply, I appreciate it. You’re probably right. To clarify, why I think I’m different is that when the fan mode is set to Standard or Optimal in the BMC, the Noctua’s never run above 300RPM when idle. From what I understand, that makes sense for Optimal, but Standard should have them running at 50%-60%. I’m just not seeing that.
Thant’s why I was kinda hoping someone else running these fans would post what RPMs they’re seeing in those modes.
You’re absolutely right and at those RPM’s they do. But with the BMC only running them at 300RPM, unless the CPU temps go up, they don’t. Admittedly, the reason I did the fan swap was to cut down on the noise from the stock fans. As such, I’d rather not run the new fans in the “Full Speed” mode.
The good news is that Heavy IO seems to be landing me in a good place: CPU fan/zone runs at ~%30 and the peripheral fans/zone running at ~50%.
Correct, no mesh or coverings/shrouds. The ambient temp is generally ~68F/20C.
Thanks for sharing this! I had actually started toying around with a bash script to do the same with ipmitool’s raw commands. Yours is obviously a lot more elegant.
I got sidetracked on other stuff so haven’t played with it myself yet, but forum member @pcHome pointed me to linux packages hddtemp and lm-sensors. There are various scripts around that use these to get the probe data to control the fans. if you google something like “linux control fan speed from hdd temperature”, for example;
The items I recommend to @DigitalGarden were variations of other projects individuals completed to have more detail detected within the Proxmox web UI.
I believe these would work for what you want to monitor the hd and fans.