Running through initial configs and starting to poke around this morning. My HL15 is a pre-built and running the Rocky Linux install provided.
I’m noticing a couple oddities, mainly around dnf update showing errors regarding best candidate, likely due to EPEL. It also appears that the Updates tab from the User Guide is missing from Houston UI. My options to continue are: --allowerasing--skip-broken--nobest
Looking through the packages, it seems a fair amount of the 45Drives dependencies for Houston UI are in the EPEL and 45Drives repo. Apologies, as an RPM based distro is not my usual forte as I spent most of my time in the realm of deb, but am using this as an excuse to learn!
Additionally, after installing the AOC-SLG3-2M2 Dual NVME card from the shop, it doesn’t appear to be detected. I’ve checked through lspci and lsblk. I’ve also looked through /dev/nvm* and am not seeing my drives in there. The NVME drives are known working. I’ll take a look through Supermicro’s page for eligible firmware and bios updates, as well as try another known working PCIE card in that slot and that PCIE card in another slot to rule out hardware issues.
I will add the output of dnf update in a reply below, as I cannot find an option to allow txt or md files to be uploaded. This output remains the same even after a dnf clean all
`Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, kpatch, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade
DNF version: 4.7.0
cachedir: /var/cache/dnf
User-Agent: constructed: ‘libdnf (Rocky Linux 8.8; generic; Linux.x86_64)’
repo: using cache for: 45drives_stable
45drives_stable: using metadata from Sun 29 Oct 2023 12:51:27 PM EDT.
repo: using cache for: appstream
appstream: using metadata from Thu 02 Nov 2023 03:26:33 PM EDT.
repo: using cache for: baseos
baseos: using metadata from Thu 02 Nov 2023 03:26:39 PM EDT.
repo: using cache for: extras
extras: using metadata from Thu 02 Nov 2023 02:48:16 PM EDT.
repo: using cache for: powertools
powertools: using metadata from Thu 02 Nov 2023 03:25:59 PM EDT.
repo: using cache for: epel
epel: using metadata from Tue 07 Nov 2023 08:36:02 PM EST.
repo: using cache for: zfs
zfs: using metadata from Wed 27 Sep 2023 07:17:59 PM EDT.
Last metadata expiration check: -1 day, 20:33:48 ago on Wed 08 Nov 2023 12:44:48 PM EST.
→ Starting dependency resolution
→ Finished dependency resolution
Error:
Problem 1: samba-client-libs-4.17.5-3.el8_8.i686 has inferior architecture
package sssd-ad-2.8.2-3.el8_8.x86_64 requires samba-client-libs >= 4.17.5-3.el8_8, but none of the providers can be installed
cannot install both samba-client-libs-4.17.5-3.el8_8.x86_64 and samba-client-libs-4.17.5-3.el8.x86_64
cannot install the best update candidate for package sssd-ad-2.8.2-2.el8.x86_64
cannot install the best update candidate for package samba-client-libs-4.17.5-3.el8.x86_64
Problem 2: package sssd-ad-2.8.2-3.el8_8.x86_64 requires samba-client-libs >= 4.17.5-3.el8_8, but none of the providers can be installed
package samba-client-libs-4.17.5-3.el8_8.i686 requires samba-common = 4.17.5-3.el8_8, but none of the providers can be installed
package samba-client-libs-4.17.5-3.el8_8.x86_64 requires samba-common = 4.17.5-3.el8_8, but none of the providers can be installed
package sssd-2.8.2-3.el8_8.x86_64 requires sssd-ad = 2.8.2-3.el8_8, but none of the providers can be installed
cannot install both samba-common-4.17.5-3.el8_8.noarch and samba-common-4.17.5-3.el8.noarch
cannot install the best update candidate for package sssd-2.8.2-2.el8.x86_64
cannot install the best update candidate for package samba-common-4.17.5-3.el8.noarch
Problem 3: package sssd-ipa-2.8.2-3.el8_8.x86_64 requires samba-client-libs >= 4.17.5-3.el8_8, but none of the providers can be installed
package samba-client-libs-4.17.5-3.el8_8.i686 requires samba-common-libs = 4.17.5-3.el8_8, but none of the providers can be installed
package samba-client-libs-4.17.5-3.el8_8.x86_64 requires samba-common-libs = 4.17.5-3.el8_8, but none of the providers can be installed
package samba-common-libs-4.17.5-3.el8_8.i686 obsoletes samba-dc-libs < 4.17.5-3.el8_8 provided by samba-dc-libs-4.17.5-3.el8.x86_64
cannot install both samba-common-libs-4.17.5-3.el8_8.x86_64 and samba-common-libs-4.17.5-3.el8.x86_64
cannot install the best update candidate for package sssd-ipa-2.8.2-2.el8.x86_64
cannot install the best update candidate for package samba-dc-libs-4.17.5-3.el8.x86_64
cannot install the best update candidate for package samba-common-libs-4.17.5-3.el8.x86_64
Problem 4: package sssd-ad-2.8.2-3.el8_8.x86_64 requires samba-client-libs >= 4.17.5-3.el8_8, but none of the providers can be installed
package samba-client-libs-4.17.5-3.el8_8.i686 requires libwbclient = 4.17.5-3.el8_8, but none of the providers can be installed
package samba-client-libs-4.17.5-3.el8_8.x86_64 requires libwbclient = 4.17.5-3.el8_8, but none of the providers can be installed
problem with installed package sssd-ad-2.8.2-2.el8.x86_64
libwbclient-4.17.5-3.el8_8.i686 has inferior architecture
package sssd-ad-2.8.2-2.el8.x86_64 requires sssd-common = 2.8.2-2.el8, but none of the providers can be installed
cannot install both libwbclient-4.17.5-3.el8_8.x86_64 and libwbclient-4.17.5-3.el8.x86_64
package libsss_autofs-2.8.2-3.el8_8.x86_64 conflicts with sssd-common < 2.8.2-3.el8_8 provided by sssd-common-2.8.2-2.el8.x86_64
cannot install the best update candidate for package libwbclient-4.17.5-3.el8.x86_64
cannot install the best update candidate for package libsss_autofs-2.8.2-2.el8.x86_64
Problem 5: package samba-winbind-clients-4.17.5-3.el8.x86_64 requires samba-common = 4.17.5-3.el8, but none of the providers can be installed
cannot install both samba-common-4.17.5-3.el8_8.noarch and samba-common-4.17.5-3.el8.noarch
package samba-client-libs-4.17.5-3.el8_8.i686 requires samba-common = 4.17.5-3.el8_8, but none of the providers can be installed
package sssd-ipa-2.8.2-3.el8_8.x86_64 requires samba-client-libs >= 4.17.5-3.el8_8, but none of the providers can be installed
problem with installed package sssd-ipa-2.8.2-2.el8.x86_64
package sssd-ipa-2.8.2-2.el8.x86_64 requires libipa_hbac(x86-64) = 2.8.2-2.el8, but none of the providers can be installed
cannot install both libipa_hbac-2.8.2-3.el8_8.x86_64 and libipa_hbac-2.8.2-2.el8.x86_64
package samba-winbind-modules-4.17.5-3.el8.x86_64 requires samba-client-libs = 4.17.5-3.el8, but none of the providers can be installed
cannot install both samba-client-libs-4.17.5-3.el8_8.x86_64 and samba-client-libs-4.17.5-3.el8.x86_64
cannot install the best update candidate for package samba-winbind-modules-4.17.5-3.el8.x86_64
cannot install the best update candidate for package samba-winbind-clients-4.17.5-3.el8.x86_64
cannot install the best update candidate for package libipa_hbac-2.8.2-2.el8.x86_64
Problem 6: package samba-client-libs-4.17.5-3.el8.x86_64 requires samba-common = 4.17.5-3.el8, but none of the providers can be installed
cannot install both samba-common-4.17.5-3.el8_8.noarch and samba-common-4.17.5-3.el8.noarch
package samba-client-libs-4.17.5-3.el8_8.i686 requires samba-common = 4.17.5-3.el8_8, but none of the providers can be installed
package sssd-ipa-2.8.2-3.el8_8.x86_64 requires samba-client-libs >= 4.17.5-3.el8_8, but none of the providers can be installed
cannot install both samba-client-libs-4.17.5-3.el8_8.x86_64 and samba-client-libs-4.17.5-3.el8.x86_64
package samba-winbind-4.17.5-3.el8.x86_64 requires samba-client-libs = 4.17.5-3.el8, but none of the providers can be installed
package sssd-2.8.2-3.el8_8.x86_64 requires sssd-ipa = 2.8.2-3.el8_8, but none of the providers can be installed
problem with installed package sssd-2.8.2-2.el8.x86_64
package sssd-2.8.2-2.el8.x86_64 requires sssd-common = 2.8.2-2.el8, but none of the providers can be installed
package libsss_certmap-2.8.2-3.el8_8.x86_64 conflicts with sssd-common < 2.8.2-3.el8_8 provided by sssd-common-2.8.2-2.el8.x86_64
cannot install the best update candidate for package libsss_certmap-2.8.2-2.el8.x86_64
cannot install the best update candidate for package samba-winbind-4.17.5-3.el8.x86_64
(try to add ‘–allowerasing’ to command line to replace conflicting packages or ‘–skip-broken’ to skip uninstallable packages or ‘–nobest’ to use not only best candidate packages)
`
I’ve unfortunately not gotten much further with the AOC-SLG3-2M2 card. It is still not showing up and does not appear on the compatibility list from Supermicro either look at the card’s page or the board’s page.
I’m able to update the BMC/IPMI firmware without issue, but the BIOS update is asking me for a license key.
I was able to get dnf to update via dnf update --nobest without any issue. Though the “Software Updates” tab is still not present in Houston UI.
Thanks daemon. I’ve tried this and had some issues figuring out which one is the correct PCIE bridge, as I’m presented with br1, 2, and 3. Performing any changes to br2 causes a boot driver issue for Intel 40GbE NIC, which is strange as I have no extra network cards installed and am just using the onboard 10GbE SFP+ modules.
Slot 2 appears to only be an x4 slot, though slot 3 did present one of the NVME drives. Originally, I was trying Slot 5 and have tried 4x4x4x4, 4x4x8, and 8x4x4 on br3. I’m going to grab a generic Connect-x3 10GbE card and toss that in slot 5 to see if I can at least get something to populate on it, as well as try Slot 3 again.
I’ve stumbled across this post which hopefully gives us the info we need.
I think slot 3 is the good choice, try it with bri1 set to x4x4x4x4. I think bri2 shares lanes with the X577 chipset and that is why you saw intel NIC errors.
I just gave that a go, I was getting clean updates though after running with --nobest once. I’ll have to dig later, but I don’t think realmd requires that package specifically for AD join. I haven’t noticed any issues so far, but I’ll dig a bit more into dnf later. The last RHEL based systems I worked in were CentOS 6 IIRC and we were still using yum.
@daemon1001 I did manage to get the NVME carrier running in slot 3. After a bit of testing and troubleshooting last night, it seems Br2 was the right choice (IOU1) it just needed x4x4x8, as choosing x4x4x4x4 causes the error to pop up again Re: the Intel NIC drivers.
In case anyone else with this board and the 2x NVME carrier from 45HL, I did the following:
Installed NVME card in PCIe Slot 3 (CPU Slot 3 is silk screened on the Motherboard.)
Boot into BIOS hitting Del at the SuperMicro splash screen.
In BIOS select Advanced > Chipset configuration > North Bridge > IIO Configuration > CPU Configuration > IOU1 (IIO PCIe Br2).
@orix
Well the x4x4x8 makes sense if it is sharing PCIe lanes with the onboard nic. A 10Gb nic needs 8 PCIe lanes. Glad it’s working. Mine is in production, hopefully be here soon.
Does that mean a gpu can still be used in slot 1 and then the nvme carrier card in slot 3? If that’s the case I may need to pick up one of those cards.
As there are two NVME Riser cards that can be purchased, supermicro has the guides here:
AOC-SLG3-2M2
AOC-SHG3-4M2P
On either product pages the motherboard for the HL-15 are not listed within “Validated Platforms”.
I did notice there are Additional Settings listed for the two cards. @orix - I don’t know if you saw this, but I will post the text here
This was copy/paste from page 3-5 from the 2-slot card
Depending on the system, motherboard, and BIOS version, the following BIOS
settings may be necessary for the proper operation of M.2 NVMe drives:
• Having the CPU IOU settings set to x4x4x4x4 PCIe bifurcation. This option may
be found under BIOS Setup -> Advanced -> Chipset Configuration -> North
Bridge -> IIO Configuration -> CPU Configuration -> IOU Setting -> x4x4x4x4.
• Having the NVMe Firmware Source set to AMI Native Support. This option may
be found under BIOS Setup -> Advanced -> PCIe/PCI/PnP Configuration ->
NVMe Firmware Source -> AMI Native Support.
Refer to the applicable system or motherboard user's manual.
The 4-slot seems to have similar settings recommendation. I am not in any position to work this week. I wanted to get other items configured before I enable the carrier card.
Thanks, I just edited the link using the forum tools, which seems to have fixed it!
I found similar regarding the x4x4x4x4, however that caused Intel NIC pre-boot driver errors. I changed it to x4x4x8 and it has been stable with no return of those Intel errors through a few reboots. For the 4x NVME card though, I would imagine you’ll need the x4x4x4x4.
Oh, and the Native AMI did NOT work for me I should mention. I reverted back to the Vendor firmware source and all has been fine.
