I received my HL15 2.0 this week, and I’m unable to get backplane drive slots 9 through 12 to work consistently, and for 13 through 15 to work at all. Slots 1 through 8 work fine.
With all 15 slots filled, only the first 12 are recognized by the OS (checked with lsblk).
With the first 9 or 10 filled, only the first 8 are recognized by the OS. I have swapped (with power off) a drive in the 1-8 with the 9th slot a few times to ensure it’s not the drive that’s the issue.
In the course of testing I also tried hot-swapping and noticed that slots 13-15 never powered up, and that 9-12 would power up for a minute or so and then power down randomly. Adding another drive during this time could also trigger a previously working slot to stop working. Reseating a drive after this state could cause it to work again temporarily.
Other details, in case they’re relevant:
- PCIE2 slot (x16): the included
LSI 9400-16i
- PCIE3 slot (x4x4x4x4):
AOC-SLG3-2M2 (with both M.2 slots filled)
- PCIE4 slot (x4x4x4x4):
AOC-SLG3-2M2 (with both M.2 slots filled)
- No other PCIE slots used
- No hardware other than what came in the box from 45Drives
I don’t see any obviously loose cables, and none that I’ve tried pushing in had any give (power supply, HBA card). I thought I’d post here to get ideas on more surgical debugging instead of taking everything apart right away.
If you’ve checked for visible loose data and power cables, that is the first step. What you can’t check as easily is the connections under the backplane that aren’t visible.
The options are basically; bad backplane, bad cable(s) and/or bad ports on the HBA. You shouldn’t have to “take everything apart” to narrow it down, just strategically plug and unplug cables at the HBA.based on the known good components.
You could swap the cables into the HBA and see if the dead slots remain as is or follow the change. Ie, if cable A is connected to slots 1-4 (working) and cable D is connected to slots 13-15 (dead), flip cables D and A where they connect into the HBA. Are slots 13-15 still dead, or are slots 1-4 now dead? This will help narrow down if it’s a backplane or cable issue.
I’d send a note to info@45homelab.com. If everything still seems plugged in ok and your issue remains with ports 9-15, I think they have a way to test for a bad backplane.
Thanks for the ideas.
I reversed the cable insertions on the HBA from A-B-C-D to D-C-B-A and the drive slots that were working before continued to work and those that weren’t still didn’t. I’ll report back tomorrow after I’ve had a chance to take things apart to fiddle with the connections on the backplane.
That all seems to point to either a problem with the backplane, or the cables for slots 9-15 loose or unplugged under the backplane then. Because you say slots 9-12 are iffy, I’d suspect an electrical issue with the backplane. If so, someone at the support email will get you a replacement.
2 Likes
Hi @CantDriveFortyFive
Kindly reach out to me at info@45homelab.com, and i will make sure someone from our team reaches out to you and help get this fixed as soon as possible.
1 Like
I spent my morning testing various configurations to narrow down the issue. Below are my results.
Based on running these trials, I think that BP2 (at minimum) has problems with its data cable or power connection causing it to be unstable. While all drive slots worked at the conclusion of testing, I’m concerned that the process of putting it back together or inserting/removing drives will again cause it to stop working again.
Given that I haven’t found a real solution yet, I’ll send an e-mail as suggested.
Terminology:
- BP0-3 refer to the groups of slots on the backplane (0 right-most, 3 left-most)
- HBA card ports are listed top-to-bottom (C0 through C3)
Trial 0 (Stock baseline)
Configuration:
- Backplane: A-B-C-D
- HBA card: A-B-C-D
Behavior:
- Slots 1-8 (BP0+1) worked consistently
- Slots 9-12 (BP2) worked sporadically unless all slots in BP2 filled
- Slots 13-15 (BP3) didn’t work at all
Trial 1 (HBA port test)
Setup:
- Reversed order of data cable connections on the HBA card side.
Configuration:
- Backplane: A-B-C-D
- HBA card: D-C-B-A
Behavior:
Conclusion:
- HBA card connectors are not the issue
Trial 2 (BP data cable test #1)
Setup:
- Reversed order of data cable connections on the backplane side.
Purpose:
- Test whether the data cables are the issue
Configuration:
- Backplane: D-C-B-A
- HBA card: D-C-B-A
Behavior:
- Slots 1-8 (BP0 and BP1) worked
- Slots 9-12 (BP2) stopped working at all
- Slots 13-15 (BP3) worked for the first time
Conclusion:
- I conducted this test twice, cold-swapping drives in slots 10-12 with 13-15 just to ensure individual drives were not causing a problem. The behavior stayed the same.
- Data cables are not necessarily the issue: C+D cables (formerly connected to BP3+2) were now working fine with BP1+0. It could have been a problem with cable bends that disappeared due to all the shuffling in the case. However, C+D are shorter than A+B, and strain to reach the HBA card in this configuration. Given this, I would assume C+D would be more likely to have problems than A or B.
Trial 3 (BP data cable test #2)
Setup:
- Swapped B and A data cable connections on the backplane.
Purpose:
- Test whether cable B now had a problem
Configuration:
- Backplane: D-C-A-B
- HBA card: D-C-B-A
Behavior:
- Slots 1-8 (BP0 and BP1) worked
- Slots 9-12 (BP2) drive lights lit up late in the boot process or once in the OS; initially they were off. Confirmed working once OS booted with
lsblk.
- Slots 13-15 (BP3) worked
Conclusion:
- This surprising result (i.e. that cable B is working now) leads me to believe one of the following:
- BP3 and BP2 themselves have a problem to cause the inconsistency
- The data cables are damaged but work sometimes due to being physically re-arranged
- The power cables are damaged but work sometimes due to being physically re-arranged
Trial 4 (BP data cable test #3)
Setup:
- Swapped A and B data cable connections on the backplane, returning to trial #2 configuration.
Configuration:
- Backplane: D-C-B-A
- HBA card: D-C-B-A
Behavior:
Conclusion:
- Having the same behavior as trial #2 makes a problem with the cable seem unlikely. More likely to me is a problem with the data cable or power connection on BP2 that is being jostled between trials to cause slots to work and not work.
Trial 5 (BP data cable test #4)
Setup:
- Continually swap A and B data cables and their drives around between to trigger flaky issues.
Configuration:
- Backplane: D-C-B-A and D-C-A-B alternating
- HBA card: D-C-B-A
Behavior:
- All drive slots have worked over several swaps
1 Like
Hey @CantDriveFortyFive
Like Vikram said, if you reach out to info@45homelab.com we can help troubleshoot and get to the bottom of this!
If you would like to continue troubleshooting on your own, I would suggest trying one drive per set of cables to see if the issues continue rather than loading them up full to the brim. If they all show up as expected then maybe add drives until they stop showing up. The back planes are hotswap capable!
I sent an e-mail earlier today. I’ll wait for a reply before spending more time spinning my wheels on this.
As for being drives being hot-swappable: I am aware of that but for debugging I wanted to focus on one problem a time. FWIW, most recently I added a drive via hotswap only to cause the rest of the drives in the group to go and stay offline (again in BP2 aka 9-12). They all stayed offline even after powering down and back up again. So I’m back to not all slots working. 
Hey @CantDriveFortyFive (sweet name btw)
Sounds great!
I just wanted to follow up and make sure that we got a ticket created and specialist working on that for you!
Thanks for your time,