Troubleshooting TrueNAS Core Drive Issues

Great news! And now that you’ve made this declaration, we will shortly find if your optimism was well founded or witness Murphy’s law in full force. :rofl:

If you’re still going strong in a few more days, consider adding “[SOLVED]” to the title of this thread.

can i ask for confirmation, can this all be flashed within truenas, assuming all my disks attached to the controller being flashed are removed from the system?

right now i have this in my HL15, which shows it is running in IR mode.


root@truenas[~]# sas3flash -list
Avago Technologies SAS3 Flash Utility
Version 16.00.00.00 (2017.05.02)
Copyright 2008-2017 Avago Technologies. All rights reserved.

        Adapter Selected is a Avago SAS: SAS3008(C0)

        Controller Number              : 0
        Controller                     : SAS3008(C0)
        PCI Address                    : 00:19:00:00
        SAS Address                    : 5003048-0-1d02-750c
        NVDATA Version (Default)       : 0e.00.21.00
        NVDATA Version (Persistent)    : 0e.00.21.00
        Firmware Product ID            : 0x2721 (IR)
        Firmware Version               : 15.00.03.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : LSI3008-IR
        BIOS Version                   : 08.35.00.00
        UEFI BSD Version               : 17.00.00.00
        FCODE Version                  : N/A
        Board Name                     : LSI3008-IR
        Board Assembly                 : N/A
        Board Tracer Number            : N/A

        Finished Processing Commands Successfully.
        Exiting SAS3Flash.

the article you linked to

has the firmware files for 16.00.12.00, but it seems like supermicro has version 16.00.14.00

if i run sas3flash -o -f SAS9300_xx_IT.bin that will not only update it to the the new 16.00.12.00 firmware but also change it from IR to IT?

since you successfully performed the flashing multiple times, can you give more details into exactly the process you used?

It’s been a while so I don’t really have a step by step on the exact process I performed. It should very much be possible inside of TrueNAS, but I’d suggest going with a method specifically outlined by SuperMicro. As I mentioned above, I used the UEFI method via the instructions provided in RAR file. Copy the files to a FAT32 thumb drive and boot into UEFI shell buy pressing F11 on startup. Supermicro has an FAQ on this linked below.

From there, follow the instructions to run the script from the IT folder. It should update just fine and be in IT mode.

1 Like

Do you suggest 16.00.12.00 that is available on the truenas forums or the newer version directly off supermicro?

I have been having reboots, power shutdowns, and seen a few times when I get kernel messages about power on resets of two of my drives, etc like others have so I am wondering if this can be the fix.

Do you recall if your system was received in IR mode? Do you recall doing anything specific to get it to IT mode?

Just checking as this is new to me and wish to make sure as looking online for “cross flashing from IR to IT” seems to be more involved where I appear to need to erase things first etc.

Maybe I’m missing something, but that sounds like more severe symptoms than where this thread started. I wouldn’t expect HBA firmware to cause this, but maybe you are seeing otherwise. You might want to re-seat your RAM and run a RAM test. Have you contacted info@45omelab.com? If you received an X11SPH -based system that is crashing and is in RAID mode, that seems like they didn’t properly test it before shipping.

Which PSU do you have? The 1000W one?
Which CPU?
Do you have any other PCIe cards installed like a GPU?

i am actually getting an RMA replacement system due to the issues i have as we have performed troubleshooting over the last several weeks and have not yet identified the issue. However, reading the replies in the threads linked to, it sounds like some people have had system crash issues (possibly) because of the old firmware on the HBA? just concerned that the new system might behave the same way?

Howdy! I never had system issues per se: no unexplained reboots or kernel panics. The actual system has always been very stable: I was having issues solely with drive reliability (unexplained illusory SMART errors, constant warnings etc). Your problem could be related to the HBA, but I concur with DigitalGarden, I think that your problem is a touch more severe.
Also, update after a year: my system is still running well!

2 Likes

thanks. Agreed it is likely more severe, which is why 45drives is replacing the system for me. just waiting on the new system to arrive and they will give me a shipping label to send the old one back

My opinion: always go with the latest revision that’s available unless you have a reason not to.

Your system should have arrived in IT mode. There must have been some kind of mistake in your build. IR mode doesn’t play nice with ZFS.

This has been my experience as well. System itself is stable but the issues are with the hard drives and the ZFS pool.

I’d suggest holding off at this point on doing anything else. You’re getting a replacement and I doubt you’ll see the same problems twice (although it’s possible). Also, 45HomeLab will probably want to check out your system for themselves to figure out what went wrong.

1 Like