Troubleshooting LSI RAID issues in Proxmox with CLI

Managing LSI RAID in Proxmox with CLI tools

I had a Proxmox host with an LSI-based controller where a pulled drive did not autorebuild when I put it back. The kernel had the megaraid_sas driver, but no vendor CLI tools were installed. I used the vendor utilities to inspect controller settings, start a rebuild, and check drive health. The steps below are what worked for me with Proxmox LSI RAID.

What you see

The symptoms are blunt and repeatable. The array shows a degraded state in the controller, but the OS sees all physical drives present. I have seen reports like “it wont autorebuild when removing a drive and then a few moments later reinsert it” and front-panel drive LEDs not lighting even when the drive is in a RAID.

Look in these places for concrete signs.

• Proxmox GUI: VM or storage warnings, datastore shown as degraded.
• dmesg and syslog: look for megaraid_sas messages or disk errors.
• Controller CLI, when available: shows the logical drive as degraded, missing, or foreign.

Useful quick checks:

• Check the kernel driver and controller presence:

lsmod | grep megaraid_sas

Expected: a line with megaraid_sas. If it is absent, the driver is not loaded.

lspci -nnk | grep -A3 -i raid

Expected: an LSI/SAS controller entry with driver=megaraid_sas.

dmesg | grep -i megaraid

Look for firmware events, device resets, or failed commands.

If drives are present to the OS but the controller reports a degraded array, the controller configuration is usually what controls rebuild behaviour.

Where it happens

This is Proxmox, not mdadm. Proxmox LSI RAID typically uses the megaraid_sas kernel module to present the adapter and logical drives. The hardware RAID state and rebuild policy live on the controller firmware. That means the OS can show disks while the controller still refuses to auto-rebuild, depending on its settings.

Controller settings to check:

• Auto-rebuild policy: on some LSI firmware this can be disabled.
• Hot-spare assignment and drive state: a drive can be marked as unconfigured good, foreign, or online.
• Migration and patrol read settings: these affect rebuild timing and LED behaviour.

CLI tool availability:

• Proxmox does not always ship vendor CLI tools like storcli or MegaCLI by default. If they are missing, install the vendor package from Broadcom/LSI or copy it to the host if it is offline. The kernel driver can be present while the utilities are not.
• storcli and MegaCLI expose the controller state and let you start or monitor rebuilds from the OS rather than the BIOS.

Reference: a discussion of missing CLI tools and the megaraid_sas driver on a fresh Proxmox install is available here: https://reddit.com/r/Proxmox/comments/1pnw2cv/missing_clitools_for_lsi_raidadapters/ and Broadcom’s support pages list StorCLI as the vendor tool for MegaRAID controllers: https://www.broadcom.com/support/download-search

Find the cause

I use a short checklist to narrow the problem to controller policy, physical fault, or missing tools.

1) Map host devices to controller objects

storcli /c0 show

Expected: controller present and firmware version.

If storcli is missing, use megacli -AdpAllInfo -aAll or check /proc/scsi/scsi for device mapping.

storcli /c0/vall show

Expected: logical volumes listed and their state, such as Optimal or Degraded.

If the vendor tools are not present, you only see SCSI block devices in the OS. That can hide controller-level flags like foreign or no rebuild.

2) Check physical drive state

storcli /c0/eall/sall show

Expected: each slot shows State: Online or UGood (Unconfigured Good) if it is not assigned.

Alternative:

smartctl -a /dev/sdX

Use that for individual drives to confirm SMART health.

3) Inspect driver and kernel logs

dmesg | grep -iE 'megaraid|raid|error|failed'

Expected: occasional info messages for normal operation.

Problem indicators: repeated firmware events, I/O errors to a physical disk, or controller timeouts.

4) Check BIOS and firmware settings

Boot to the controller BIOS, usually with Ctrl+R or the vendor key, and verify:

• Auto-rebuild enabled.
• Foreign configuration handling, usually Import vs Clear.
• Hot-spare behaviour and global hot-spare settings.

Root causes I have seen:

• Auto-rebuild disabled in firmware.
• Drive reinserted marked as Foreign configuration and not automatically imported.
• Drive marked Unconfigured Good but not added to the VD because of rebuild policy.
• Missing CLI tools, so I could not view or change controller flags from the OS.

Fix

Get the vendor CLI on the host, inspect the controller, and change the policy or force a rebuild where that is appropriate. Be careful here. Hardware RAID changes are destructive if used wrong.

Install the CLI tools.

• If you are online, download storcli from Broadcom for your controller and install it, usually as an RPM or Debian package. If the host is offline, download it on another machine and copy the package to the host with scp.

storcli /c0 show

Expected: controller information.

Common corrective actions with storcli or MegaCLI:

• Check the rebuild setting and enable auto-rebuild if it is disabled.

Example storcli command, syntax varies by version:

storcli /c0 show rebuild settings

• If the drive is Unconfigured Good, add it to the VD.

Example:

storcli /c0/v0 add spares=E:S

Here E is the enclosure and S the slot. Use the right add or replace command for the RAID level.

• Force a rebuild when it is safe to do so.

Example commands vary with firmware. If you are not sure, import the foreign config first:

storcli /c0 /fall show foreign

storcli /c0 /fall import

Then start the rebuild if the controller does not auto-start it.

If you only have MegaCLI:

• megacli -LDInfo -Lall -aAll lists logical drives.
• megacli -PDList -aAll lists physical drives.
• megacli -CfgForeign -Scan -aAll shows foreign configurations.
• MegaCLI has commands to import or start a rebuild, but the syntax is older and less friendly.

Rebuilding a logical drive happens on the controller. Start it only when the physical drive is confirmed healthy.

Check the drive is not failing at SMART level before you rebuild.

If the controller refuses to rebuild:

• Check for firmware bugs and consider updating firmware if the vendor notes match the behaviour.
• If the drive LEDs do not light but the controller marks the drive online, check the backplane cabling and LED connectors.

Check it is fixed

Verification needs to be explicit. I use the vendor CLI and host tools together.

1) Confirm logical drive state

storcli /c0/vall show

Expected: state moves from Degraded to Optimal, or shows progress and 100% completion.

Alternative:

megacli -LDInfo -Lall -aAll

2) Monitor rebuild progress

storcli /c0/v0 show rebuild

Or check the /progress fields.

Expected: the progress percentage increments until complete.

Also watch dmesg for repeated errors while rebuilding.

3) Validate drive health

smartctl -a /dev/sdX

Expected: SMART PASSED, no reallocated sectors, no pending sectors.

Check that the front-panel LEDs reflect activity if you rely on that for monitoring.

4) Lock the settings you changed

• If you enabled auto-rebuild or changed foreign import behaviour, note the controller defaults and document the change in your runbook.
• Reboot to BIOS and confirm the setting persisted if the controller stores it in NVRAM.

Final checks:

• Run a short scrub or patrol if the controller supports it.
• Run a quick I/O test or boot a VM that uses that storage and watch for I/O errors.

Takeaways

• Proxmox LSI RAID relies on the controller. Install storcli or MegaCLI to see and change controller-level behaviour.
• Map OS block devices to controller slots before you act.
• If a drive reinsertion did not trigger a rebuild, check firmware rebuild policy and foreign config handling.
• Use SMART to confirm the physical drive is healthy before forcing a rebuild.