This is my third post this week, possibly a record for me. All three are centered around ensuring the correct firmware and drivers are installed and running. The content of this post was created by my colleague, David Boone, who works with VMware customers to ensure successful Virtual SAN deployments. When it comes to VSAN, its important to use qualified hardware but equally important to make sure the correct firmware and drivers are installed.
Download the Correct I/O Controller Driver
Navigate to the VMware Compatibility Guide for Virtual SAN, scroll down and select “Build Your Own based on Certified Components”, then find the controller in the database. Here’s the link for the Cisco 12G SAS Modular Raid Controller and the link to download the correct driver for it (as of Nov. 20, 2015): https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESX55-LSI-SCSI-MEGARAID-SAS-660606001VMW&productId=353
Install the Correct Driver
Use your favorite way to install the driver. This might include creating a custom vSphere install image to deploy on multiple hosts, rolling out via vSphere Update Manager (VUM), or manually installing on each host.
Here’s one manual way:
Note: if this is a new cluster, these steps work fine. If this cluster has existing workloads, consider putting the host in maintenance mode prior to executing these steps.
- Use something like WinSCP to copy the downloaded driver zip file into /tmp on each ESXi host.
- SSH to the ESXi host.
- Unzip it from the /tmp directory.
[root@esxi01:~] cd /tmp [root@esxi01:/tmp] unzip megaraid_sas-6.606.06.00-2351571.zip Archive: megaraid_sas-6.606.06.00-2351571.zip
- Install / upgrade the driver. (This example uses the offline bundle as the depot.)
[root@esxi01:/tmp] esxcli software vib install -d /tmp/megaraid_sas-6.606.06.00-offline_bundle-2351571.zip Installation Result Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective. Reboot Required: true VIBs Installed: LSI_bootbank_scsi-megaraid-sas_6.606.06.00-1OEM.5184.108.40.2061820 VIBs Removed: VMware_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.600.1.17.3029758 VIBs Skipped:
- [root@esxi01:/tmp] reboot
- Repeat for each ESXi host being installed.
Verify existing driver being used
- SSH to the ESXi host
- To confirm VIB install and version, run the command:
[root@esxi01:~] esxcli software vib list | grep scsi-megaraid-sas scsi-megaraid-sas 6.606.06.00-1OEM.5220.127.116.111820 LSI VMwareCertified 2015-11-20
- To show the LSI adapter and the driver it loaded for it, run the command:
[root@esxi01:~] esxcli storage core adapter list
- If you see an entry with ‘lsi_mr3’ like the one below,
vmhba1 lsi_mr3 link-n/a sas.578da6e715c33a10 (0000:0e:00.0) LSI MegaRAID SAS Invader Controller
then you need to disable the vSphere native lsi_mr3 driver.
Disable the lsi_mr3 driver (if necessary)
- Enter Maintenance Mode
[root@esxi01:~] vim-cmd hostsvc/maintenance_mode_enter 'vim.Task:haTask-ha-host-vim.HostSystem.enterMaintenanceMode-307'
- Disable the Driver:
[root@esxi01:~] esxcli system module set --enabled=false --module=lsi_mr3
- Exit Maintenance Mode:
[root@esxi01:~] vim-cmd hostsvc/maintenance_mode_exit 'vim.Task:haTask-ha-host-vim.HostSystem.exitMaintenanceMode-356'
Verify the Correct Driver is Now Downloaded
After the reboot, execute ‘esxcli storage core adapter list’ and verify megaraid_sas is the driver for the LSI HBA.
You should see an entry with ‘megaraid_sas’ similar to the one below,
vmhba1 megaraid_sas link-n/a sas.578da6e715c33a10 (0000:0e:00.0) LSI MegaRAID SAS Adapter
You can also use ‘vmkload_mod -s megaraid_sas’ to verify driver version and view other settings.
You are now running the correct version of the driver for Virtual SAN.
Difference between ASYNC and INBOX drivers by VMware
Determining which storage or network driver is actively being used on ESXi host(1034674)
Installing async drivers on VMware ESXi 5.x and ESXi 6.0.x (2005205)
Thanks again to David Boone for creating this content and our friends at Cisco for providing the hardware and support.
2 thoughts on “Steps to Implement the Correct I/O Controller Driver for the Cisco 12G SAS Modular Raid Controller for Virtual SAN”
Seriously some good stuff here. I used update manager to update all of my hosts with the driver, but still had to disable the mr3 driver from the SSH console. Thank you so much for posting this!!
Hell yes! Thanks, man. This looks like it will save the day.