Steps to Implement the Correct I/O Controller Driver for the Cisco 12G SAS Modular Raid Controller for Virtual SAN

This is my third post this week, possibly a record for me. All three are centered around ensuring the correct firmware and drivers are installed and running. The content of this post was created by my colleague, David Boone, who works with VMware customers to ensure successful Virtual SAN deployments. When it comes to VSAN, its important to use qualified hardware but equally important to make sure the correct firmware and drivers are installed.

Download the Correct I/O Controller Driver

Navigate to the VMware Compatibility Guide for Virtual SAN, scroll down and select “Build Your Own based on Certified Components”, then find the controller in the database. Here’s the link for the Cisco 12G SAS Modular Raid Controller and the link to download the correct driver for it (as of Nov. 20, 2015): https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESX55-LSI-SCSI-MEGARAID-SAS-660606001VMW&productId=353

Install the Correct Driver

Use your favorite way to install the driver. This might include creating a custom vSphere install image to deploy on multiple hosts, rolling out via vSphere Update Manager (VUM), or manually installing on each host.

Here’s one manual way:
Note: if this is a new cluster, these steps work fine. If this cluster has existing workloads, consider putting the host in maintenance mode prior to executing these steps.

1. Use something like WinSCP to copy the downloaded driver zip file into /tmp on each ESXi host.
2. SSH to the ESXi host.
3. Unzip it from the /tmp directory.
   [root@esxi01:~] cd /tmp [root@esxi01:/tmp] unzip megaraid_sas-6.606.06.00-2351571.zip Archive: megaraid_sas-6.606.06.00-2351571.zip

   inflating: megaraid_sas-6.606.06.00-offline_bundle-2351571.zip
   inflating: scsi-megaraid-sas-6.606.06.00-1OEM.550.0.0.1331820.x86_64.vib
   inflating: doc/README.txt
   inflating: source/driver_source_scsi-megaraid-sas_6.606.06.00-1OEM.550.0.0.1331820.tgz
   inflating: doc/open_source_licenses_scsi-megaraid-sas_6.606.06.00-1OEM.550.0.0.1331820.txt
   inflating: doc/release_note_scsi-megaraid-sas_6.606.06.00-1OEM.550.0.0.1331820.txt

4. Install / upgrade the driver. (This example uses the offline bundle as the depot.)
   [root@esxi01:/tmp] esxcli software vib install -d /tmp/megaraid_sas-6.606.06.00-offline_bundle-2351571.zip Installation Result Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective. Reboot Required: true VIBs Installed: LSI_bootbank_scsi-megaraid-sas_6.606.06.00-1OEM.550.0.0.1331820 VIBs Removed: VMware_bootbank_scsi-megaraid-sas_6.603.55.00-2vmw.600.1.17.3029758 VIBs Skipped:
5. [root@esxi01:/tmp] reboot
6. Repeat for each ESXi host being installed.

 

Verify existing driver being used

1. SSH to the ESXi host
2. To confirm VIB install and version, run the command:

   [root@esxi01:~] esxcli software vib list | grep scsi-megaraid-sas
   scsi-megaraid-sas             6.606.06.00-1OEM.550.0.0.1331820       LSI     VMwareCertified   2015-11-20

3. To show the LSI adapter and the driver it loaded for it, run the command:

   [root@esxi01:~] esxcli storage core adapter list

4. If you see an entry with ‘lsi_mr3’ like the one below,
   

   vmhba1    lsi_mr3      link-n/a    sas.578da6e715c33a10                (0000:0e:00.0) LSI MegaRAID SAS Invader Controller

then you need to disable the vSphere native lsi_mr3 driver.

Disable the lsi_mr3 driver (if necessary)

1. Enter Maintenance Mode

   [root@esxi01:~] vim-cmd hostsvc/maintenance_mode_enter
   'vim.Task:haTask-ha-host-vim.HostSystem.enterMaintenanceMode-307'

2. Disable the Driver:

   [root@esxi01:~] esxcli system module set --enabled=false --module=lsi_mr3

3. Exit Maintenance Mode:

   [root@esxi01:~] vim-cmd hostsvc/maintenance_mode_exit 
   'vim.Task:haTask-ha-host-vim.HostSystem.exitMaintenanceMode-356'

4. Reboot

   [root@esxi01:/tmp] reboot

Verify the Correct Driver is Now Downloaded

After the reboot, execute ‘esxcli storage core adapter list’ and verify megaraid_sas is the driver for the LSI HBA.

You should see an entry with ‘megaraid_sas’ similar to the one below,

vmhba1    megaraid_sas      link-n/a    sas.578da6e715c33a10                (0000:0e:00.0) LSI MegaRAID SAS Adapter

You can also use ‘vmkload_mod -s megaraid_sas’ to verify driver version and view other settings.

You are now running the correct version of the driver for Virtual SAN.

Other references:

Difference between ASYNC and INBOX drivers by VMware

Determining which storage or network driver is actively being used on ESXi host(1034674)

Installing async drivers on VMware ESXi 5.x and ESXi 6.0.x (2005205)

Thanks again to David Boone for creating this content and our friends at Cisco for providing the hardware and support.