How to delete the RAID configuration from drives managed by the Cisco 12G SAS Modular Raid Controller

The content of this blog post was created by a couple of colleagues of mine, David Boone and Bradford Garvey. These guys do a phenomenal job making sure VMware Virtual SAN customers get a great experience with the product by helping them plan, configure, and test VSAN. Because of this, they end up uncovering interesting information like what’s to follow.

Sometimes Cisco UCS hardware intended to be used for VMware Virtual SAN has previously been configured for other uses. In these cases, sometimes a RAID configuration has already been configured on the drives. VSAN requires the individual drives be presented to ESXi either raw via the I/O controller set in Pass-Through Mode (See: How To Configure the Cisco 12G SAS Modular Raid Controller for Pass-Through Mode) or disks set in their own RAID 0 disk groups. Best practice is to set the I/O Controller in Pass-Through Mode (Enable JBOD).  However, if a RAID configuration previously existed, on the Cisco UCS platform there are a few extra steps to complete after enabling JBOD mode for the controller.

If drives were already configured as RAID virtual devices, delete the RAID configuration from the drives.  One way to do that is to Clear the entire VD configuration:

Clear the entire VD configuration

  • Log into the Cisco UCS Manager
  • Open a console to the host
  • Reboot the host
  • On boot up hit Ctrl+R to enter the Cisco 12G SAS Modular Raid Controller BIOS Configuration Utility
  • Hit Ctrl-N until the “VD Mgmt” page is selected
  • In the “VD Mgmt” screen, navigate to the controller, and press the F2 key.
  • Navigate to “Clear Configuration” and press Enter.  You should see this popup:

CiscoUCS - Remove RAID 1

  • Press “Yes” to delete all the virtual drives

Drives will then be in an “Unconfigured Good” state.  They might look something like this:

CiscoUCS - Remove RAID 2

If you see this, these 10 drives are in an “Unconfigured Good” state. They need to be converted to a JBOD state.

There are two options. You can convert a bunch of Unconfigured Good drives to JBOD drives (from the “VD Mgmt” screen) or you can convert a particular Unconfigured Good drive to a JBOD drive (from the “Drive Management” screen)

Option 1: Convert a bunch of Unconfigured Good drives to JBOD drives

Perform the following steps to convert a bunch of Unconfigured Good drives to JBOD drives:

  • In the “VD Mgmt” screen, navigate to the controller and press the F2 key.
  • Navigate to “Make JBOD”, and press Enter.
    The “Convert Unconfigured Good to JBOD” dialog appears, which shows all Unconfigured Good drives in the system.

CiscoUCS - Remove RAID 3

 

  • Select the Unconfigured Good drives which you want configured as JBODs for VSAN.
    To select or deselect all the Unconfigured Good drives at one go, select the topmost square backets in the “Unconfig good drives” box.
  • Press “OK”.
    The selected Unconfigured Good drives are converted to JBOD drives.

Option 2: Convert a particular Unconfigured Good drive to a JBOD drive

Perform the following steps to convert a particular Unconfigured Good drive to a JBOD drive:

  • In the “Drive Management” screen, navigate to an Unconfigured Good drive, and press the F2 key.
  • Navigate to “Make JBOD”, and press Enter.
  • Press “OK” in the message confirmation box to continue.

After converting all the 10 drives above to JBOD, the screen looks like this:

CiscoUCS - Remove RAID 4

Result

After rebooting, the BIOS will report all 10 drives and ESXi will see all of them in a JBOD (Pass-Through) configuration, with all the benefits of JBOD like being able to retrieve S.M.A.R.T.S. info from the physical drives.

The information obtained to create this post was gathered from the Avago – 12Gb/s MegaRAID® SAS Software – User Guide

Thanks again to David Boone and Bradford Garvey for providing this information.

Advertisements

How To Configure the Cisco 12G SAS Modular Raid Controller for Pass-Through Mode

Yesterday I was at the New England VTUG event which is always a great event to meet up with familiar faces and be introduced to some new ones. I met up with a relatively new VMware Virtual SAN customer and we discussed lots of fun things about VSAN and their implementation experience. One frustrating thing they mentioned is that they couldn’t find anywhere that documented how to put the Cisco 12G SAS Modular Raid Controller in Pass-Through mode. They explained that after lots of searching on VMware and Cisco’s site, they contacted Cisco and were provided the information. They were kind enough to capture a screenshot of the setting and provide it to me.

The procedure is:

  • Log into the Cisco UCS Manager
  • Open a console to the host
  • Reboot the host
  • On boot up hit Ctrl+R to enter the Cisco 12G SAS Modular Raid Controller BIOS Configuration Utility
  • Hit Ctrl-N until the “Ctrl Mgmt” page is selected
  • In the bottom right hand corner, make sure the “Enable JBOD” field shows an X per the screen shot below.
  • Hit Ctrl-S to save Reboot

Cisco 12G SAS Enable JBOD

That’s it. Easy.

If this is a brand new, unconfigured host, the unclaimed disks in the host will now get passed to ESXi and VSAN can use them for the VSAN datastore.

However, if this host IO Controller had previously been configured with RAID, you should check out: How to delete the RAID configuration from drives managed by the Cisco 12G SAS Modular Raid Controller

I hope that helps others save some time in getting VSAN up and running.

Special thanks to Stephanie Forde and Matthew Gabrick from the Boston Water and Sewer Commission for pointing this out and providing the screenshot.

Queue Depth and the FBWC Controller Cache module on the Cisco 12G SAS Modular Raid Controller for Virtual SAN

If you scan the bill of materials for the various Cisco UCS VSAN ReadyNodes you’ll see a line item for:

Controller Cache: Cisco 12Gbps SAS 1GB FBWC Cache module (Raid 0/1/5/6)

If you’ve followed Virtual SAN for awhile you might wonder, why would the ReadyNodes include controller cache when VMware recommends disabling controller cache when implementing Virtual SAN. Well, it turns out that the presence of the FBWC Cache module allows the queue depth of the Cisco 12G SAS Modular Raid Controller to go from the low 200’s to the advertised 895. The minimum queue depth requirement for Virtual SAN is 256 so including the FBWC Cache module allows the queue depth to increase above that minimum requirement and improve Virtual SAN performance.

Steps to Implement the Correct I/O Controller Driver for the Cisco 12G SAS Modular Raid Controller for Virtual SAN

This is my third post this week, possibly a record for me. All three are centered around ensuring the correct firmware and drivers are installed and running. The content of this post was created by my colleague, David Boone, who works with VMware customers to ensure successful Virtual SAN deployments. When it comes to VSAN, its important to use qualified hardware but equally important to make sure the correct firmware and drivers are installed.

Download the Correct I/O Controller Driver

Navigate to the VMware Compatibility Guide for Virtual SAN, scroll down and select “Build Your Own based on Certified Components”, then find the controller in the database. Here’s the link for the Cisco 12G SAS Modular Raid Controller and the link to download the correct driver for it (as of Nov. 20, 2015): https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESX55-LSI-SCSI-MEGARAID-SAS-660606001VMW&productId=353

Install the Correct Driver

Use your favorite way to install the driver. This might include creating a custom vSphere install image to deploy on multiple hosts, rolling out via vSphere Update Manager (VUM), or manually installing on each host.

Continue reading “Steps to Implement the Correct I/O Controller Driver for the Cisco 12G SAS Modular Raid Controller for Virtual SAN”

What if the SSD and HDD Firmware Versions are Newer Than What is Listed on the VMware Compatibility Guide (VCG) for Virtual SAN?

No problem, this is OK.

If you want to know more detail, keep reading…

Last week I was working with a customer to implement a VSAN ReadyNode. Before enabling VSAN on a cluster it’s a best practice to validate that the firmware of the host I/O Controller, SSD’s (SAS, SATA, PCIe, NVMe, or UltraDIMM), and HDD’s (SAS, NL-SAS, or SATA) are up to the required versions. Each hardware vendor has a different way of doing this.

In reviewing this particular customers hardware, we found that the SSD and HDD Firmware Versions were newer than what is listed on the VCG.

Note that for SSD’s and HDD’s, the hardware vendors provides the VMware Virtual SAN team with the firmware version they tested and qualified for VSAN. VMware then lists that firmware version for that model of disk on the VMware Compatibility Guide (VCG) for Virtual SAN. If the hardware vendor comes out with “new firmware” then it does not require VSAN re-certification of the SSD or HDD. VMware supports disks with “newer firmware” for Virtual SAN but VMware leaves the VCG alone and continues listing the “old firmware”. However, if the hardware vendor wants VMware to remove the “old firmware” from the VCG listing and replace it with the “new firmware” VMware would do that upon their request. This would typically happen if the hardware vendor discovers an issue/bug with the “old firmware”.

I hope this helps clarify how VMware treats SSD and HDD Firmware Version listings on the VMware Compatibility Guide for Virtual SAN.

Verifying the Correct Version of the Cisco UCS C240 Server I/O Controller Firmware – Cisco 12G SAS Modular Raid Controller

Today I was working with Cisco to setup UCS C240 servers for Virtual SAN. As part of the process we needed to verify the Cisco 12G SAS Modular Raid Controller had the correct Firmware Version.

First we went to the VMware Compatibility Guide for Virtual SAN, navigated to the bottom of the page to the link for Build Your Own based on Certified Components. Under “Search For:” we selected “I/O Controller” and under “Brand Name:” we selected “Cisco” and found the listing for the Cisco 12G SAS Modular Raid Controller. It requires Firmware Version 4.270.00-4238.

vcg-Cisco 12G

Next we went into the Cisco UCS Manager and navigated to the Host Firmware Packages and found that the Storage Controller Firmware Package was 24.7.0-0047.

CiscoUCS1

Through UCS Manager there is no way to get the I/O Controller Firmware Version. So, we had to reboot the host and hit “CTRL-R” to get into the Cisco 12G SAS Modular Raid Controller Bios Configuration Utility.

CiscoUCS2

From here we hit CTRL-N to get into Properties.

CiscoUCS3

On this screen you can see:

Package: 24.7.0-0047

FW Version: 4.270.00-4238

Thus, we were able to confirm that we had the correct Firmware on the I/O Controller. If the FW Version was different than what VMware Virtual SAN supports, you would need to download the correct Firmware Package from Cisco and upgrade.

I hope this helps others save time trying to verify Firmware Versions. Thanks to my VMware Virtual SAN colleague, David Boone, who did most of the work that led to this post and our friends at Cisco for being a great partner and helping navigate UCS Manager and grabbing screenshots.