I’ve been working with many customers over the last several months and found that many are very familiar with HP hardware and just know how to set things up. Others are looking for guidance from VMware on how to configure for VSAN. There are things I’ve discovered that might not be obvious but can help in the VSAN setup. Bear in mind, I am not an HP server hardware expert, so your comments are greatly appreciated.
Before I go too far, there is a bug in the HP async controller driver for the HP 420i that is included in the HP ESXi image. The bug reduces the queue depth to 28, instead of 1020, causing poor performance in VSAN.
Here’s how to check your hosts IO Controller (storage adapter) queue depth:
- Run the esxtop command on the ESXi shell / SSH session
- Press d
- Press f and select Queue Stats (d)
- The value listed under AQLEN is the queue depth of the storage adapter
To resolve, follow these directions to implement the correct driver:
HP ProLiant Smart Array Controller Driver for VMware vSphere 5.5 (VIB file)
OK, a little background/overview on I/O Controller guidance for VSAN. In general, VSAN recommends disabling Read and Write cache for any I/O Controller. Since VSAN handles Read and Write caching at the software layer, there’s no need to do it at the hardware level. Also, when destaging write cache, we want to ensure that the writes are committed to disk and not in I/O Controller cache.
In the case of the HP P420i, you cannot disable the I/O Controller cache so VSAN recommends setting it to 100% Read which essentially disables Write cache. I recently discovered that you can also selectively pick and choose which disks to enable cache for.
So, you can deselect all of the disks that will be used for VSAN, thus disabling Read cache for them.
Finally, there’s an option to enable HP SSD Smart Path and enabling it can accelerate reads and writes.
<added on October 17, 2014> DO NOT “Enable HP SSD Smart Path”. Leave Smart Path disabled.
Here’s the description from the HP “Optimized solid-state drive performance with HP SSD Smart Path” Technical white paper: “HP SSD Smart Path improves the performance of select HP Smart Array Controllers in SSD-based HP ProLiant Gen8 servers. HP SSD Smart Path technology allows I/O requests that meet certain requirements to bypass the normal I/O path involving firmware layers, and instead use an accelerated I/O operation called HP SSD Smart Path (seen in figure 1). This process accelerates reads for all RAID levels and writes for RAID 0” Here’s a link to the full PDF details: Optimized solid-state drive performance with HP SSD Smart Path
P420i Setup Walkthrough for VSAN
I captured some screen shots that step though the P420i setup. Here they are:
Upon booting the server, when the P420i Smart Array Controller message appears, press <F5>
Configure –> Cache Manager
Configure –> Cache Manager –> Caching Settings
Set “Cache Ratio” to 100% Read
Make sure all the drives that are to be used for VSAN are not selected under “Select the Logical Drives for which caching is to be enabled”. In this case the 279GB drive will be used to install ESXi so caching can be left enabled for it, as indicated by the green check mark.
Leave “Enable Write Cache When Battery/Capacity Not Present or Not Completely Charged” set to Disabled
Click “OK” to save settings.
Next, we need to create individual RAID-0 disk groups for each disk drive (HDD and SSD). From the main screen, click Unassigned Drives on the left hand column, choose a single drive, and click “Create Array”.
Click “Create Array” to create individual RAID 0 disk groups for each individual disk.
You’ll be prompted for various options for this “Logical Drive”. Accept all the following defaults except you should Disable Caching. Click “Create Logical Drive”.
Review the settings and click “Finish”
Repeat the process for each disk (HDD and SSD).
When completed click on Arrays and select one that you created and it should look like this:
Selecting “Array B – 1 Logical Drive(s) –> “View More Details” shows the following details for one of the HDD’s:
Selecting “Array H – 1 Logical Drive(s) shows the following for one of the SSD’s:
Click “Enable HP SSD Smart Path”
<added on October 17, 2014> DO NOT CLICK “Enable HP SSD Smart Path”. Leave it with Smart Path disabled.
You can now exit the Smart Array Storage Administrator and reboot the host. Once ESXi is installed you will then need to tag the HDD’s as local and the SSD’s as local and SSD. This is done by going into esxcli and running the appropriate command. An example is below for HDD’s and SSD’s.
For HDD’s (MD)
esxcli storage nmp satp rule add –satp VMW_SATP_LOCAL –device <device id> –option “enable_local”
esxcli storage nmp satp rule add –satp VMW_SATP_LOCAL –device <device id> –option “enable_local enable_ssd”
For more details on tagging disks check out:
Enabling the SSD option on SSD based disks/LUNs that are not detected as SSD by default (2013188)
That should do it.
If you have any comments or have had experience running different configurations successfully I’d be happy to hear them.
19 thoughts on “Configuring HP Smart Array P420i I/O Controller for VSAN”
Does the RAID 0 setting for the disks affect the Pass-Thru mode that VSAN prefers?
The P420i does not support Pass-Through mode. It only supports RAID 0 mode, thus, its the only option. Yes, ideally you’d use a Pass-Through mode controller but he P420i works well, has high queue depth, and supports a large number of drives so it’s a good choice for VSAN.
Nice article, hope you continue to do them. Why set the Non-SSD disks to VMW_SATP_LOCAL? what does that help to do?
VSAN will only utilize local disks. By default, ESXi does not see the RAID-0 disk groups as local so you have to run the command to tag them as local disks and run the other command to tag as SSD and local.
Great information. Any idea if the new Gen9 P440 smart array controllers need to be configured the same?
The P440 is currently not yet on the VSAN HCL. It will support both passthrough and RAID 0 mode but we’ll have to wait and see what gets added to the HCL after testing.
Why do you recommend to disable SSD Smart Path? Thanks
Enabling Smart Path has been known to cause stability problems. I originally suggested it be enabled since it is supposed to bypass the controller but upon further review it causes more trouble than what little gain it may have provided.
Is Anyone has tested the HP P440 in passthrough mode with vSAN? I know, it’s not on the HCL yet but I would like to see if it works or not.
When vSphere 6 is generally available you can expect there to be updates to the VMware Compatibility Guide for Virtual SAN for both vSphere 5.5 and vSphere 6. No guarantees as to when, but the P440 controller is an obvious choice to add to the list.
hello from my own experience with the exact same array controller, when adding new disks to an existing cluster that does not allow pass through, you can use the following command to create the Raid0 array with out restarting the esxi host or using maintenance mode.
-Create New RAID 0 Logical Drive:
/opt/hp/hpssacli/bin/hpssacli ctrl slot=0 create type=ld drives=2:6
Thanks for the article, I’ve referenced it many times already. Quick question: I confirmed the QDepth was 28 instead of 1020. I installed the new driver packaged linked in the article and rebooted my host and still see 28 as the depth. Should I be manually adjusting the queue depth and if so, how can I do that? Please and thanks!
You should not manually adjust the queue depth, it should get picked up automatically with the right driver.
I’ve followed your guide, HDD is showed but the SSD not show when i add disk group of VSAN,althought ESXi node detect that SSD is Flash type. Have you encountered this situation yet ? Thanks.
The end of the article talks about how to go into esxcli to tag the HDD as local and SSD’s as local and SSD. So you’ll need to do that for the SSD’s.
a little confused. where do u install esx do you designate like first 2 drives as raid1 and install it there… then rest of drives as raid0?
Usually install to SD card, SATA-DOM, a single HDD, or 2 HDD’s in RAID 1. Then for the VSAN disks, put the IO Controller in passthrough mode if it supports it and let the disks be managed by ESXi. If passthrough is not supported then put each individual disk in its own RAID0 disk group.
I realize this is old and you may have moved away from this hardware. If not, have you had the chance to try vSAN and the P420i after using the CLI to change it to HBA mode?
[./ssacli controller slot=0 modify hbamode=on forced]
I have just started experimenting with this myself and am curious to see if it has any benefits over the RAID0 per drive approach.
Note to self – I was able to configure one of our lab servers both with RAID0 and HBA mode. HBA queue depth is limited to 28 or less, RAID0 mode is 1020. Current firmware and drivers automatically identify SSD LUNs so there should be no need to tag the drives.