vSphere 7 Assignable Hardware and Dynamic DirectPath I/O for Latency Sensitive Applications

Lately I’ve been looking into virtualizing latency sensitive applications like the ones used in a lot of financial institutions. In vSphere 6 and below, configuring SR-IOV to hardware accelerate network traffic for latency sensitive VM’s would limit the vSphere features per the SR-IOV Support documentation. This is also true with DirectPath I/O. However, the good news is that vSphere 7 has a new feature called Assignable Hardware that has two consumers; The new Dynamic DirectPath I/O and NVIDIA vGPU. Dynamic DirectPath I/O helps by providing the same functionality as ‘legacy’ DirectPath I/O, but does not pin a workload/VM to a host. This brings back HA and DRS Initial placement to VM’s configured for latency sensitive applications. vMotion is not supported because vSphere cannot live migrate a VM that is directly tied to a physical device. SR-IOV devices are also supported by Assignable Hardware when used as pure passthrough devices.

The VMware High Performance Compute team has been working with our vSphere development teams to bring more capabilities to high performance compute and latency sensitive virtualized workloads without sacrificing bare metal performance.

For more details on Assignable Hardware, check out:

vSphere 7 – Assignable Hardware

vSphere 7 – Using Assignable Hardware with Dynamic DirectPath IO

IoT is Cool, But What About the Plumbing?

When Internet of Things (IoT) is mentioned, people tend to get really excited about all the interesting use cases around smart building, smart city, smart manufacturing, smart retail, smart healthcare, etc. Let’s face it, the reason IoT has become so popular lately is that the data from IoT devices is being utilized in more and more intelligent ways. We commonly refer to this place where data is being collected and decisions are being made as the insight plane or content plane.

What tends to get ignored in IoT is IoT & Edge Device Management.

In my discussions with customers they tend to talk about various projects that got started, but, never moved from concept to reality because they didn’t have a way to operationalize it at scale. There are many things to think about when it comes to this that start to sound pretty boring compared to facial recognition, autonomous driving, and medical imaging analysis. First, what good is it to have a really good analytic engine making decisions on data if it can’t get the data? What if a temperature sensor battery is low, or a video surveillance camera is tampered with, or you need a software update to a edge gateway system that’s collecting data from things? How do you get all the devices onboarded under management at scale in a consistently configured manner? Then what if there’s a configuration changed needed across all devices? How about monitoring the health of the devices, or collecting data off devices? What if there’s a security patch that needs to be deployed to thousands of devices? How do you make sure all of this is done in a secure and trusted manner? We commonly refer to all this as the Management or Control Plane. Another way to describe it is that it’s the plumbing of IoT. No one really wants to deal with it, but, they know they want it all to just work. That’s where VMware’s Pulse IoT Center comes in. As I mentioned in a previous blog here, VMware has historically been great at providing world class software to manage infrastructure and help it run more efficiently. That’s what Pulse IoT Center is doing. Yes we are excited about all the extremely interesting use cases our customers are deploying. We can help them with data collection and supplying that data to their intelligent applications, but, we are also happy to help deal with the complexities of managing all these hundreds to thousands to millions of devices.

VMworld 2018 – vSAN Technical Customer Panel on vSAN Experiences (HCI1615PU)

We had a great time at VMworld 2018 during the vSAN Technical Customer Panel with these 4 great vSAN customers:


They introduced themselves, discussed how they are using vSAN in their environment, and the benefits achieved. After that, we had a stream of questions from the audience that provoked some interesting discussions. If you want to check it out you can view the recorded session here:

vSAN Technical Customer Panel on vSAN Experiences (HCI1615PU)

Also, there is a great TechTarget Converged Infrastructure summary of the session by Dave Raffo here:

vSAN hyper-converged users offer buying, implementing advice

This is the third year in the row I’ve been fortunate enough to host this session. This year was the best attended and had the best audience questions. FYI, my colleague, Lee Dilworth, will be hosing this session in Barcelona so we look forward to a good crowd with more good questions and discussion.

vSAN and Data-At-Rest Encryption – Why SED’s are not Supported (i.e. Part 3)

I first wrote about vSAN and Encryption here: Virtual SAN and Data-At-Rest Encryption

And then again here: vSAN and Data-At-Rest Encryption – Rebooted (i.e. Part 2)

And then vSAN Encryption went live in vSAN 6.6 announced here: vSAN 6.6 – Native Data-at-Rest Encryption

Today I was asked if vSAN supports Self Encrypting Drives (SED). The answer is No. The vSAN product team looked at SEDs but there are too few choices, they are too expensive, and they increase the operational burden.

vSAN only supports vSAN Encryption, VM Encryption, or other 3rd party VM encryption solutions like HyTrust DataControl.

vSAN is Software Defined Storage so the product team decided to focus on software-based encryption to allow vSAN to support data at rest encryption (D@RE) on any storage device that exists today or will come in the future. When vSAN went live supporting Intel Optane, this new flash device was immediately capable of D@RE. The vSAN Encryption operational model is simple. Just click a check box to enable it on the vSAN datastore and point to a Key Management Server. One encryption key to manage for the entire vSAN datastore. The additional benefits of vSAN Encryption is that it supports vSAN Dedupe and Compression and vSAN 6.7 encryption has achieved FIPS 140-2 validation.

Another choice is to leverage VMware’s VM Encryption described here: What’s new in vSphere 6.5: Security
This is per VM encryption, so you point vCenter to a Key Management Server and then enable encryption per VM via policy. This flexibility allows some VM’s to be encrypted and some not to be. And, if the VM is migrated to another vSphere cluster or to VMware Cloud on AWS, the encryption and key management follows the VM. This requires the administrator to manage a key per VM, and because the encryption happens immediately as the write leaves the VM and goes through the VAIO filter, no storage system will be able to dedupe the VM’s data since each block is unique.

Finally, there are various 3rd party per VM encryption solutions on the market that vSAN would also support. For instance, HyTrust Datacontrol.

I hope this helps clear up what options there are for vSAN encryption and the various tradeoffs.

VMworld 2018 – My 2 Breakout Sessions

I’m looking forward to VMworld 2018 in a few weeks. It’s always a long week but a great time. I look forward to catching up with coworkers, partners, customers, and friends. And, I’ll also have to do a little work. This year I have 2 breakout speaking sessions.

vSAN Technical Customer Panel on vSAN Experiences [HCI1615PU]
Monday, Aug 27, 12:30 p.m. – 1:30 p.m.

The Panel will consist of 4 vSAN customers: General Motors, United States Senate Federal Credit Union, Rent-A-Center, and Brinks Oakland University. Brinks is a great vSAN customer but is doing an NSX session at the same time as the vSAN session so we are lucky to add Oakland University to the panel. I will moderate the session, ask the customers to describe their company, role, environment, and how they are using vSAN. General Motors will talk about their large VDI deployment. Unites States Federal Credit Union will discuss their use of vSAN in remote offices, VVols, and Storage Policy Based Management (SPBM). Rent-A-Center will discuss vSAN for management clusters, VDI, and the benefit of VxRail. Oakland University will discuss their vSAN stretched cluster, Data at Rest Encryption, and Dedupe/Compression. After each panelist does this, we’ll take questions from the audience.

Here’s a recording of last year’s session to give you an idea: https://youtu.be/x4ioatHqQOI 
On the panel we had Sanofi, Travelers, Sekisui Pharmaceutical, and Herbalife. The year before we had Stanley Black and Decker, Synergent Bank, M&T Bank, and Baystate Health. Both were great sessions and this year looks like it will be too.

Achieving a GDPR-Ready Architecture Leveraging VMware vSAN [HCI3452BU]
Wednesday, Aug 29, 12:30 p.m. – 1:30 p.m.

When it comes to security in vSAN, most think Data at Rest Encryption and to make this all work you need a key management server. It’s tough to beat HyTrust for this. They offer the software for free and support for a small fee. But that’s not all they do. Check out this session to find out more. Dave Siles and I will discuss GDPR-Ready Architecture and how vSAN encryption can help.

What Capacity Utilization Will I have after I Evacuate a vSAN Host?

To fully evacuate a vSAN host and satisfy FTT=1, FTM=RAID1 you must have at least 4 hosts in the cluster. When a host is put in maintenance mode and fully evacuated, that host data is spread across the surviving hosts. In other words, if you follow the vSAN best practice guidance to stay less than or equal to 70% utilized, then the capacity that represents the 70% utilization must now fit on 3 hosts, which means those 3 hosts become 93% utilized (70% utilized * 4 nodes / 3 nodes = 93.3% utilized). The more hosts you have in the cluster, the less utilized your cluster will be when putting a host in maintenance mode. For example: 70% utilized * 10 nodes / 9 nodes = 77.7% utilized after evacuation of a host.

The formula for this is:

% Utilization after evacuation = (% Utilization before evacuation * # nodes) / (# nodes – 1)

vSphere 6.7 and vSAN 6.7 in the News

Yesterday was a big day for vSphere and vSAN with the launch of the 6.7 release. There are many great blogs written so rather than repeat the content, here’s a list with links.

VMware Written Content

VMware Web Site: What’s New: vSAN 6.7
VMware Virtual Blocks: Extending Hybrid Cloud Leadership with vSAN 6.7
VMware Virtual Blocks: What’s New with VMware vSAN 6.7
VMware Virtual Blocks: vSpeaking Podcast Episode 75: What’s New in vSAN 6.7
Yellow-Bricks.com: vSphere 6.7 announced!
CormacHogan.com: What’s in the vSphere and vSAN 6.7 release?
Tohuw.Net: The Art in the Architecture – vSAN & Shared Nothing

Industry Analyst Content


vSAN ReadyNode Sizer

If you plan on implementing HCI to support your workloads and are looking to size an environment, the vSAN ReadyNode Sizer tool is the place to go.


There are 3 ways to use this.

  • Don’t log in – use the tool in “Evaluation” mode
  • Login using your My VMware account
  • Login using your Partner Central account

In “Evaluation” mode you’ll be able to create some basic configurations for general purpose workloads but will have no ability to customize or download the sizing results.

If you log in using your My VMware account or Partner Central account, you’ll have a lot more functionality. First, you’ll be asked if you want to configure an All Flash cluster or Hybrid cluster.

vSAN Sizer 1

Previously, the only place to size a Hybrid cluster was using the old vSAN Sizing tool. The ability to configure Hybrid clusters was just added to the new tool so now there is one place to size either option.

Next you’ll be asked if you want to size for a “Single Workload Cluster” or “Multi-Workload Cluster”

vSAN Sizer 2

The Single Workload Cluster provides options to configure for VDI, Relational Databases, or General Purpose workloads.

vSAN Sizer 3

The Multi-Workload Cluster choice is I helpful if you plan to have different types of VM’s and want to input the various workload specifics. There are a ton of customization options including block size, IO Pattern, vCPU/Core, etc. And of course, either option allows you to choose what vSAN protection level and method for each workload. You can even size for stretched clusters.

Our great product team at VMware has put a ton of work into this tool including some complex math to come up with a simple and easy way to configure clusters for vSAN. Check out the tool and see for yourself. But, also, feel free to contact your favorite partner or VMware Systems Engineer to also help. The vSAN SE team has done hundreds and thousands of these configurations and can help make sure you’ll be good to go.



Public Speaking Advice

Over my career, I’ve had the opportunity to publicly speak at VMUGs, vForums, Partner events, and other technology focused events. Recently I was asked to provide some Public Speaking Advice and I quickly jotted down some notes in an email and sent them. This is by no means complete, but perhaps someone else will benefit from this:

Be prepared for the worst-case scenarios

  • No network – Assume you will have no network connectivity. Perhaps you will have network connectivity and will be able to link to your live demo system, however, you should be prepared to deliver your message assuming the connectivity is too slow or broken.
  • Broken Laptop – Assume your laptop won’t boot up or can’t connect to the overhead projector. Have your presentation on a USB stick or cloud storage so you can access your presentation from someone else’s device or from your secondary device.
  • Test and verify – Do a dry run of your presentation ahead of time if possible, if not, arrive early and make sure your setup will work.

If it’s a Web based presentation

  • Have a plan if people cannot connect, maybe use an alternate method (e.g. WebEx, Skype, GoToMeeting, etc.) or be prepared to just talk through your material without visuals.
  • Don’t move your mouse all over the place, it’s annoying to the viewers.
  • Make sure you engage the audience. Ask them questions. Don’t just talk and hope they are hearing you.
  • Leverage the web presentation tools to enhance your presentation – whiteboard, highlighter, marker, etc.

If it’s an in person presentation

  • Dress for success – In other words, dress how you want to be perceived. If its your first meeting with a customer you should probably dress up. Likewise, if you are on stage at a big event, then you’ll probably want to wear a dress shirt, sport coat, polished shoes. But if it’s a technical deep dive at a customer or a technical breakout session at a conference, then you might want to dress more casual, perhaps in your company golf shirt.
  • Empty your pockets – this will prevent you from fidgeting with your cell phone, wallet, coins, etc.
  • Eliminate other Distractions – Take off your badge, lanyard, or anything that’s distracting so that the focus is on you and what you are saying. Also, don’t pick up pens or markers and click them or open and close the cap repeatedly
  • Setup the Room – Sometimes it is not possible to rearrange the room, but, if it is, then make it so you can move around the room and engage the audience. A U-shape works well for this.
  • Posture – Stand tall, arms at your side in a relaxed confident manner to start the presentation and as much as possible throughout.
  • Hand Gestures & Movement – Use as many hand gestures as possible. It shows your passion and emphasizes the content. Also move around the room as much as possible. It forces people to pay more attention. If someone is on their phone or falling asleep, move closer to them.
  • Maintain Eye Contact – This is a hard skill to master, but, extremely effective when you do. It is the #1 way to help eliminate saying “um” and “ah” which is the #1 complaint against a public speaker. To practice, cut out faces and paste them on the wall. Say a sentence to one face, then randomly make eye contact with another face and say the next sentence or complete thought. Continue to move around and randomly change who you are looking at throughout the presentation.


  • Don’t introduce yourself. Have your opening slide with your name & title on it but don’t repeat that in an opening statement. Start your conversation with an interesting opening statement that makes the audience want to hear more.
  • Keep them as simple as possible. The focus should be on you, the presenter, not the slides. Technical presentations tend to have a lot of details to convey so it may be hard to avoid showing some complex slides, but use them as a trigger for your talk track; never read them.
  • Present what you know (i.e. deleting slides is OK). We in the tech industry often get handed slide decks from corporate marketing which are great, but, often there are certain slides that just don’t make sense or you cannot figure out how to talk to it. Its best to just hide or delete the slide than try to fumble around trying to talk to it, or turn your back to the audience and read it. Build a story that you can tell by only glancing at the slides once in a while.
  • Incorporate a product demo into your presentation if possible. People like to see things in action. Best to do this earlier than leaving it until the end.

Public Speaking Training



VMworld Hands-on-Labs – 9,640 Labs Were delivered by vSAN

The Hands-on-Labs (HoL) at VMworld are always a big hit. A ton of work goes into putting them on and supporting them and everyone seems to love them. This was a big year for vSAN in the HoL. At VMworld Las Vegas, 11,444 labs were completed and the vSAN lab, HOL-1808-01-HCI – vSAN 6.6, was the #2 overall lab completed. Our NSX friends held the #1 spot.

The HoL’s were delivered from 5 different data centers. Each handled approximately 20% of the workloads. vSAN was the storage in 4 of the data centers. 2 of the 4 were VMware data centers running vSphere, NSX, and vSAN for software defined compute, network and storage. Another was IBM BlueMix (SoftLayer) built with VMware Cloud Foundation (vSphere, NSX, vSAN, and SDDC Manager). And the other was VMware Cloud on ASW also built with VMware Cloud Foundation (vSphere, NSX, vSAN, and SDDC Manager). The 5th data center was another VMware data center running traditional storage. This is a great Hybrid Cloud / Multi Cloud example leveraging 3 of our own datacenters and 2 of the largest public cloud data centers offering Infrastructure as a Service (Iaas).


VMware Cross Cloud Architecture


9,640 of the HoL’s were deployed across the 4 vSAN data centers. This represents 84% of the labs delivered at VMworld US were delivered by vSAN. To support the HoL’s, over 90,000 VM’s were provisioned in just 5 days. Actually, more than that since extra HoL’s are pre-provision that don’t all get used. This is a huge win for HCI and vSAN as it performed like a champ for this heavy workload.

These stats are too impressive not to share and they are a great testament to all the people that make it happen.