vSAN and Data-At-Rest Encryption – Rebooted (i.e. Part 2)

 

Encryption is here, now shipping with vSphere 6.5.

I first wrote about vSAN and Encryption here:

Virtual SAN and Data-At-Rest Encryption – https://livevirtually.net/2015/10/21/virtual-san-and-data-at-rest-encryption/

At the time, I knew what was coming but couldn’t say. Also, the vSAN team had plans that changed. So, let’s set the record straight.

vSAN

  • Does not support Self Encrypting Drives (SEDs) with encryption enabled.
  • Does not support controller based encryption.
  • Supports 3rd party software based encryption solutions like HyTrust DataControl and Dell EMC Cloud Link.
  • Supports the VMware VM Encryption released with vSphere 6.5
  • Will support its own VMware vSAN Encryption in a future release.

At VMworld 2016 in Barcelona VMware announced vSphere 6.5 and with it, VM Encryption. In the past, VMware relied on 3rd party encryption solutions, but now, VMware has its own. For more details, check out:

What’s new in vSphere 6.5: Security – https://blogs.vmware.com/vsphere/2016/10/whats-new-in-vsphere-6-5-security.html

In this, Mike Foley briefly highlights a few advantages of VM Encryption. Stay tuned for more from him on this topic.

In addition to what Mike highlighted, VM encryption is implemented using VAIO Filters, can be enabled per VM object (vmdk), will encrypt VM data no matter what storage solution is implemented (e.g. object, file, block using vendors like VMware vSAN, Dell Technologies, NetApp, IBM, HDS, etc.), and satisfies data-in-flight and data-at-rest encryption. The solution does not require SED’s so it works with all the commodity HDD, SSD, PCIe, and NVMe devices and integrates with several third party Key Management solutions. Since VM Encryption is set via policy, that policy could extended across to public clouds like Cloud Foundation on IBM SoftLayer, VMware Cloud on AWS, VMware vCloud Air or to any vCloud Air Network partner. This is great because your VM’s could live in the cloud but you will own and control the encryption keys. And you can use different keys for different VM’s.

At VMworld 2016 in Las Vegas VMware announced the upcoming vSAN Beta. For more details see:

Virtual SAN Beta – Register Today! – https://blogs.vmware.com/virtualblocks/2016/09/07/virtual-san-beta-register-today/

This vSAN Beta includes vSAN encryption targeted for a future release of vSphere. vSAN Encryption will satisfy data-at-rest encryption. You might ask why vSAN Encryption would be necessary if vSphere has VM Encryption? I will say that you should always look to use VM Encryption first. The one downside to VM Encryption is that since the VM’s data is encrypted as soon as it leaves the VM and hits the ESXi kernel, each block is unique, so no matter what storage system that data goes to (e.g. VMware vSAN, Dell Technologies, NetApp, IBM, HDS, etc.) that block can’t be deduped or compressed. The benefit of vSAN encryption will be that the encryption will be done at the vSAN level. Data will be send to the vSAN cache and encrypted at that tier. When it is later destaged, it will be decrypted, deduped, compressed, and encrypted when its written to the capacity tier. This satisfies the data-at-rest encryption requirements but not data-in-flight. It does allow you to take advantage of the vSAN dedupe and compression data services and it’s one key for the entire vSAN datastore.

It should be noted that both solutions will require a 3rd party Key Management Server (KMS) and the same one can be used for both VM Encryption and vSAN Encryption. The KMS must support the Key Management Interoperability Protocol (KMIP) 1.1 standard. There are many that do and VMware has tested a lot of them. We’ll soon be publishing a list, but for now, check with your KMS vendor or your VMware SE for details.

VMware is all about customer choice. So, we offer a number of software based encryption options depending on your requirements.

It’s worth restating that VM Encryption should be the standard for software based encryption for VM’s. After reviewing vSAN Encryption, some may choose it instead to go with vSAN encryption if they want to take advantage of deduplication and compression. Duncan Epping provides a little more detail here:

The difference between VM Encryption in vSphere 6.5 and vSAN encryption – http://www.yellow-bricks.com/2016/11/07/the-difference-between-vm-encryption-in-vsphere-6-5-and-vsan-encryption/

 

In summary:

  1. Use VM Encryption for Hybrid vSAN clusters
  2. Use VM Encryption on All-Flash if storage efficiency (dedupe/compression) is not critical
  3. Wait for vSAN native software data at rest encryption if you must have dedupe/compression on All-Flash

 

Correlating vSAN versions with vSphere (vCenter & ESXi) Versions

I often get asked if a certain version of vSAN can be deployed on a different version of vSphere. The answer is no. vSAN is built into the vSphere version. That means vCenter needs to be upgraded to the correct version of vCenter and all the hosts in the cluster need to be upgraded to the correct version of ESXi in order to get the features of that version of vSAN. Lastly, vSAN formats each disk drive with an on-disk format, so to get the full features of a specific release, you may need to update the on-disk format.

Here’s basically how everything breaks down:

  • If you have vSphere 5.5 (vCenter Server 5.0 & ESXi 5.0) then you have vSAN 5.5.
  • If you have vSphere 6.0 (vCenter Server 6.0 & ESXi 6.0) then you have vSAN 6.0.
  • If you have vSphere 6.0 U1 (vCenter Server 6.0 Update 2 & ESXi 6.0 Update 1) then you have vSAN 6.1.
  • If you have vSphere 6.0 U2 (vCenter Server 6.0 Update 2 & ESXi 6.0 Update 2) then you have vSAN 6.2.
  • If you have vSphere 6.5 (vCenter Server 6.5 & ESXi 6.5) then you have vSAN 6.5.
  • If you have vSphere 6.5.0d (vCenter Server 6.5.0d & ESXi 6.5.0d) then you have vSAN 6.6.
  • If you have vSphere 6.5 Update 1 (vCenter Server 6.5 Update 1 & ESXi 6.5 Update 1) then you have vSAN 6.6.1.
  • If you have vSphere 6.7 (vCenter Server 6.7 & ESXi 6.7) then you have vSAN 6.7

Here’s a more detailed matrix:

Version Release

Date

Build

Number

Installer Build Number vSAN Version vSAN

On-Disk Format

(Web Client)

ESXi 6.5 U2 2018-05-03 8294253 N/A 6.6.1 U2 5
ESXi 6.7 GA 2018-04-17 8169922 N/A 6.7 GA 6
ESXi 6.6.1 Patch 02 2017-12-19 7388607 N/A 6.6.1 Patch 02 5
ESXi 6.5 Express Patch 4 2017-10-05 6765664 N/A 6.6.1 Express Patch 4 5
ESXi 6.5 Update 1 2017-07-27 5969303 N/A 6.6.1 5
ESXi 6.5.0d 2017-04-18 5310538 N/A 6.6 5
ESXi 6.5. Express Patch 1a 2017-03-28 5224529 N/A 6.5 Express Patch 1a 3
ESXi 6.5. Patch 01 2017-03-09 5146846 5146843 6.5 Patch 01 3
ESXi 6.5.0a 2017-02-02 4887370 N/A 6.5.0a 3
ESXi 6.5 GA 2016-11-15 4564106 N/A 6.5 3
ESXi 6.0 Patch 7 2018-07-26 9239799 N/A 6.2 Patch 7 3
ESXi 6.0 Patch 6 2017-11-09 6921384 N/A 6.2 Patch 6 3
ESXi 6.0 Express Patch 11 2017-10-05 6765062 N/A 6.2 Express Patch 11 3
ESXi 6.0 Patch 5 2017-06-06 5572656 N/A 6.2 Patch 5 3
ESXi 6.0 Express Patch 7c 2017-03-28 5251623 N/A 6.2 Express Patch 7c 3
ESXi 6.0 Express Patch 7a 2017-03-28 5224934 N/A 6.2 Express Patch 7a 3
ESXi 6.0 Update 3 2017-02-24 5050593 N/A 6.2 Update 3 3
ESXi 6.0 Patch 4 2016-11-22 4600944 N/A 6.2 Patch 4 3
ESXi 6.0 Express Patch 7 2016-10-17 4510822 N/A 6.2 Express Patch 7 3
ESXi 6.0 Patch 3 2016-08-04 4192238 N/A 6.2 Patch 3 3
ESXi 6.0 Express Patch 6 2016-05-12 3825889 N/A 6.2 Express Patch 6 3
ESXi 6.0 Update 2 2016-03-16 3620759 N/A 6.2 3
ESXi 6.0 Express Patch 5 2016-02-23 3568940 N/A 6.1 Express Patch 5 2
ESXi 6.0 Update 1b 2016-01-07 3380124 N/A 6.1 Update 1b 2
ESXi 6.0 Express Patch 4 2015-11-25 3247720 N/A 6.1 Express Patch 4 2
ESXi 6.0 U1a (Express Patch 3) 2015-10-06 3073146 N/A 6.1 U1a (Express Patch 3) 2
ESXi 6.0 U1 2015-09-10 3029758 N/A 6.1 2
ESXi 6.0.0b 2015-07-07 2809209 N/A 6.0.0b 2
ESXi 6.0 Express Patch 2 2015-05-14 2715440 N/A 6.0 Express Patch 2 2
ESXi 6.0 Express Patch 1 2015-04-09 2615704 2615979 6.0 Express Patch 1 2
ESXi 6.0 GA 2015-03-12 2494585 N/A 6.0 2
ESXi 5.5 Patch 10 2016-12-20 4722766 4761836 5.5 Patch 10 1
ESXi 5.5 Patch 9 2016-09-15 4345813 4362114 5.5 Patch 9 1
ESXi 5.5 Patch 8 2016-08-04 4179633 N/A 5.5 Patch 8 1
ESXi 5.5 Express Patch 10 2016-02-22 3568722 N/A 5.5 Express Patch 10 1
ESXi 5.5 Express Patch 9 2016-01-04 3343343 N/A 5.5 Express Patch 9 1
ESXi 5.5 Update 3b 2015-12-08 3248547 N/A 5.5 Update 3b 1
ESXi 5.5 Update 3a 2015-10-06 3116895 N/A 5.5 Update 3a 1
ESXi 5.5 Update 3 2015-09-16 3029944 N/A 5.5 Update 3 1
ESXi 5.5 Patch 5 re-release 2015-05-08 2718055 N/A 5.5 Patch 5 re-release 1
ESXi 5.5 Express Patch 7 2015-04-07 2638301 N/A 5.5 Express Patch 7 1
ESXi 5.5 Express Patch 6 2015-02-05 2456374 N/A 5.5 Express Patch 6 1
ESXi 5.5 Patch 4 2015-01-27 2403361 N/A 5.5 Patch 4 1
ESXi 5.5 Express Patch 5 2014-12-02 2302651 N/A 5.5 Express Patch 5 1
ESXi 5.5 Patch 3 2014-10-15 2143827 N/A 5.5 Patch 3 1
ESXi 5.5 Update 2 2014-09-09 2068190 N/A 5.5 Update 2 1
ESXi 5.5 Patch 2 2014-07-01 1892794 N/A 5.5 Patch 2 1
ESXi 5.5 Express Patch 4 2014-06-11 1881737 N/A 5.5 Express Patch 4 1
ESXi 5.5 Update 1a 2014-04-19 1746018 N/A 5.5 Update 1a 1
ESXi 5.5 Express Patch 3 2014-04-19 1746974 N/A 5.5 Express Patch 3 1
ESXi 5.5 Update 1 2014-03-11 1623387 N/A 5.5 Update 1 1
ESXi 5.5 Patch 1 2013-12-22 1474528 N/A 5.5 Patch 1 1
ESXi 5.5 GA 2013-09-22 1331820 N/A 5.5 1

As a reference, see:

Build numbers and versions of VMware vSAN (2150753) – This is a new KB post that went up on July 31, 2017 which provides the same information as above.

Build numbers and versions of VMware vCenter Server (2143838)

Build numbers and versions of VMware ESXi/ESX (2143832)

Understanding vSAN on-disk format versions (2145267)

 

 

 

 

 

Citrix & VSAN

There are many VMware and Citrix customers happily running Citrix XenApp and XenDesktop on VMware vSphere clusters with Virtual SAN enabled.

Citrix XenApp is fully supported on VSAN.

Citrix XenDesktop PVS is fully supported on VSAN.

Citrix XenDesktop MCS is still not supported on VSAN by Citrix at the time of this writing on October 7, 2016. Citrix has a fix that is in 7.8 and 7.9 already and customers have reported that the fix works, however Citrix claims the fix has not been qualified by them and thus is not supported. ETA for their official support is unclear at this point but is the responsibility of Citrix. If you are needing this feature, please reach out to Citrix to let them know.

Our friends at Dell Technologies (EMC/VCE) have tested XenApp, XenDesktop PVS and MCS on VxRail and have produced a report here:

Citrix XenDesktop 7.9 and VMware vSphere 6.0 with VCE VxRail Appliance
http://www.emc.com/collateral/technical-documentation/h15433-euc-citrix-xendesktop-vxrail-sg.pdf

In it they state “Citrix official support of MCS on VMware Virtual SAN is expected in a future release of XenDesktop. EMC tested this configuration and found no observable issues.

For the record, I’ve been a fan of Citrix since I first deployed Citrix WinView in my data center and remote sites back in 1994. Yes, I’m that old. I’m sure this will all get worked out.

Replays of Virtual SAN Sessions at VMworld 2016 That You Didn’t Want to Miss

What a great week last week at VMworld 2016. I had many good meetings with customers, participated in 3 breakout sessions, met up with some old friends and met some new ones. If you missed VMworld, well, then you missed a bunch of great sessions. There’s no way you could have seen them all, so, VMware has made them available here: http://www.vmworld.com/en/sessions/2016.html.

I participated in two sessions:

The first one was a customer panel discussion on Tuesday afternoon. I need to thank Glenn Brown from Stanley Black & Decker, Mike Caruso from Synergent, Tom Cronin from M&T Bank, and Andrew Schilling from Baystate Health who all did a fantastic job representing themselves, their companies, and their use of Virtual SAN. We had great interaction from the audience with lots of good questions. For a replay of the session check this out:

Four Unique Enterprise Customers Deployment of VMware Virtual SAN [STO7560]
Glen Brown
, System Engineer, Stanley Black and Decker
Michael Caruso, AVP Corporate Information Systems, Synergent
Tom Cronin, Sr. Staff Specialist – Platforms Engineering Group, M&T Bank
Frank Gesino, Senior Technical Account Manager, VMware
Andrew Schilling, Team Leader – IT Infrastructure, Baystate Health Inc.
Tuesday, Aug 30, 5:00 p.m. – 6:00 p.m.

The other session I was involved in was on Wednesday and repeated on Thursday. I had the good fortune to present with two VSAN Product Managers who are responsible for making VSAN great. Vahid Fereydounkolahi is responsible for driving new features into the VSAN product and Rakesh Radhakrishnan is responsible for making sure all the vendor hardware components are properly qualified for VSAN and for looking out into the future of new technologies like NVMe and RDMA to adopt into VSAN. For a replay of the two sessions check these out:

Peter Keilty, Office of the CTO, Americas Field – Storage and Availability, VMware, Inc.
Rakesh Radhakrishnan, Product Management & Strategy Leader, VMware
Wednesday, Aug 31, 2:00 p.m. – 3:00 p.m.
Vahid Fereydounkolahi kicked this one off discussion VSAN features, capabilities, and how it works, I took over in the middle to discuss Day 2 operations, and Rakesh Radhakrishnan finished it off discussing the Ready Node program as well as current and future flash and IO technology that VSAN incorporates or will incorporate.
Virtual SAN Technical Deep Dive and What’s New [STO8246R]

Thursday, Sep 01, 10:30 a.m. – 11:30 a.m.
Vahid wasn’t able to make this time so I kicked things off talking about VSAN features, capabilities, how it works, and Day 2 operations, and Rakesh Radhakrishnan finished it off discussing the Ready Node program as well as current and future flash and IO technology that VSAN incorporates or will incorporate.
Virtual SAN Technical Deep Dive and What’s New [STO8246R]

In my previous blog post I highlighted the sessions you wouldn’t want to miss. So here, I’ll provide the links to those sessions. A few either haven’t been uploaded yet or won’t because of legal or future looking reasons:

Christos Karamanolis is literally the brains behind VSAN since its inception and our chief visionary for Storage. If you want the whole picture wrapped up in a 1 hour session, this is it.
An Industry Roadmap: From storage to data management [STO7903]
Christos Karamanolis, VMware Fellow – CTO of Storage and Availability, VMware
Wednesday, Aug 31, 4:00 p.m. – 5:00 p.m.

Continue reading “Replays of Virtual SAN Sessions at VMworld 2016 That You Didn’t Want to Miss”

Virtual SAN Sessions You Won’t Want to Miss at VMworld 2016

Shameless self-promotion here. I’m very excited to be presenting 2 sessions at the upcoming VMworld 2016 in Las Vegas. So, of course I think you shouldn’t miss them. The first is a customer panel session that I’ll be hosting. I’ve worked with each of these customers who have had VSAN running production workloads for well over a year. Everything wasn’t always perfect, but, they continue to expand their usage of VSAN in their data centers. In two of the customers, they are now standardized on VSAN for any new workloads. These customers will provide an overview of their deployments, answer some of my questions, then take questions from the audience.

Four Unique Enterprise Customers Deployment of VMware Virtual SAN [STO7560]
Glen Brown, System Engineer, Stanley Black and Decker
Michael Caruso, AVP Corporate Information Systems, Synergent
Tom Cronin, Sr. Staff Specialist – Platforms Engineering Group, M&T Bank
Frank Gesino, Senior Technical Account Manager, VMware
Andrew Schilling, Team Leader – IT Infrastructure, Baystate Health Inc.
Tuesday, Aug 30, 5:00 p.m. – 6:00 p.m.

This VSAN Deep Dive session will cover features of the latest VSAN release, how they work, and some best practices for deploying VSAN. I’ll be presenting along with our lead VSAN Product Managers. This session will be held on two different days.

Virtual SAN Technical Deep Dive and What’s New [STO8246R]
Peter Keilty, Office of the CTO, Americas Field – Storage and Availability, VMware, Inc.
Rakesh Radhakrishnan, Product Management & Strategy Leader, VMware
Wednesday, Aug 31, 2:00 p.m. – 3:00 p.m.
Thursday, Sep 01, 10:30 a.m. – 11:30 a.m.

Other VSAN Sessions You Won’t Want to Miss

There are so many great VSAN sessions it’s hard to pick just a few. So, here are the ones I am most familiar with that I’m confident will be great. But that doesn’t mean that some of the others won’t be.

Christos Karamanolis is literally the brains behind VSAN since its inception and our chief visionary for Storage. If you want the whole picture wrapped up in a 1 hour session, this is it.

An Industry Roadmap: From storage to data management [STO7903]
Christos Karamanolis, VMware Fellow – CTO of Storage and Availability, VMware
Wednesday, Aug 31, 4:00 p.m. – 5:00 p.m.

Continue reading “Virtual SAN Sessions You Won’t Want to Miss at VMworld 2016”

VSAN In 3 Minutes Series

These are so cool I had to recognize them. If you are like me and would rather see things in action than read about them in a manual, then the VSAN In 3 Minutes Series is for you.

VSAN in 3 Minutes Series

Check the videos out. A big shout out to my colleague Greg Mulholland who does a great job putting these together.

VMware Virtual SAN at Storage Field Day 9 (SFD9) – Making Storage Great Again!

On Friday, March 18 I took the opportunity to watch the live Webcast of Storage Field Day 9. If you can carve our some time, I highly recommend this.

Tech Field Day‎@TechFieldDay
VMware Storage Presents at Storage Field Day 9

The panel of industry experts ask all the tough questions and the great VMware Storage team answers them all.

Storage Industry Experts VMware Virtual SAN Experts
  • Alex Galbraith @AlexGalbraith
  • Chris M Evans @ChrisMEvans
  • Dave Henry @DaveMHenry
  • Enrico Signoretti @ESignoretti
  • Howard Marks @DeepStorageNet
  • Justin Warren @JPWarren
  • Mark May @CincyStorage
  • Matthew Leib @MBLeib
  • Richard Arnold @3ParDude
  • Scott D. Lowe @OtherScottLowe
  • Vipin V.K. @VipinVK111
  • W. Curtis Preston @WCPreston
  • Yanbing Le @ybhighheels
  • Christos Karamanolis @XtosK
  • Rawlinson Rivera @PunchingClouds
  • Vahid Fereydouny @vahidfk
  • Gaetan Castelein @gcastelein1
  • Anita Kibunguchy @kibuanita

 

The ~2 hour presentation was broken up into easily consumable chunks. Here’s a breakdown or the recoded session:

VMware Virtual SAN Overview

In this Introduction, Yanbing Le, Senior Vice President and General Manager, Storage and Availability, discusses VMware’s company success, the state of the storage market, and the success of HCI market leading Virtual SAN in over 3000 customers.

What Is VMware Virtual SAN?

Christos Karamanolis, CTO, Storage and Availability BU, jumps into how Virtual SAN works, answers questions on the use of high endurance and commodity SSD, and how Virtual SAN service levels can be managed through VMware’s common control plane – Storage Policy Based Management.

VMware Virtual SAN 6.2 Features and Enhancements

Christos continues the discussion around VSAN features as they’ve progressed from the 1st generation Virtual SAN released in March 12, 2014 to the 2nd, 3rd, and now 4th generation Virtual SAN that was just released March 16, 2016. The discussion in this section focuses a lot on data protection features like stretched clustering and vSphere Replication. They dove deep into how vSphere Replication can deliver application consistent protection as well as a true 5 minute RPO based on the built in intelligent scheduler sending the data deltas within the 5 minute window, monitoring the SLAs, and alerting if they cannot be met due to network issues.

VMware Virtual SAN Space Efficiency

Deduplication, Compression, Distributed RAID 5 & 6 Erasure Coding are all now available to all flash Virtual SAN configurations. Christos provides the skinny on all these data reduction space efficiency features and how enabling these add very little overhead on the vSphere hosts. Rawlinson chimes on the automated way Virtual SAN can build the cluster of disks and disk groups that deliver the capacity for the shared VSAN datastore. These can certainly be built manually but VMware’s design goal is to make the storage system as automated as possible. The conversation moves to checksum and how Virtual SAN is protecting the integrity of data on disks.

VMware Virtual SAN Performance

OK, this part was incredible! Christos laid down the gauntlet, so to speak. He presented the data behind the testing that shows minimal impact on the hosts when enabling the space efficiency features. Also, he presents performance data for OLTP workloads, VDI, Oracle RACK, etc. All cards on the table here. I can’t begin to summarize, you’ll just need to watch.

VMware Virtual SAN Operational Model

Rawlinson Rivera takes over and does what he does best, throwing all caution to the wind and delivering live demonstrations. He showed the Virtual SAN Health Check and the new Virtual SAN Performance Monitoring and Capacity Management views built into the vSphere Web Client. Towards the end, Howard Marks asked about supporting future Intel NVMe capabilities and Christos’s response was that it’s safe to say VMware is working closely with Intel on ensuring the VMware storage stack can utilize the next generation devices. Virtual SAN already supports the Intel P3700 and P3600 NVMe devices.

This was such a great session I thought I’d promote it and make it easy to check it out. By the way, here’s Rawlinson wearing a special hat!

Make Storage Great Again