Category: VASA

vSAN and Data-At-Rest Encryption – Rebooted (i.e. Part 2)

by pgkeilty, posted in HCI, Hyper-converged, SDDC, SDS, Software Defined Storage, SPBM, SSD, Uncategorized, VAIO Filters, VASA, Virtual SAN, Virtual SAN 6, VMware, VSAN, vSphere 6, VxRACK, VxRail

 

Encryption is here, now shipping with vSphere 6.5.

I first wrote about vSAN and Encryption here:

Virtual SAN and Data-At-Rest Encryption – https://livevirtually.net/2015/10/21/virtual-san-and-data-at-rest-encryption/

At the time, I knew what was coming but couldn’t say. Also, the vSAN team had plans that changed. So, let’s set the record straight.

vSAN

  • Does not support Self Encrypting Drives (SEDs) with encryption enabled.
  • Does not support controller based encryption.
  • Supports 3rd party software based encryption solutions like HyTrust DataControl and Dell EMC Cloud Link.
  • Supports the VMware VM Encryption released with vSphere 6.5
  • Will support its own VMware vSAN Encryption in a future release.

At VMworld 2016 in Barcelona VMware announced vSphere 6.5 and with it, VM Encryption. In the past, VMware relied on 3rd party encryption solutions, but now, VMware has its own. For more details, check out:

What’s new in vSphere 6.5: Security – https://blogs.vmware.com/vsphere/2016/10/whats-new-in-vsphere-6-5-security.html

In this, Mike Foley briefly highlights a few advantages of VM Encryption. Stay tuned for more from him on this topic.

In addition to what Mike highlighted, VM encryption is implemented using VAIO Filters, can be enabled per VM object (vmdk), will encrypt VM data no matter what storage solution is implemented (e.g. object, file, block using vendors like VMware vSAN, Dell Technologies, NetApp, IBM, HDS, etc.), and satisfies data-in-flight and data-at-rest encryption. The solution does not require SED’s so it works with all the commodity HDD, SSD, PCIe, and NVMe devices and integrates with several third party Key Management solutions. Since VM Encryption is set via policy, that policy could extended across to public clouds like Cloud Foundation on IBM SoftLayer, VMware Cloud on AWS, VMware vCloud Air or to any vCloud Air Network partner. This is great because your VM’s could live in the cloud but you will own and control the encryption keys. And you can use different keys for different VM’s.

At VMworld 2016 in Las Vegas VMware announced the upcoming vSAN Beta. For more details see:

Virtual SAN Beta – Register Today! – https://blogs.vmware.com/virtualblocks/2016/09/07/virtual-san-beta-register-today/

This vSAN Beta includes vSAN encryption targeted for a future release of vSphere. vSAN Encryption will satisfy data-at-rest encryption. You might ask why vSAN Encryption would be necessary if vSphere has VM Encryption? I will say that you should always look to use VM Encryption first. The one downside to VM Encryption is that since the VM’s data is encrypted as soon as it leaves the VM and hits the ESXi kernel, each block is unique, so no matter what storage system that data goes to (e.g. VMware vSAN, Dell Technologies, NetApp, IBM, HDS, etc.) that block can’t be deduped or compressed. The benefit of vSAN encryption will be that the encryption will be done at the vSAN level. Data will be send to the vSAN cache and encrypted at that tier. When it is later destaged, it will be decrypted, deduped, compressed, and encrypted when its written to the capacity tier. This satisfies the data-at-rest encryption requirements but not data-in-flight. It does allow you to take advantage of the vSAN dedupe and compression data services and it’s one key for the entire vSAN datastore.

It should be noted that both solutions will require a 3rd party Key Management Server (KMS) and the same one can be used for both VM Encryption and vSAN Encryption. The KMS must support the Key Management Interoperability Protocol (KMIP) 1.1 standard. There are many that do and VMware has tested a lot of them. We’ll soon be publishing a list, but for now, check with your KMS vendor or your VMware SE for details.

VMware is all about customer choice. So, we offer a number of software based encryption options depending on your requirements.

It’s worth restating that VM Encryption should be the standard for software based encryption for VM’s. After reviewing vSAN Encryption, some may choose it instead to go with vSAN encryption if they want to take advantage of deduplication and compression. Duncan Epping provides a little more detail here:

The difference between VM Encryption in vSphere 6.5 and vSAN encryption – http://www.yellow-bricks.com/2016/11/07/the-difference-between-vm-encryption-in-vsphere-6-5-and-vsan-encryption/

 

In summary:

  1. Use VM Encryption for Hybrid vSAN clusters
  2. Use VM Encryption on All-Flash if storage efficiency (dedupe/compression) is not critical
  3. Wait for vSAN native software data at rest encryption if you must have dedupe/compression on All-Flash

 

VMware Storage Technology Names & Acronyms

by pgkeilty, posted in SDS, Software Defined Storage, SPBM, SRM, VAIO Filters, VASA, Virtual SAN, VSAN, VVols
  • vSAN = VMware’s Software Defined Storage Solution formerly known as Virtual SAN or VSAN. Now the only acceptible name is “vSAN” with the little “v”.
  • SPBM = Storage Policy Based Management
  • VASA = vSphere API’s for Storage Awareness
  • VVol = Virtual Volume
  • PE = Protocol Endpoint
  • VAAI = vSphere API’s for Array Integration
  • VAIO Filtering = vSphere API’s for IO Filtering
  • VR = vSphere Replication
  • SRM = Site Recovery Manager
  • VDP = vSphere Data Protection
  • vFRC = vSphere Flash Read Cache
  • VSA = vSphere Storage Appliance (end of life)
  • VMFS = Virtual Machine File System
  • SvMotion = Storage vMotion
  • XvMotion – Across Host, Cluster, vCenter vMotion (without shared storage)
  • SDRS = Storage Distributed Resource Scheduler
  • SIOC = Storage Input Output Control
  • MPIO = Multi Path Input Output

What Makes VSAN Different?

by pgkeilty, posted in SDS, Software Defined Storage, SPBM, VASA, Virtual SAN, Virtual SAN 6, VMware, VSAN, VVols

I had a question today asking how VMware Virtual SAN (VSAN) compares to XYZ company. There are over a dozen virtual machine software based solutions that leverage the local disks in ESXi hosts to present storage back to the hosts in the vSphere cluster. Those solutions require a vSphere cluster to be created then their virtual machine must be installed on every host to handle the storage services. Some are more efficient at this than others but there is always level of effort to “build-your-own” storage on top of the vSphere cluster and those virtual machines can take up significant host resources to deliver on the storage services they offer. So converged infrastructure itself is nothing new or unique. Its how it’s done that is important.

Here’s what makes VMware Virtual SAN (VSAN) different:

  • VSAN is the ONLY software defined storage solution that is embedded into the ESXi hypervisor making it the most efficient data path for performance.  VM’s send their data through the hypervisor right to disk, there’s no middle man.  In addition, VSAN is the most efficient in its use of the host resources to deliver on the storage service. VSAN is designed to take up no more than 10% of the host CPU and memory resources and testing with vSphere 6 show significantly less impact than that. Since VSAN is not a VM on top of the hypervisor, it has this distinct advantage. This was a positive tradeoff for the fact that VSAN is a VMware vSphere only solution.
  • Being built in also makes it simple and easy to manage. There is no VSAN install, it is simply enabled as a feature of the hypervisor by clicking a check box. When enabled, VSAN will collect all the local disks on all the hosts and create the VSAN Datastore. Bear in mind, the server IO controller and disks must be in place and networking configurations must be completed to make sure VSAN will work when you click that check box.

VSAN Checkbox

  • VSAN is fully integrated into VMware Storage Policy Based Management (SPBM), VASA, and VVOLs. When that check box is clicked, the VSAN datastore is created and it’s VASA provider is registered with vCenter to expose it’s capabilities to SPBM. This allows different policy to be created so the same pool of capacity can deliver different service levels to different VM’s based on performance, availability, and protection. When VM’s are attached to a policy service level, their VM objects get created on the VSAN datastore in the form of Virtual Volume (VVOL) objects. VSAN further breaks these VVOL objects up into components to deliver on the defined protection and performance service levels.

VSAN and SPBM

  • VSAN deals with data protection at the software layer so it doesn’t suffer the performance and capacity penalty of hardware RAID. Different “tiers” of protection can be defined by policy and set for different VM’s using the same pool of disks in the VSAN datastore.  Numbers of Failures to Tolerate settings determine how many data replicas are written to different hosts to deliver the desired protection level for VM’s.
  • VSAN now supports a feature called “Rack Diversity”.  I wrote about the benefits here.  This brings Software Defined Self Healing with Failure Domains.  Hosts in the same rack can be placed into the same fault domain so that if an entire rack is lost then data remains available since another replica copy of the data resides on another host in another rack.

VSAN Rack Diversity

  • VSAN is a hybrid storage solution leveraging SSD as cache to accelerate both reads and writes and low cost high capacity hard disks to persist the data. This results in near All-Flash array performance at a fraction of the cost. With vSphere 6 along with Virtual SAN 6, an All-Flash VSAN is supported delivering extreme performance.

VMware Virtual SAN™ 6.0 Performance

  • VSAN is one of the few software based storage solutions that can leverage the in host SSD/Flash for Read AND Write caching. There are many solutions that can leverage in host SSD/Flash for read caching. Write back caching is more difficult to implement but VSAN does it while maintaining high availability of those writes across the cluster.

All other converged software based storage solutions require running a Virtual Machine on top of ESXi. So all VM’s have to go through their own IO path, through the hypervisor, then through that single VM IO path, then back through the hypervisor, then to the disks. In some cases, the disks themselves need to be setup with a hardware RAID configuration then their VM solution implements software RAID in addition to the underlying hardware RAID paying a double performance and capacity penalty. Each of these VM’s take on additional host CPU and Memory. Some require 2-4 vCPU’s and 16GB or more of RAM. And some are limited to the number of nodes they can scale to and how much total capacity can be supported. Again, some solutions are more efficient and scalable than others so do the homework and ask the right questions when comparing. Finally, most don’t support VMware’s Storage Policy Based Management which is the VMware framework for managing all vSphere storage going forward.

VMware’s vision for Virtual SAN is that it be the best storage solution for Virtual Machines. With the release of vSphere 6 and Virtual SAN 6, VMware is closer to that vision. There are many software defined storage choices out there.  Hopefully this helps in that decision making process.