This may not be the most sophisticated definition but Scale Up storage means you buy a box with a certain amount of storage capacity and performance then sometime later you add more storage (HDD or SSD) to it to increased both capacity and IOPS performance. Scale Out means you buy a box then sometime later add more boxes to increase both capacity and IOPS performance. VMware Virtual SAN is both. You have options to Scale Out as well as Scale Up. Lets investigate.
The minimum configuration for a VSAN is 3 hosts (boxes) with 1 SSD and 1 HDD in each. Lets say you started with that and stored enough data that you need more capacity. You have the option to add another host (box) with SSD and HDD. This would be a Scale Out approach to solving the problem.
But in your analysis you might find that you don’t need the extra host CPU and memory to support more Virtual Machines. So rather than adding another host, you can simply add more HDDs to the existing VSAN disk groups on the existing hosts for increased capacity and as a side benefit you get increased IOPS performance too. This would be a Scale Up approach to solving the capacity problem.
Lets say in your VSAN analysis (see here, here, and here for good info on VSAN analysis) you are seeing a lot of read cache misses. To improve performance caused by this problem you could increase the number of disk stripes for the VMs. However, this doesn’t necessarily address fixing the problem. Reducing the number of read cache misses might be better accomplished by adding more SSD caching capacity. Like the previous example, you have the option to add another host with SSD and HDD. But if you don’t need the extra host CPU and memory to support more Virtual Machines you can add more SSD to the existing hosts. For each host in the VSAN enabled cluster you’ll need to create a second disk group, or up to five disk groups, and add at least 1 SSD and 1 HHD to each group. This will increased IOPS performance (Scale Up for performance) and as a side benefit you get increased capacity.
A few VMware VSAN Beta Highlights & Best Practices to keep in mind:
- Min 1 SSD & 1 HDD per host, Max 1 SSD & 7 HDD per disk group, Max 5 disk groups per host
- Min 3 Hosts, Max 8 Hosts, Max 1 VSAN datastore per cluster (support for more hosts may increase in the future)
- Max vsanDatastore = (8 hosts * 5 disk groups * 7 disks * size of disks) = 280 * size of disks
- SSD capacity should be ~10% of your HDD capacity (e.g. 1 GB of SSD to every 10 GB of SAS/SATA).
VMware Virtual SAN Scales Out as your vSphere cluster Scales Out but don’t forget that you can Scale Up individual hosts as well for both increased capacity and IOPS performance.
Live Virtually 