Intel has a very thorough article that explains what happens when the workload data volume on a Storage Spaces Direct (S2D) Hyper-Converged Infrastructure (HCI) cluster starts to "spill over" to the capacity drives in a NVMe/SSD Cache with HDD Capacity for storage.
Essentially, any workload data that needs to be shuffled over to the hard disk layer will suffer a performance hit and suffer it big time.
In a setup where we would have either NVMe PCIe Add-in Cards (AiCs) or U.2 2.5" drives for cache and SATA SSDs for capacity the performance hit would not be as drastic but it would still be felt depending on workload IOPS demands.
So, what do we do to make sure we don't shortchange ourselves on the cache?
We baseline our intended workloads using Performance Monitor (PerfMon).
Here is a previous post that has an outline of what we do along with links to quite a few other posts we've done on the topic: Hyper-V Virtualization 101: Hardware and Performance
We always try to have the right amount of cache in place for the workloads of today but also with the workloads of tomorrow across the solution's lifetime.
S2D Cache Tip
TIP: When looking to set up a S2D cluster we suggest running with a higher count smaller volume cache drive set versus just two larger capacity drives.
Why?
For one, we get a lot more bandwidth/performance out of three or four cache devices versus two.
Secondly, in a 24 drive 2U chassis if we start off with four cache devices and lose one we still maintain a decent ratio of cache to capacity (1:6 with four versus 1:8 with three).
Here are some starting points based on a 2U S2D node setup we would look at putting into production.
- Example 1 - NVMe Cache and HDD Capacity
- 4x 400GB NVMe PCIe AiC
- 12x xTB HDD (some 2U platforms can do 16 3.5" drives)
- Example 2 - SATA SSD Cache and Capacity
- 4x 960GB Read/Write Endurance SATA SSD (Intel SSD D3-4610 as of this writing)
- 20x 960GB Light Endurance SATA SSD (Intel SSD D3-4510 as of this writing)
- Example 3 - Intel Optane AiC Cache and SATA SSD Capacity
- 4x 375GB Intel Optane P4800X AiC
- 24x 960GB Light Endurance SATA SSD (Intel SSD D3-4510 as of this
writing)
One thing to keep in mind when it comes to a 2U server with 12 front facing 3.5" drives along with four or more internally mounted 3.5" drives is their heat and available PCIe slots. Plus, the additional drives could also place a constraint on the processors that are able to be installed also due to thermal restrictions.
Intel Optane DC Persistent Memory
We are gearing up for a lab refresh when Intel releases the "R" code Intel Server Systems R2xxxWF series platforms hopefully sometime this year.
That's the platform Microsoft set an IOPS record with set up with S2D and Intel Optane DC persistent memory:
We have yet to see to see any type of compatibility matrix as far as the how/what/where Optane DC can be set up but one should be happening soon!
It should be noted that they will probably be frightfully expensive with the value seen in online transaction setups where every microsecond counts.
TIP: Excellent NVMe PCIe AiC for lab setups that are Power Loss Protected: Intel SSD 750 Series
Intel SSD 750 Series Power Loss Protection: YES
These SSDs can be found on most auction sites with some being new and most being used. Always ask for an Intel SSD Toolbox snip of the drive's wear indicators to make sure there is enough life left in the unit for the thrashing it would get in a S2D lab! :D
Acronym Refresher
Yeah, gotta love 'em! Being dyslexic has its challenges with them too. ;)
- IOPS: Inputs Outputs per Second
- AiC: Add-in Card
- PCIe: Peripheral Component Interconnect Express
- NVMe: Non-Volatile Memory Express
- SSD: Solid-State Drive
- HDD: Hard Disk Drive
- SATA: Serial ATA
- Intel DC: Data Centre (US: Center)
Thanks for reading!
Philip Elder
Microsoft High Availability MVP
MPECS Inc.
Co-Author: SBS 2008 Blueprint Book
www.s2d.rocks !
Our Web Site
Our Cloud Service
