Saturday, 29 June 2013

A Storage Spaces and Hyper-V Cluster Lab Guide

Here is a good start for a lab environment:

A set of DAS JBOD units can used with two small nodes and 2 SAS based HBAs per node to stand up a Storage Spaces cluster using the Server 2012 R2 bits. A couple of MicroServer Gen8 boxes would round out the Hyper-V side of it.

On this blog there are a lot of configurations discussed that utilize intelligent direct attached storage.

  • Basic setup
    • Intel Server System SR1695GPRX2AC pair with Promise VTrak E610sD or E310sD
    • This category search on the blog has a number of really good posts including configuration examples based on the SR1695GPRX2AC (blog category link)
      • This server unit has been our go-to for base configurations as it is an excellent and flexible platform
  • Advanced setup
    • Intel Server System R2208GZ4GC pair with the Promise VTrak E610sD or E310sD
  • All-Out Setup
    • All of the above plus LSI SAS 6160 Switch pair and Intel Modular Server with 3 nodes.

In the above setups the key is the intelligent storage providing mitigation services to the SAS HBAs and OS access to the central storage.

With the 2012 R2 bits we are going to put together a redundant JBOD setup for a Storage Spaces cluster. This is the next direction we are delving into as we can put together a small SS cluster for a very reasonable cost.

Today, we are working on the following (similar to David Ziembicki’s setup) setup for clustered Storage Spaces:

  • Basic
    • Intel Server System R1208JP4OC with pair of SAS HBAs (RS25GB008) (2 nodes)
      • 32GB of ECC per node to start
    • Intel Storage System JBOD2224S2D2 JBOD2224S2DP Intel JBOD units (2 units)
      • JBOD is dual expander and dual port backplane
      • Seagate Savvio SAS drives are dual port
    • 1m SAS Cables (4)
    • Windows Server 2012 R2 beta – Storage Spaces Cluster Setup
    • Intel Server System R2208GZ4GC pair for Hyper-V nodes (we have had these in our lab for a year or so now).
      • 64GB to 128GB of ECC
    • Windows Server 2012 R2 beta or RTM – Hyper-V Nodes
  • Advanced
    • Add a pair of 8-Port or 12-Port NETGEAR 10Gbit Ethernet switches
      • Ports on each NIC would be split between switches for redundancy
    • Add a pair of Dual-Port 10Gbit PCIe and/or I/O Module NICs to each node
      • 10Gbit Ethernet would SMBv3 Storage Spaces located VHDX
      • 10Gbit Ethernet would be for Live Migration Network
    • LSI SAS Switches (we have a pair of these in our lab setting)
    • Additional Intel JBOD units to test switches and scaling storage out

Using David Ziembicki’s setup though one would be able to start at the base level and put together a similar configuration on a budget.

An HP MicroServer Gen8 would make an excellent platform for testing as they are relatively inexpensive and have pretty close to the full Intel virtualization Acceleration feature set.

Note that the Sans Digital MS28X listed in his blog post splits drives 0-3 and 4-7 between the two available external SAS connections. That means that there is no ability to use this storage unit without an LSI SAS 6160 Switch pair (Sans Digital MS28X Quick Installation Guide PDF)!

However, the Sans Digital MS8X6 unit does support redundancy and therefore they could be used to test Storage Spaces clustering configurations (Sans Digital MS8X6 Quick Installation Guide PDF).

Of course, for the added functionality there will be an extra cost involved, however one could drop the LSI SAS Switch for a set of these units for about the cost of the original MS28X plus SAS Switch!

  • Storage Spaces Cluster
    • Storage Spaces Node
      • Intel Xeon E3-1230
      • Intel Server Board S1200BTLSH
      • 16GB ECC
      • Intel Integrated RAID RMS2AF040
      • 120GB Intel 320 Series SSD (or small 10K SAS) RAID 1 pair for host OS
      • Quad-Port Intel Gigabit Server NIC PCIe
      • Intel certified chassis (whether Intel or other)
    • Storage
      • Sans Digital MS8X6
        • 300GB 15K 3.5" SAS drives can be found for a good deal today
    • Hyper-V Node
      • Intel Xeon E3-1230
      • Intel Server Board S1200BTLSH
      • 32GB ECC
      • Intel Integrated RAID RMS2AF040
      • 120GB Intel 320 Series SSD (or small 10K SAS) RAID 1 pair for host OS
      • Quad-Port Intel Gigabit Server NIC PCIe
      • Intel certified chassis (whether Intel or other)
      • Add Intel RMM4 for full KVM over IP
      • Add Dual-Port 10Gbit Intel Ethernet for SMBv3 and Live Migration Networks
      • Add Intel Storage Systems JBOD2224S2DP at a later date for full SAS Dual Port Redundancy

There are so many different ways to go about this.

The main thing is to start small and work one’s way up to a full scale server grade lab as the jobs come in! That’s how we built our own lab systems up and how we built up the knowledgebase and experience!

EDIT: Oops, Star Wars on the mind. Intel Storage Systems part number should be JBOD2224S2DP (I had JBOD2224S2D2 above!). :)

Philip Elder
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

Chef de partie in the SMBKitchen
Find out more at

Windows Live Writer


Ken Wallewein said...

Hey Phil, I always find your postings really interesting.

Quick question: for a minimum "lab" setup to test basic storage spaces clustering, do I really need a JBOD/DAS enclosure and SAS switch?

Seems to me I should be able to get SS and clustering working with just direct connection between the SAS HBAs and one or two SAS drives. Why bother with SAS switches and JBODs for only one or two hard drives?

-- Ken

Philip Elder Cluster MVP said...


You do need a JBOD.

Minimum would be two servers with SAS HBAs and a JBOD that would support both connected to the same disk group via SAS (2 ports).

With that you could build an asymmetric cluster based on Storage Spaces. In this configuration we set up a small 1.5GB Space for the Quorum/Witness after standing up the cluster. Then we split the rest of the storage up 50/50 with each portion attached to one node.

Not perfect, as that means that only one node deals with I/O for its Space but it can be done quite inexpensively.