Repeat After Me: SATA Does Not Belong In Servers Part Deux

For the last number of years we have stopped deploying servers with SATA drives installed.

There are so many reasons why we stopped but here are a few comparisons to SCSI/SAS:

• SATA does not have the ability to manage a high I/O workload
• SATA only offers a single inbound and outbound data port while SAS offers dual ports for redundant paths
• SATA does not have the health monitoring capabilities with SMART certainly not cutting it
• SATA does not offer anywhere near the capabilities and command set that SAS does for server related tasks, disk redundancy, disk sharing, and so much more

There is a reason why disk manufacturers have tacked on SAS controllers to SATA platter sets. These so-called NearLine drives offer all of the SAS goodness but with SATA capacities.

Here is the first public, that I know of, presentation from Microsoft on the _why_ SATA does not belong in servers.

• Microsoft KB2801713: Hyper-V storage: Caching layers and implications for consistency

To quote specifically:

1.Use the per I/O control mechanism that is known as Force Unit Access (FUA). This flag specifies that the drive should write the data to stable media storage before signaling (sic) is finished. Applications that have to do this make sure that data is stable on the disk issue FUA to make sure that data is not lost if a power failure occurs.

Server-class disk drives (SCSI and Fibre Channel) generally support the FUA flag. On commodity drives (ATA, SATA, and USB), FUA might not be honored. (emphasis added) This can potentially leave data in an inconsistent state unless the drive's write cache is disabled. Make sure that the disk subsystem handles FUA correctly if you depend on this mechanism

When listening to a discussion on this the above applies even when SATA disks are used in a properly configured RAID setup whether software (host-based) or hardware RAID on Chip.

In addition, if one were to be setting up a Storage Spaces cluster with multiple paths to the JBOD unit then one would be required to set it up with SAS based SSDs for the high performance storage tier. SATA will work in a single server and single enclosure lab like setting but _not_ in production.

We have had other posts on this topic that outline many other reasons for our decision to drop SATA in servers. The SATA category and the SAS category would be one place to start. :)

Philip Elder
Microsoft MVP
MPECS Inc.
Co-Author: SBS 2008 Blueprint Book

Chef de partie in the SMBKitchen
Find out more at
Third Tier: Enterprise Solutions for Small Business

Repeat after me: SATA does not belong in servers.

One of the very last servers we deployed with SATA drives had yet another failure in it.

There is a new Intel R2208GZ4GC 2U server in place with eight 600GB 10K SAS drives configured in a RAID 6 array already installed and waiting for tax season to slow down for them (they are an accounting firm).

Our client recently moved to a new location with the servers now located in a dedicated room in the basement. The little A/C unit in that room was a leftover from the previous occupant that we were not too sure about.

Well, the hot spare in this server, an Intel Server System SR1560SFHS with three 750GB Seagate ES series SATA drives, died about four months ago. Since the system was slated for replacement we left the remaining two in a RAID 1 array alone.

Well, that ended this morning with one of the drives in the pair having gone full stop. This was probably due to the fact that the temp in the room upon arrival this afternoon was close to 90F.

Someone had fired up the A/C unit without realizing that the hose that puts the heat outside was not connected to the back of the unit. Thus all of the heat it was trying to pull out plus its own heat yielded a very high temperature in that room.

Once the hose was affixed to the back of the unit the temperature started to come down.

So, here we are writing this blog post at 2216Hrs on a Wednesday evening after having logged in to check on the progress of the array rebuild and the above was what we saw.

The RAID controller is an Intel RAID Controller SRCSASRB with battery backup.

SATA does not belong in a server when it comes to spindled hard drives. This experience with the blind failure and the dismal rebuild times, during off hours no less, are definitely a part of it.

SAS/SCSI was designed and engineered to run in server environments. SATA was not.

The firmware tweaks that the hard drive vendors have introduced, along with the pretty much failed NCQ effort, to try and mimic a SAS setup within the SATA controller do not come close to the performance, longevity, and stability that SAS drives offer.

By the way, this goes for NearLine SAS drives as well. These drive types are SATA internals with SAS electronics slapped on to the external of the drive. There is a very good reason why the drives are called "NearLine". :)

The cost on 2.5" 10K SAS drives in 300GB and 600GB sizes have come down quite a bit in the last year. The 900GB 10K SAS drives are still relatively expensive per Gigabyte but provide an opportunity for a large aggregate of storage when needed.

Another way to look at it is this: How many RMA efforts have gone in to server setups with SATA drives in them? Compare that with the servers that have SAS setups. In our case, where we have lots of servers deployed, there is virtually no comparison. Over time the SAS drives have completely trumped the SATA drives in all aspects.

Even with 24x7x365 by 4 hour response times most vendors require time wasted on the phone prior to initiating that on-site visit to replace the failed drive. This time is expensive and to some extent a waste.

Oh, and one more thing: If going with parity in an array go RAID 6 with at least eight 10K spindles and make sure the RAID controller has either flash backed cache or a battery backup.

Storage is almost always the weakest point in a server both for hardware failures and I/O bottlenecks. Kill both. Use a wide array of eight spindles or more and make sure the drives 10K SAS.

The risk when using SATA is just not worth the "savings" IMNSHO (in my not so humble opinion).

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

Chef de partie in the SMBKitchen
Find out more at
www.thirdtier.net/enterprise-solutions-for-small-business/

Windows Live Writer

SAS versus SATA and Hardware RAID versus Software RAID

In the last few years we have made some changes to the server configurations that we either build and deploy or Tier 1 provides:

• We install SAS 10K or SAS 15K drives over SATA.
• Performance for one is vastly superior on the SAS drives.
• SAS drives are a lot more sensitive to bad sector behaviour and are better able to recover from bad data being tossed up.
• SAS drives use many forms of ECC which protects the integrity of the data.
• We install a hardware RAID controller with battery backup or SSD Cache over using the on board software RAID.
• The on board “RAID” is software driven. All RAID calculations are completed by the server’s CPU and in many cases require the software driver to rebuild or function properly – meaning we need to boot into the OS.
• We have had difficulties with on board software RAID recoveries and ShadowProtect due to driver issues.
• Hardware RAID on Chip with the battery backup or SSD Cache virtually eliminates the parity write cost of RAID 5 and greatly reduces that cost for RAID 6. SSD Cache almost renders the whole discussion moot as the most frequently requested data sits on the SSD.
• Hardware RAID controllers have the ability to mitigate the failure of a drive by keeping the server up. In our experience software RAID tends to freeze the box if a drive fails.
• Hardware RAID controllers tied to SAS drives have a much better chance of mitigating or eliminating the possibility of data corruption if sectors on an array member are dying.

Intel has a couple of resources for the above discussion as do many other RAID related vendors.

One Intel resource is the following Intel support page: Intel Server Products: Choosing between SAS vs. SATA Hard Disk for your Server RAID System.

There we find this grid that gives us a pretty good idea of some of the significant differences between SAS and SATA drives:

We are given an extensive explanation in the following document which is linked to below the above table: Intel – Enterprise-class versus Desktop-class Hard Drives (Link to PDF document download).

We put our server configurations through a lot of testing before we deploy them to client sites or within our own organization. We do this because we want to make absolutely sure that the server configuration we are going to deploy will meet the needs of our client over the life of the box which is about 36 months.

The extra cost for the hardware RAID controller, battery backup, and 15K SAS drives (not much of a cost difference between 300GB 15K SAS and Seagate Enterprise SATA these days) when taken over the life of the box (divide that cost by 36) is actually quite small relative to the performance, data protection, and overall storage stability benefits.

Tier 1 Caveat

Keep in mind that cost inferences mentioned here are for our in-house Intel server solution components. When it comes to Tier 1 the costs of some server components and server storage can be extremely high relative to components supplied in our solutions.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Intel 6 Series/C200 Series Chipset Design Error

It seems that there are some issues with the SATA ports on the new 6 series chipset based motherboards not holding up over time.

• PCWorld: Intel Finds Design Flaw in Sandy Bridge Chipset

We have been waiting for the new Intel DQ67xx Executive Series Micro-ATX and Mini-ATX boards for a while now. We had hoped to see the boards by the end of this month. It looks as though we will be waiting just a bit longer for them.

Now that this problem has been discovered, board makers that have a 6 Series chipset board on the market are going to have a serious problem on their hands if their production volume was ramped up and product has been sold.

Intel has a support page here:

• Intel Support: Chipsets: Alert for Intel® 6 Series Express Chipsets and Intel® Xeon® C200 Chipsets users

For other manufacturer’s products that use the 6 Series chipset, we will need to check their support site to see what sort of remedies they will have for us.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Seagate’s New Model Numbering Scheme

We just received a Seagate Partner e-mail that indicates that Seagate is changing the product model numbering scheme.

• Seagate Is Rolling Out New Model Numbers. Are You Ready?
• A registration form fronts the video which is hosted on a third party site.

As we can see, the model numbers have changed significantly relative to the current model number structures. In the current scheme The prefix ST3 means 3.5” drive, ST9 means 2.5” drive while the suffix AS is SATA, ES is Enterprise SATA, and SS is SAS. The numbers in between the prefix and suffix would be the size of the drive and then three numbers indicating model specifics.

A breakdown of the model number and what each means is this:

What we will end up with will actually be a simpler way to order Seagate drives. No more needing to know the various model numbers for whatever feature we want with a 500GB drive for example. With the new model number scheme the number will remain the same for the 320GB drive across all of is various feature sets while the model number will be specific to those feature sets.

It means that internally we can track all of the various feature sets for inventory purposes but our externally facing collateral will use the one model number to describe a 320GB drive. This makes for a much simpler presentation too.

Ultimately, when it comes to ordering product the new model numbering scheme will not make too much of a difference when searching our distribution channels for product. Now, we can search ST31000*SS to find any 1TB SAS drives, ST3500*NS to find any Enterprise SATA drives, and so on. After modifying our wildcard location to ST500* we will find any and all 500GB SKUs available to us. We can then use the same methodology as above to limit our search results to ST500MX* for Mission Critical 500GB drive models or ST*MX to bring up all Mission Critical products.

Note that not all distribution search engines support wildcard searches beyond dropping the asterisk at the end of some alpha characters.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer