Friday 18 July 2008

SBS on Dell with PERC5i drive failure and DIY

We were invited into one of our neighbour's office to have a look at an SBS box that was not doing too good.

The setup was put together by the business owner with a little assistance from Dell.

By the time we were done, we had spent 10 hours at the location to get things setup correctly.

The server was headless, so we needed to walk back to the shop and pick up a small LCD monitor, a keyboard, and a mouse.

The business owner took care of plugging in the keyboard and mouse on the front of the server while we worked at connecting the monitor at the back.

Once logged into the server, we began the process of bringing the setup online.

The straw that broke the proverbial camels back: The owner had purchased a Linksys wireless N router and could not get it to setup properly. So, they called Linksys. Apparently Linksys had them disable DHCP on the server manually, enable it on the router, and get the wireless setup for access.

Keep in mind that none of the SBS 2003 R2 Standard key features were being used. Remote Web Workplace was not being used, SharePoint, and more. The R2 Technologies disk had not been installed as well, so no WSUS or integrated WSUS management or Exchange SP2.

The owner was connecting to the server via 3389 along with anyone else who needed bookkeeping and other application access. Do we all know what permission levels those users had?!? Yep: domain admin rights.

We essentially needed to run through our SBS 2003 R2 Setup Checklist (previous blog post) to make sure we made the necessary changes.

Once we had the server running stable, we needed to go to the client computers and work on getting them correctly connected to the SBS network. The ConnectComputer wizard was not used.

While working on the Setup Checklist, a trip back into the shop area where the little server cabinet held the 2650 caused a bit of a panic: Both the status light and the activity light were blinking yellow in unison on one of the two drives in the RAID 1 array.

Eriq Neale blogged about this situation last year:
The PERC 5/i has no audible alarm on it.

The business owner in this case, had no backups in place!

So, for who knows how long, their business was hanging by a thread ... their whole livelihood was on one hard disk that could have followed its partner.

Once this situation was discovered, we immediately grabbed our ShadowProtect I.T. Edition disk and a USB hard drive, rebooted the server into the ShadowProtect Recovery Environment, and created a full system snapshot.

Dell had us download a utility to run tests on the system which turned up all positive. This was a bit on the strange side. So, the technician had us pull the "defective" drive and reseat it. Sure enough, the array started to rebuild.

The Dell technician assured us that this was a "common" support call problem that they deal with. Apparently things like power outages, as explained by our Dell technician, can knock one of the drives, or even both, out of the RAID array.

This particular SBS server had an APC ES series 750VA for a UPS, but no protection on the ADSL telephone or Ethernet setup. The UPS was not tested while we were there.

This explanation by the Dell technician came as quite a surprise as the drive failures we have experienced have been extremely rare relative to the number of servers we have at production sites. And, those drive failures tended to be just that: Drive failures.

By the end of the day, we had a fully functionally SBS 2003 R2 Standard box, DNS and DHCP on a proper IP range and functional, Remote Web Workplace and all its features, desktops reasonably connected to the SBS network, and email Exchange integrated.

Some further training will be required to get the users up to the task of utilizing the full compliment of SBS features.

This situation really causes us to pause when it comes to the Do It Yourself (DIY) type of setup. We have encountered so many situations where the SBS setup was done by the business owner or a friend of the business owner where things were precariously close to total loss or there was no security on the Internet facing services.

There is a reason why a certified professional with actual experience with the SBS product should be driving it. And this example is but one where things are very clear that a Small Business Specialist company with experienced SBS technicians should be on the job.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists

*All Mac on SBS posts are posted on our in-house iMac via the Safari Web browser.

3 comments:

Anonymous said...

Also remember to update the firmware and driver on the 5i controller.

There is also firmware for both SATA and SAS hard drives themselves.

The Dell management console also had updates, as does the BIOS etc.

These yellow blinking RAID failures are not normal on our Dell boxes, so definitely do that firmware.

Great post though, shows exactly why SBS is not truly meant for the IT 'generalist,' even if Dell and/or Microsoft propose it is.

Have a great day!
Brian Williams
www.smbthoughts.com

Anonymous said...

I hope the business owner appreciates how close that bullet came! Nice post - I'm going to link to it and refer to it when I'm talking to prospective clients.

Philip Elder Cluster MVP said...

Brian,

Yes, we did setup the Dell utilities after the fact and ran through some hardware and software based updates.

Agreed ... more to come on that too.

Bruce,

I think he appreciates it ... given the number of different ways the analogy of walking over a cliff seemed to make it clear.

It can be really tough for a business owner to let go of the DIY mentality though ... and that may end up being the case here too.

Thanks for the comments guys.

Philip