The ONLY Place To Snapshot A DC VM is in the Lab Right?

And that is only to work through all of the KBs that follow in this blog post to gain AD recovery skills right?

On one of the lists we are a part of there is an active discussion going on about having a second DC on a smaller network for "redundancy" purposes.

When it comes to an SBS Standard based network there are some caveats for that second DC whether it is virtual or physical:

• MPECS Inc. Blog: SBS Disaster Recovery with a Second DC Caveats

As we have learned in our past recovery situations that second DC can actually be a hindrance instead of a help when there is a need to restore Small Business Server Standard.

Virtual DCs

Now, when everything is virtualized one may be tempted to snapshot a DC prior to making any changes to provide a "fall-back" if things go sideways.

• Microsoft KB: 888794: Things to consider when you host Active Directory domain controllers in virtual hosting environments

Some things to consider via the mentioned KB:

• DC should remain running continuously.
• Do not pause the DC VM for long periods of time.
• Problems may happen.
• System State backups are critical but have a shelf life.
• In multi-DC environments daily DC System State backups of at least two (2) DCs should be the norm.

When a DC is recovered back from a snapshot the following KB may be applicable:

• Microsoft KB: 875495: How to detect and recover from a USN rollback in Windows Server 2003, Windows Server 2008, and Windows Server 2008 R2
• Implications of rolling back a DC and the processes to possibly recover.

Now, take all of the above and read the following:

• Ask the Directory Services Team: DC's and VM's - Avoiding the Do-Over
• Mark Ramey goes over the possibilities when it comes to recovering DCs with some excellent visuals.

The point we are making?

It's okay to have a DC or three in a virtual lab that are used to break and tear apart then step back using a snapshot to then run through the above processes to figure out the recovery path of a restored-from-snapshot DC VM.

However, in a production environment, whether it be our own or our client's location, DC VM snapshots should _never_ be used. Period.

• Microsoft TechNet: Running Domain Controllers in Hyper-V (Deployment Considerations for Virtualized Domain Controllers)

A good backup, that is one that has been fully recovered to bare metal and/or hypervisor, along with a System State backup, are the only way to go. Then, being familiar with the above processes and caveats to having multiple DCs in a production environment is a must.

WS Backup & StorageCraft ShadowProtect

All of our current, as of Windows Server 2008 R2, smaller client networks with the exception of those running on Hyper-V failover clusters (Win2K8 R2) are running a single DC.

In most cases that DC is Small Business Server 2008/2011 Standard.

Why?

Because we test our client's backups on a quarterly basis as part of our ongoing services we provide them.

Test restoring our client's systems on a regular basis gives us full confidence in our ability to restore their single SBS/DC using ShadowProtect and in some cases the native Windows Server Backup.

Introducing a second DC into the mix, in the case of SBS networks, brings about caveats that we need not deal with (see first blog post link) especially when times may be stressful already.

The key to being confident in a single DC environment is in the backup solution set.

To repeat: Confidence in our backup solution is the key to our deploying a single DC solution.

If we are not versed in restoring the backups we deploy at our client sites, at that on a regular basis, then how can we have the confidence to recommend a single DC solution to our clients? If we don't restore our client's backups how will we be aware of what is needed if things really go sideways and a restore is required?

We _are_ confident in our backup solutions built upon Windows Server Backup and now especially on StorageCraft's ShadowProtect Version 4. SP v4 has proven that once again we will be deploying ShadowProtect at all of our client sites as the Hyper-V restore throughput problems we saw in the past are no more.

ShadowProtect's Hardware Independent Restore feature is also a must for P2V and V2V restore situations even between Hypervisor versions.

• MPECS Inc. Blog: ShadowProtect Restore to Hyper-V VM Tips

In the end, it is our preference to keep a single DC in our small to medium solution sets. KISS is our preference. And, a single DC with no snapshots taken follows that line of simplicity. Plus, recovery becomes that much simpler.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Intel Server Board S3420GPRX At Full Capacity – 32GB RAM – A Great Virtualization Platform

After deliberating over which set of 8GB sticks of RAM to purchase to outfit our single socket virtualization platforms with a full 32GB compliment we decided to go with the following part:

• Kingston Value RAM: KVR1066D3Q8R7S/8Gi

There is also a 1333MHz version of the Intel certified RAM from Kingston but there were some questions around whether that particular model would back itself down to 800MHz which is where a memory channel will go speed wise when two quad ranked sticks of RAM are installed.

The screenshot above is via an Intel Remote Management Module.

• Intel Server Board S3420GPRX with I/O Shield (previous blog post)
• Intel Xeon Processor X3470 at 2.93GHz
• 32GB Kingston ECC 1066MHz
• Intel RS2BL080 RAID + Battery Backup
• 6x 300GB 15K Seagate SAS (5x RAID 5 + 1x Hot Spare)
• Intel Server Chassis SC5650DP with hot swap backplane.

The system is fairly quiet and is quite quick. We are expecting this configuration to more than meet our client’s needs.

One thing to note when setting up standalone virtualization boxes with the full GUI OS installed is to change the power settings to High Performance.

This particular configuration will be running:

• Windows Small Business Server 2011 Standard.
• 2x vCPUs, 16GB RAM
• Windows Server 2008 R2 Standard in RDS/RemoteApp mode.
• 2x vCPUs, 4GB RAM
• Windows Server 2003 R2 Standard (P2V) running legacy LoB.
• 1x vCPU, 2GB RAM
• Windows 7 Enterprise x64 (one to four or five desktops).
• 2x vCPUs, 1.5GB-2GB RAM each.

Our client is a small non-profit with 8-11 seats depending on the time of year. We are quite confident that they will be happy with their new setup once we have migrated them from their current SBS 2003 R2 Premium.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Windows 2008 R2 Web – .NET Install Error –You must use the Role Management Tool

We are trying to get .NET 3.5 SP1 installed on a Windows Web Server 2008 R2 clustered VM.

When we tried to run the full version of the install file we received the following error:

Microsoft .NET Framework 3.5 SP1 Setup

You must use the Role Management Tool to install or configure Microsoft .NET Framework 3.5 SP1.

However, when we open the Add Role Services wizard on the server .NET 3.5 is nowhere to be seen.

A quick search turned up the answer on an MSDN blog:

• MSDN Blog: The Way I See It: You must use the Role Management Tool to install or configure Microsoft .NET Framework 3.5.

Vijay points out that .NET 3.5 will be found under the Features install.

Indeed, there it was:

We selected the root check box and away we went.

Note that once .NET v3.5 is installed some of the updates that will follow will require reboots of the server.

Our Web server VM is now ready for an LoB that required .NET 3.5.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Intel Modular Server – Storage For the Hyper-V Server 2008 R2 OSs

As part of the ongoing set up process for our Hyper-V cluster that will be running three server OSs and five desktop OSs, we need to set aside a bit of storage space for the actual Compute Module Hyper-V Server 2008 R2 OS:

So, what we have done here is created a Storage Group called HyperV_OS based on drives 1 and 2 which are both Seagate Savvio 146GB 15K.2 SAS 6.0Gb/s drives.

All of the drives populating the 13 drive bays are the same Seagate model.

Click on either one of the drives in slot 1 or 2 and we get the following:

The information breakdown is quite comprehensive as we drill down into the various menus in the Modular Server Control console.

The plan for the balance of the drives is to have slots 3 through 6 set up as a dedicated RAID 10 array for Small Business Server 2008 and the balance of the drives in slots 8 through 13 set up as a RAID 10 for the other two servers and the five desktop OSs.

An Intel SR1630HGP 1U Intel Xeon Processor 3400 Series server with Windows Server 2008 R2 Standard x64 will be a Read-Only DC providing authentication fall back if there is a full shutdown of all systems. The Hyper-V Role is also installed on this box.

Microsoft’s System Center Virtual Machine Manager will be installed on the 1U to facilitate managing the VMs.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Server 2008 R2 MMC – Windows Server Backup Error Not Compatible

Since we do not have our network migrated to SBS 2008 yet, we need to have a Server 2008 full install box on the network for some specific Server 2008 management needs such as configuring Volume Shadow Copies and Windows Backup on our many Server Core installations.

Both features cannot be managed from a desktop OS even with the RSAT installed.

For now, that means running a full version as a VM on one of our Hyper-V servers:

The partition scheme shown above was after clicking OK in the previous warning pop-up. We ended up with the 60GB primary that the OS was going into, the 15GB we left over for the swap file partition, and then the 100MB partition that Windows Server 2008 R2 setup created.

We have seen Windows 7 installations do the same (Bing Search). For now, we will leave it be.

Once installed, we will set up an MMC that contains the Windows Firewall with Advanced Security, Computer, and Windows Server Backup snap-ins for each of the Server Core based servers we have.

Or so we thought . . .

Windows Server Backup (Server Name)

An internal error has occurred in the backup engine or the computer that you are connected to remotely is running a version of backup application that is not compatible with the version on your local computer. The local and remote computers must be running the same version of backup application for you to manage backups remotely.

Oops. So, we fired up a new VM and are installing Windows Server 2008 RTM. Once installed, we will service pack the VM up to SP2 and force WSUS updates to get things on the level with our Server Core installs. From there we will be able to manage them.

When we have a bit more time, we will migrate our VMs across our various Hyper-V servers so that we can begin updating them to Windows Server 2008 R2 Server Core versions.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*All Mac on SBS posts will not be written on a Mac until we replace our now missing iMac! (previous blog post)

Windows Live Writer

ShadowProtect Image Recovery into a Hyper-V VM

We need to get to images restored into VMs running on Hyper-V for the test Swing Migration we are doing for the messed up “Professional” SBS in this post.

The deeper we are digging into this the more and more it may be a good idea to start fresh and work the client workstations from there.

But, we have committed to the business owners to run through the Swing test anyway, so we are well into it now.

The existing SBS was imaged using ShadowProtect and has been restored to one of our lab servers using the Hardware Independent Restore feature. We cleaned up the IBM XServe hardware pieces in the Device Manager and made sure that the lab server hardware drivers were installed and stable.

• Cleanup methodology blog post: SBS & ShadowProtect - Some Hardware Independent Restore considerations.

We are running a portion of the Swing virtually with the NewSBS ending up on the same lab server we have the restored SBS on now which is a Swing Off and Swing On again.

On this particular SBS network there is a Terminal Server. So, we took an image of it as well as one critical workstation. We are restoring both images to VMs running on Hyper-V Server 2008 (yes, we got it up and running though with no add-in RAID controller).

When we went to boot from the ShadowProtect I.T. Edition ISO in the VM, the WinRE would choke on the networking portion of the ShadowProtect startup routine. We ended up needing to uninstall the standard Hyper-V NIC and use a Legacy NIC to get things rolling. We were using the Windows Vista WinRE version too.

We placed the two ShadowProtect images on a network share to run the restore process in the Hyper-V based VMs using drive mapping feature in the ShadowProtect recovery environment.

When it comes to restoring an encrypted ShadowProtect image that is password protected using AES 128bit security, CPU crunching capabilities and GHz speed sure make a huge difference on the restore times.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*All Mac on SBS posts will not be written on a Mac until we replace our now missing iMac! (previous blog post)

Windows Live Writer

Hyper-V – Installing Windows 2000 Server

We have need for a temporary legacy setup here at the shop.

We used a Windows Server 2000 Standard SP2 CD ISO to install the base OS.

We have an ISO based DVD with every conceivable Microsoft service pack and needed critical update on it that we use to service our VMs. We needed to mount that ISO and update to Service Pack 4 before we could get the Hyper-V Integration Services installed.

Once we had the Integration Services installed, we were able to configure the new VM on our SBS network.

After a few minutes, we saw this:

Windows 2000 Server with 72 Updates Waiting

WSUS picked up on all of the needed updates and the downloads began!

One of the updates in that list was Update Rollup 1 for Windows 2000 SP4 and known issues (KB891861). In the description for the Rollup, it was indicated that the update was a pre-package of previous Windows 2000 updates.

So, we ran the Rollup first to see what was left over. It turns out that there were still 68 updates waiting for us after what was an extended reboot and subsequent logon procedure.

Note that the logon procedure took quite a bit of time after the Rollup was applied so there will be a need to be patient.

When the new Win2K VM was connected to the domain and its needed services and LoB (Line of Business) applications were installed, the VM ran flawlessly.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*All Mac on SBS posts will not be written on a Mac until we replace our now missing iMac!

Windows Live Writer

Hyper-V on Server Core Rehash - Scratch to Production and Hardware Considerations

Server Core + Hyper-V after the fact: Build it and they will come! :)

Blog posts:

• Server Core - From Scratch to Hyper-V Production.
• NetSH reference: Server Core - Firewall NetSH Command Line Reference.
• Simple scripted updates for Server Core: Server Core - Updates are available + How to manually update.

When working with the Server Core installation on a domain, it is a good idea to add the principle domain user to the local domain admins group. But only that user.

For other users that will manage only certain VMs on the Hyper-V box, there is AzMan. This is just a teaser since the Tech Days presentation went through AzMan and Hyper-V rather quickly, we will need more time to delve into the methodology. Anyone already familiar please feel free to drop some hints/suggestions/tidbits into the comments! ;)

Once the above steps have been completed, Hyper-V Manager on Vista SP1 or in Server 2008 (either Hyper-V Role enabled or management tool) will be able to connect.

Make sure to enable the remote advanced firewall management on the Server Core box for all profiles! If all else fails, at least we can still work the firewall rules via a GUI.

When it comes to the Hyper-V on Server Core hardware, make sure the box has at least 2 network cards! One NIC needs to be tied to the internal domain/workgroup and needs to be dedicated to management use only.

The second NIC, or any additional ones, will be for the Hyper-V virtual switch. Once the second NIC is bound via the Virtual Network Manager in Hyper-V, it is okay to disable the NIC in the Device Manager of the Hyper-V Server Core box. The Hyper-V Switch will continue to function since it is riding directly on the NIC hardware.

Also, when it comes to the hardware configuration and the number of cores enabled for a VM, there may not be a logical progression for core assignment. Here is an example (CPU focus only):

• Hardware has a pair of E5440 Xeons (8 Cores).
• 1 Virtual CPU: Vista VM
• 1 Virtual CPU: Vista VM
• 1 Virtual CPU: XP Pro VM
• 1 Virtual CPU: XP Pro VM
• 2 Virtual CPUs: Server 2008 VM + SQL 2008 Standard
• 2 Virtual CPUs: Server 2008 VM + Exchange 2007 Standard

The logical thought progression would be to settle each VM on a core for the singles and a pair of cores for the VMs that have two virtual CPUs. Eight virtual CPUs demanded by the VMs with 8 cores available. But, that is not how the actual CPU tasks are served up across the cores. The above example may run just fine for a small shop with relatively low CPU to VM usage. But, as the VMs demand more processor horsepower, the more difficult it becomes to divvy up a pair of cores for the VMs with 2 virtual CPUs.

Now, take the above configuration and tweak it a bit:

• Hardware has a pair of E5440 Xeons (8 Cores).
• 2 Virtual CPUs: Vista VM
• 2 Virtual CPUs: Vista VM
• 4 Virtual CPUs: Server 2008 VM + SQL 2008 Standard
• 4 Virtual CPUs: Server 2008 VM + Exchange 2007 Standard

We dropped the XP Pro single CPU VMs by moving them to another Hyper-V box because we knew the load on this particular Hyper-V box was going to jump.

Now, the question is, will increasing the number of virtual CPUs actually improve VM performance?

What we actually end up with is a competition for CPU cycles between the low virtual CPU count Vista VMs that are easier to assign CPU cycles to and the high virtual CPU count Server 2008 VMs. As demand increases VM performance would actually decrease due to the need to juggle virtual CPU demands across the 8 Xeon cores.

We could throw the most expensive pair of Xeon X5492 Quad Cores that we have bumped up to 3.9GHz into the Hyper-V box running the second set of VMs and still we would not see top performance out of them due to the virtual CPU count in the VMs.

Ultimately, the point is that increasing the VM virtual CPU count does not necessarily mean that the VM's performance will increase.

Some of the Tech Days virtualization sessions discussed the above hardware considerations. We tied them into our own virtualization experiences.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists

*All Mac on SBS posts are posted on our in-house iMac via the Safari Web browser.