And that is only to work through all of the KBs that follow in this blog post to gain AD recovery skills right?
On one of the lists we are a part of there is an active discussion going on about having a second DC on a smaller network for "redundancy" purposes.
When it comes to an SBS Standard based network there are some caveats for that second DC whether it is virtual or physical:
- MPECS Inc. Blog: SBS Disaster Recovery with a Second DC Caveats
As we have learned in our past recovery situations that second DC can actually be a hindrance instead of a help when there is a need to restore Small Business Server Standard.
Virtual DCs
Now, when everything is virtualized one may be tempted to snapshot a DC prior to making any changes to provide a "fall-back" if things go sideways.
Some things to consider via the mentioned KB:
- DC should remain running continuously.
- Do not pause the DC VM for long periods of time.
- Problems may happen.
- System State backups are critical but have a shelf life.
- In multi-DC environments daily DC System State backups of at least two (2) DCs should be the norm.
When a DC is recovered back from a snapshot the following KB may be applicable:
- Microsoft KB: 875495: How to detect and recover from a USN rollback in Windows Server 2003, Windows Server 2008, and Windows Server 2008 R2
- Implications of rolling back a DC and the processes to possibly recover.
Now, take all of the above and read the following:
- Ask the Directory Services Team: DC's and VM's - Avoiding the Do-Over
- Mark Ramey goes over the possibilities when it comes to recovering DCs with some excellent visuals.
The point we are making?
It's okay to have a DC or three in a virtual lab that are used to break and tear apart then step back using a snapshot to then run through the above processes to figure out the recovery path of a restored-from-snapshot DC VM.
However, in a production environment, whether it be our own or our client's location, DC VM snapshots should _never_ be used. Period.
A good backup, that is one that has been fully recovered to bare metal and/or hypervisor, along with a System State backup, are the only way to go. Then, being familiar with the above processes and caveats to having multiple DCs in a production environment is a must.
WS Backup & StorageCraft ShadowProtect
All of our current, as of Windows Server 2008 R2, smaller client networks with the exception of those running on Hyper-V failover clusters (Win2K8 R2) are running a single DC.
In most cases that DC is Small Business Server 2008/2011 Standard.
Why?
Because we test our client's backups on a quarterly basis as part of our ongoing services we provide them.
Test restoring our client's systems on a regular basis gives us full confidence in our ability to restore their single SBS/DC using ShadowProtect and in some cases the native Windows Server Backup.
Introducing a second DC into the mix, in the case of SBS networks, brings about caveats that we need not deal with (see first blog post link) especially when times may be stressful already.
The key to being confident in a single DC environment is in the backup solution set.
To repeat: Confidence in our backup solution is the key to our deploying a single DC solution.
If we are not versed in restoring the backups we deploy at our client sites, at that on a regular basis, then how can we have the confidence to recommend a single DC solution to our clients? If we don't restore our client's backups how will we be aware of what is needed if things really go sideways and a restore is required?
We _are_ confident in our backup solutions built upon Windows Server Backup and now especially on StorageCraft's ShadowProtect Version 4. SP v4 has proven that once again we will be deploying ShadowProtect at all of our client sites as the Hyper-V restore throughput problems we saw in the past are no more.
ShadowProtect's Hardware Independent Restore feature is also a must for P2V and V2V restore situations even between Hypervisor versions.
In the end, it is our preference to keep a single DC in our small to medium solution sets. KISS is our preference. And, a single DC with no snapshots taken follows that line of simplicity. Plus, recovery becomes that much simpler.
Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book
*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.
Hi Philip
ReplyDeleteIn Hyper-V virtualized environment are you installing SP on each individual VM or on the Hyper-v host?
What are you using for off-site portion? have you tried Shadow Stream? Thanks in advance! Vik
Vik,
ReplyDeleteAt this point we back up the guests from within.
We have not taken to backing up the hosts because rebuilding a hyper-V host on its own partition can be a lot faster than doing a full host restore IMNSHO.
Things are changing though and backup tech from the various vendords may make host backup an option.
No, not tried Shadow Stream.
Off-site is managed by us for most of our clients.
Philip
Phil,
ReplyDeleteCould you do a blog covering how you typically configure your backup solution? I'm interested in what hardware you backup onto, whether you use image manager and if not how you manage retention periods, and what you use for offsite backups?
Cheers,
Troy
I'd love to hear what you're preferred offsite backup solutions are too.
ReplyDeleteGiven the choice each site I support would have TBU, HD, NAS and Offsite backup but it's simply not cost effective for most SMBs.
I've yet to see a good offsite backup solution that wasn't ridiculously expensive.. I assume you're literally just storing flat data offsite and not doing offsite exchange/sql/system state backups too?
Troy,
ReplyDeleteWe have many blog posts on the various styles of backups we deploy.
Blog Category for Backups.
A.,
Generally, we rotate our regional client's backups. That is the off-site. For remote clients the primary business contact would be the one to rotate off-site.
We prefer to grandfather at least one rotation for 6-12 months depending on retention requirements.
We don't do Internet based storage as it is too expensive and most connections only have a 1Mbit or smaller upload connection here.
Philip