The Lowercase w
30Jan/132

Cloning Windows Server 2012 Domain Controllers on vSphere 5

Microsoft introduced a lot of virtualization awareness in Windows Server 2012, particularly for domain controllers.  They are, for the most part, considered virtualization “safeguards” in that they prevent against some of the classic problems of virtualizing domain controllers.  Historically things like virtual machine snapshots, restoring from virtual machine image backups, or cloning domain controllers was either difficult or impossible.  With the introduction of the VM-GenerationID it is now safe to use virtual machine snapshots and even clone existing domain controllers.

How Does It Work?

The VM-GenerationID is a unique identifier exposed to the virtual machine by the hypervisor that helps to prevent issues with domain controller snapshots, cloning, etc.  When virtualizing Windows Server 2012 on vSphere, the VM-GenerationID is included as part of the virtual machine’s VMX file in the attribute vm.genid.  This attribute is present on all Windows Server 2012 VMs, not just those that are domain controllers.  See below for a few lines from a VMX file of a Windows Server 2012 VM with the vm.genid value highlighted.

evcCompatibilityMode = "TRUE"
softPowerOff = "FALSE"
vm.genid = "-5266020153200197717"

You need to be running a version of vSphere that supports VM-GenerationID.  That includes vSphere 5.0 Update 2 or vSphere 5.1 (if running vCenter 5.1, the ESXi hosts have to be running at least vSphere 5.0 Update 2).  You can tell if you’re running a compatible hypervisor by looking for “Hyper-V Generation Counter” in Device Manager of a Windows Server 2012 VM.

You’ll need to show hidden devices in Device Manager to see it.  Select View\show hidden devices:

spacer

Then expand System Devices and you’ll see Hyper-V Generation Counter.

spacer

When a Windows Server 2012 VM is promoted to a domain controller, the unique value of the VM-GenerationID is stored in the msDS-GenerationID attribute on its copy of the Active Directory database.  Open up Active Directory Users and Computers, select View\Advanced Features, and then right click on your domain controller and select Properties.  You can see the msDS-GenerationID attribute in the Attribute Editor tab. Note that since this value is not replicated to other domain controllers, the value will appear as "<not set>” if you try to look at the attributes of a different domain controller.

spacer

Prerequisites And Overview

Now that we’ve confirmed that VM-GenerationID is supported how do we go about cloning our domain controllers?  It’s really a very simple process, though you’ll see it doesn’t work exactly like cloning other virtual machines.  First, the prerequisites:

1) The domain controller running the PDC Emulator role must be running Windows Server 2012 and must be online during the process.  It isn’t necessary for this domain controller to be virtualized but it certainly can be.

2) The hypervisor needs to support VM-GenerationID.  Though we already stated this, it’s important to re-state it in case you have a cluster with a mix of versions (or are in the process of upgrading).  If you’re using DRS in fully automated mode then vCenter will automatically pick the best host to power up your cloned DC and if that host does not support VM-GenerationID then your clone will fail.  Yet another reason to keep all hosts in a cluster at a consistent build.

Now, on to a high level overview of the process:

1) Promote a server to a domain controller that will be used primarily for cloning.  This domain controller should not have any FSMO roles and should not be the primary/secondary DNS server for any servers on your network.  This step is not absolutely required, but you’ll see why I’m recommending this as we go through the cloning process.

2) Add the domain controller you just created to the “Cloneable Domain Controllers” group in Active Directory (located under the Users OU).

3) Create a list of “allowable” software on the DC you’re cloning.

4) Create a configuration XML file that specifies the settings for the new domain controller.

Preparing the DC for cloning

At this point you’ve promoted your domain controller and made sure it has no FSMO roles or acts as primary DNS for any servers.  Now it’s time to add the VM to the Cloneable Domain Controllers group in AD.

spacer

It is recommended that you remove the DC from this group after you’re finished cloning.

Creating the Allowable Applications List

Windows maintains a list of the applications and services that are allowed to be running on a domain controller that is used as the source for a clone.  These are mostly familiar Windows services, and you can view the full list at c:\Windows\System32\DefaultDCCloneAllowList.XML.  A Windows Server 2012 domain controller cannot contain any applications or services that may not function properly if the server is cloned.  It is intended to catch things like DHCP services that need to be authorized in AD and are better installed manually than cloned.  In order to see if any of these applications exist on the server, issue the following PowerShell command:Get-ADDCCloningExcludedApplicationList  Below is the output of that command on a domain controller in my lab:

PS C:\> Get-ADDCCloningExcludedApplicationList

Name                                                        Type
----                                                        ----
Microsoft Visual C++ 2008 Redistributable - x64 9.0.3072... Program
VMware Tools                                                Program
Microsoft Visual C++ 2008 Redistributable - x86 9.0.3072... WoW64Program
VMTools                                                     Service
vmvss                                                       Service
WLMS                                                        Service

You’ll notice some familiar applications on there, most notably the components of VMware Tools.  We’ve known for years that VMware Tools is fully compatible with cloning all virtual machines, so we simply need to “allow” these applications to be present on our domain controller before cloning by generating an XML file called CustomDCCloneAllowList.xml.  To generate the list, simply issue the same command with the –GenerateXML switch as shown below.  Note that the XML file will be created wherever you’ve stored your Active Directory database, or C:\Windows\NTDS by default.

PS C:\> Get-ADDCCloningExcludedApplicationList -GenerateXml
The inclusion list was written to 'C:\Windows\NTDS\CustomDCCloneAllowList.xml'.

Creating the Clone Configuration File

The next (and last) step you’ll need to perform is to create an XML file that contains the configuration details of your new domain controller.  This includes the new domain controller’s name, networking information, and AD Site (if required).  You can create the XML file using the PowerShell command New-ADDCCloneConfigFile.  As you can see below, you pass the necessary configuration parameters to the New-ADDCCloneConfigFile cmdlet to create the XML file:

PS C:\> New-ADDCCloneConfigFile -Static -IPv4Address "192.168.1.153" -IPv4DNSResolver "192.168.1.140" -IPv4SubnetMask "255.255.255.0" -IPv4DefaultGateway "192.168.1.1" -CloneComputerName "W12-DC5" -SiteName "NJ"

If everything worked you’ll see output similar to what you see below and a DCCloneConfig.xml file will be created in the same directory as your AD database:

spacer

What if you want to clone multiple domain controllers instead of just one at a time?  You can leave out all of the configuration information above and just provide the DNS server for the domain:

PS C:\windows\system32> New-ADDCCloneConfigFile –IPv4DNSResolver “192.168.1.120”

If you do not choose a server name for the clone, the name of the new domain controller will match the source domain with “-CLnnnn” added to the name. In our example, the source DC is named W12-DC4 so the new domain controller would be named W12-DC4-CL0001.  It is certainly possible to rename a domain controller after it has been promoted but it is a manual process that is different than renaming any other server.

You’ll need to create a new DCCloneConfig.xml file each time you want to clone the domain controller.  The file is not reusable and becomes invalid after a reboot of the source domain controller whether a clone operation has occurred or not.

Clone the VM

At this point you’re ready to clone the source domain controller.  You’ll need to power off the source domain controller before attempting to clone it.  Remember I mentioned above that you’ll want to create a dedicated DC for cloning?  This is why – even though cloning can be quick (especially with VAAI enabled arrays) it isn’t a great idea to take down a DC that other servers and workstations use as a primary or secondary DNS server or worse, one that has FSMO roles.  If you try to clone your DC without shutting it down first you’ll end up with a cloned DC that does not get customized and is an exact replica of your source DC.

Simply clone the virtual machine as you would any other VM that you clone, with one exception.  Do not try to apply a customization specification to customize the cloned VM.  The customization information is contained within the DCCloneConfig.xml file, so trying to use a guest customization from vSphere will result in a VM that does not boot.  I’m going to experiment with this a bit more but my initial testing with customization specifications was unsuccessful.

After the clone is completed, power on both the source domain controller as well as the newly cloned DC.  The cloning customization process will happen automatically.

spacer

That’s all there is to it! Now you’ve got a new fully functional cloned domain controller.

spacer

Have you tried out Windows Server 2012 yet?  I honestly believe that if people could look past the change in user interface they would see that Windows Server 2012 offers many significant improvements to the Windows Server platform.  It didn’t take me long to get used to the new interface and really enjoy working in Server Manager as a central management tool.  Give it a try!

Share this:

  • Email
  • More
    • Pin It
    • Share on Tumblr
    • Digg
    • Print
    Tagged as: Business Critical Applications, Domain Controllers, vSphere 5, vSphere 5.1, Windows 2012 2 Comments
    17Jan/1319

    vSphere Home Lab Upgrade–Synology DS1812+

    I built up my home lab back in late 2011 after finally deciding that I needed something that was completely mine and not a shared lab with others.  I built pretty much an identical lab to Jase McCarty’s (www.jasemccarty.com/blog/?p=1516) and have been very happy with it.  The only problem I’ve had is with my home lab storage.

    A funny thing happened on the way to figuring out what storage to use in my home lab.  Faced with the prospect of using a home NAS with 4 SATA drives, I wanted to see if I could find something that would give me better performance.  I had the opportunity to get my hands on a server that had 6 x 146GB 10K RPM drives and I jumped at the chance.  That server ended up being an old DL380 G4 (possibly even G3, not sure).  It seemed so smart at the time – why use 7200 RPM consumer SATA drives when I could use 10K RPM enterprise SCSI drives and get better performance.  I didn’t factor in one important thing: cache, or lack thereof.

    After seeing miserable performance I researched and bought some battery backed write cache – a whopping 128MB worth that had to be split between reads and writes.  Even with that, and using iSCSI software that let me create a RAM cache, I still had pretty bad performance.  How bad?  This bad.

    spacer

    Yep, that’s over 4,000ms of latency.  It wasn’t consistently this bad but trying to do multiple operations at once, like rebooting two VMs at once, would cause it.  The server was old, not true 64-bit, and just not the right fit.  There were probably other contributing factors beyond the lack of cache as well.  Not to mention the electricity cost of running a true server class computer in my house.  I realized my mistake and knew I needed to replace it with dedicated NAS storage.

    I know I could have used something like Nexenta Community Edition to get better performance out of the DL380.  For a variety of reasons that didn’t make sense in this situation.

    After much research and quite a bit of unnecessary delaying on my part (with the appropriate amount of ribbing from @ChrisWahl and @Millardjk) I finally decided on the Synology DS1812+.  I loaded it up with 4 x Sandisk 240GB SSD and 4 x Western Digital Red 2TB SATA drives and plan to use it for my home lab as well as for backing up my PC, pictures, videos, etc.

    spacer


    So how well does it work?  Is it a worthy replacement to the DL380? Seriously, what isn’t a worthy replacement to that old server?

    I have been extremely impressed with the Synology DSM software and how easy it is to set up volumes, create iSCSI targets, and configure link aggregation (more on that in a bit).  It also has lots of great features to use it as a home NAS so I’m very happy with my choice.  Performance has been great both on the SSDs and on the Western Digital Red drives.  The days of seconds of latency are gone – as you can see in the screenshots from esxtop I’m able to push extremely high I/O (both reads and writes) with less than 3ms of latency.  I may do some more detailed testing with the I/O Analyzer fling but for now this is good enough for me.

    spacer

    spacer

    My only disappointment is that I cannot configure true 802.3ad Dynamic Link Aggregation.  Unfortunately the switch I use in my home lab, a Dell PowerConnect 2816, only supports static link aggregation and not dynamic.  There are many posts on the Synology forum complaining about this but it’s really Dell’s issue and not anything wrong with the DS1812+.  I consider that a “nice to have” for a home lab but certainly not worth investing hundreds of dollars in a new switch that supports the proper link aggregation configuration.

    All in all I’m very happy with the addition of the Synology DS1812+ into my home lab.  The performance is great, the DSM software is very good, and there are some great things coming in the new DSM 4.2 (currently in beta).  I highly recommend any of the Synology models to folks who are looking to upgrade their home lab storage.

    Share this:

  • Email
  • More
    • Pin It
    • Share on Tumblr
    • Digg
    • Print
    Tagged as: Home Lab 19 Comments
    15Jan/132

    Honored to be an EMC Elect

    I found out this weekend that I was named as an EMC Elect in the program’s inaugural first year.  The EMC Elect award is similar to the VMware vExpert award in that it honors community involvement and knowledge sharing.  Needless to say I’m thrilled and honored to have won the award especially when I see my name associated with many other folks I respect.   Being rewarded for doing something we are all passionate about is what makes awards like the EMC Elect and vExpert all the more meaningful.

    You can learn more about the EMC Elect award here and see the full list of winners here.  If you follow the virtualization community you’ll likely see many familiar names.

    I didn’t even think I qualified for this award until Matthew Brender (@mjbrender) suggested that I nominate myself.  I ended up getting nominated by someone else in the community and subsequently won the award.  Thanks to Matt for introducing me to the program and recommending that I participate.  I still don’t feel like I truly deserve the award compared to some of the others who won, but I’m honored to be included among the winners.

    Looking forward to an exciting first year in the EMC Elect program!

    spacer

    Share this:

  • Email
  • More
    • Pin It
    • Share on Tumblr
    • Digg
    • Print
    Tagged as: EMC Elect, vExpert 2 Comments
    7Jan/130

    PVSCSI Bug Causing Exchange 2010 Jetstress to Crash

    A few weeks back I was called in to help a customer who was experiencing problems completing Jetstress testing for an Exchange 2010 deployment. It wasn’t an issue of Jetstress reporting failed tests. Rather, they were unable to get through most of their tests without the Jetstress application actually crashing (JetstressWin.exe has stopped working). They would see the following after the Jetstress testing completed but before it could write any log files to disk.

    spacer

    The only Jetstress related error in the Application log was an ESE error with Event ID 482:

    JetstressWin (3584) Instance3584.6: An attempt to write to the file “F:\DB\Jetstress006001.edb” at offset 63087017984 (0x0000000eb0478000) for 32768 (0x00008000) bytes failed over 0 seconds with system error 1117 (0x0000045d): “The request could not be performed because of an I/O device error.”. The write operation will fail with error –1022 (0xfffffc02). If this error persists then the file may be damaged and may need to be restored from a previous backup.

    During the process of Jetstress completing a test run, it generates a large amount of I/O as it flushes anything in cache to disk. It was at this point that the Jetstress application was crashing. This behavior is normal but it’s an important clue because of the high disk I/O generated.

    The customer was using vSphere 4.1 and the Exchange 2010 Mailbox servers were each configured with PVSCSI virtual SCSI controllers using VMDK files. As it turns out, they were hit with the PVSCI bug described in this VMware KB:

    Windows 2008 R2 virtual machine using a paravirtual SCSI adapter reports the error: Operating system error 1117 encountered  kb.vmware.com/kb/2004578

    The interesting thing to note here is that although Exchange is specifically called out he

    gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.