Wednesday 17 March 2010

NIC Binding Order On Server Core or Hyper-V Server 2008 RTM/R2

For those of us that have worked with SBS 2003 Premium in a 2 NIC setup, we know the importance of having the binding order correct for those 2 NICs are things really don’t work right.

The same is true for our Hyper-V implementations. There is a need to have the correct binding order for all of the NICs on a server that will be utilized by the host OS, cluster heartbeat, the Hyper-V virtual switches, and if needed iSCSI.

So, what order do we need to have those NICs in and how do we figure out which NIC is which?

Have a look at the following screenshot taken from the first node in a four node Hyper-V Server 2008 R2 cluster:

image

The images in order from top to bottom (numbers to the right of each one):

  1. HKLM\System\CurrentControlSet\services\TCPIP\Linkage
    1. Bind: Contains NIC GUIDs
    2. Export: Contains NIC GUIDs
    3. Route: Contains NIC GUIDs
  2. NIC report via the Hyper-V Server 2008 R2 menu
    • The Intel Modular Server compute modules have a maximum of 4 NICs as indicated here.
  3. NIC GUIDs in each Linkage key
    • Bottom most GUID is _first_ in the binding order.
  4. wmic nicconfig get Description,SettingID [Enter]
    1. WMIC command gives us a GUID to NIC name association.

So, to change the binding order on Server Core and Hyper-V Server 2008 RTM/R2, one moves the order of the GUIDs in each registry key:

  1. Bind
  2. Export
  3. Route

Once the GUID orders have been changed, reboot the server to get the changes to take affect.

The binding order we put the NICs in:

  1. NIC 0: Management
  2. NIC 1: Cluster Heartbeat
  3. NIC x: iSCSI (if needed)
  4. Microsoft Virtual Network Switch Adapter
  5. NIC 2+: Bind the MS Virtual Switch.
    1. NICs disappear from the binding keys after the Virtual Switch is bound to them and the host OS no longer has access.
  6. Microsoft Failover Cluster Virtual Adapter

Please make sure to take a screenshot of the default settings as above along with an export of that registry key just in case! A backup might be a good thing too.

IMS Ethernet Switch Ports To Node NICs

And one more thing when it comes to the IMS Ethernet Switch port structure relative to the NIC order listed in Hyper-V Server 2008 R2 or any other Server Core OS for that matter.

There is no direct correlation between which NIC comes in at 0, 1, 2, and 3 in the Hyper-V OS listing and the physical NIC to node NIC port in the IMS Ethernet Switch setup.

This is important to know since we already went ahead and set the NIC binding order according to the list in the Hyper-V OS but the _wrong_ NIC went down when we changed the VLAN setting in the IMS Ethernet Switch port management for port 1 on SWM 1!

In the screenshot below, we have VLAN 98 set up to be the heartbeat and migration network on physical NIC 2 on each node.

However, when we do an IPConfig /release && IPConfig /renew on the nodes the NIC assigned a 169.x.x.x IP was actually NIC 0!

image

As a result, we leave all of the node NICs connected to the management network to pick up an IP from the SBS VM. We can then change the VLAN setting to figure out which NIC in the OS belongs to which physical port in the IMS Ethernet Switch.

UPDATE: The proper binding order has finally been found! According to Chris Adams of Microsoft:

Now we have a pretty conclusive answer as to what order we need to put those NICs in:

  1. Microsoft Failover Cluster Virtual Adapter
  2. NIC 0: Management
  3. NIC 1: Cluster Heartbeat
  4. NIC x: iSCSI (if needed)
  5. Microsoft Virtual Network Switch Adapter
  6. NIC 2+: Bind the MS Virtual Switch.
    1. NICs disappear from the binding keys after the Virtual Switch is bound to them and the host OS no longer has access.

Thanks Chris!

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

6 comments:

Anonymous said...

Download nvspbind.exe from http://archive.msdn.microsoft.com/nvspbind

nvspbind.exe /++ " Management Interface" ms_tcpip


nvspbind.exe /o ms_tcpip show me order

Download nvspbind.exe from ms

Philip Elder Cluster MVP said...

A,

We saw and worked with the utility. However it became apparent, at least to us, that running through changing the binding order in the registry was as quick, if not quicker, and straight forward to keep track of.

Thanks for the pointer,

Philip

Anonymous said...

The recommended order seems to be different on all on webistes. What oder show it be inncluding live migration?

http://social.technet.microsoft.com/Forums/en/windowsserver2008r2highavailability/thread/2535c73a-a347-4152-be7a-ea7b24159520

http://blogs.netapp.com/msenviro/2009/10/hyper-v-networking-best-practices-part-three.html

Philip Elder Cluster MVP said...

Hi,

The binding order in our list is similar to the Hyper-V Best Practices Part 3 one.

MPECS Inc. Blog: Binding Order Live Migration Error

Philip

Anonymous said...

My Microsoft Failover Cluster Virtual Adapter has attched to my iscsi network card (ndis 0) instead of the cluster card. Is there a way to change this

I have set the bindings as you suggested

Anonymous said...

After much testing in my setup....Dell R610's

I ended up with Virtual Networks at the top, managment/CSV/LM in the middle, and iSCSI at the bottom. If the Virtual Networks were at the bottom of the order the file copy speed was horrible.

I also found this...

http://social.technet.microsoft.com/Forums/en-US/windowsserver2008r2highavailability/thread/2535c73a-a347-4152-be7a-ea7b24159520