Configuring Microsoft’s iSCSI Target 3.3 – Some Notes on Cluster Based Initiators

We just went through configuring an iSCSI Target setup for a three node Hyper-V Failover Cluster. Here are some thoughts on the configuration in no particular order.

iSCSI Target Name

Something to consider when connecting the cluster nodes to the iSCSI based storage is the name of the target. Only _one_ target will be used for the entire cluster. Therefore, we needed to give the target a rather generic name.

• Cluster-VM-BU-Target

Note that the naming convention for the target is very specific.

iSCSI Software Target – IQN

When we set up the target we need to log on to each node and start the iSCSI Initiator service. On Server Core or Hyper-V Server 2008 R2:

1. Open an elevated Command Prompt
2. iscsicpl.exe [Enter]
3. Click the Configuration Tab
4. Highlight the server’s Initiator name and Copy
5. Repeat for each node

Back on the iSCSI Software Target server we use the Advanced Identifiers dialogue when setting up the iSCSI Target to past in all of the cluster node’s IQNs.

When doing so we will be warned for each IQN beyond the first:

Microsoft iSCSI Software Target

You have assigned multiple initiators to this iSCSI target. Unless the server is in a cluster or uses a SAN file system, you should only allow one initiator access to an iSCSI target. Are you sure you want to allow multiple initiators to access this iSCSI target?

The answer is obviously yes for each prompt! :)

Once our target is configured we add the iSCSI Software Target server’s IP to the Targets tab on each node.

We then click the Auto Configure button shown above with all of the target VHDs showing up if everything is configured correctly.

iSCSI Target VHD to VM Connection

Once we have our storage configured and connected to the nodes, we initialize all of the new disks in Node 1’s Disk Management. From there we format them, give them specific names to identify them, and then set them to Offline status.

For the other nodes in the cluster we need only refresh the Disk Management view, ignore any Initialize Disk questions if requested, and then set the drives to Offline.

From there we can use Hyper-V Management to connect the iSCSI Target VHD to a specific VM via Hyper-V’s SCSI bus which is hot swap capable.

When we have all of the storage set up and set to Offline, we log onto the cluster based VM and set up the storage as needed.

iSCSI Software Target and MPIO

Now that we have all of the basic building blocks in place we will look at specific scenarios where the Microsoft iSCSI Software Target server has a number of LAN connections to serve up those targets. Introducing MPIO brings with it a level of complexity that may not be needed for our needs at this time though.

We will continue to test and work with this setup for backup targets for now.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

iSCSI Boot Means Disk-less Hyper-V Failover Cluster Nodes

As we are digging into the iSCSI remote disk setups for our Hyper-V Clusters we are beginning to see some neat possibilities for its use.

The one drawback to iSCSI that we can see when we look at our current commodity cluster setup using the Promise VTrak Ex10sD RAID Systems is I/O bandwidth.

With our two node setup we have an aggregate bandwidth of 48Gbit/Second via the external dual quad 3Gbit/Second SAS connections per node.

With our need for an iSCSI target for our clustered VM’s backup needs we can see a small cost savings for our two nodes by eliminating the need for internal drive storage for the host Hyper-V Server 2008 R2 OS installation.

Since I/O needs for the host OS would be relatively low, hosting the OS storage on our iSCSI Software Target setup could be an option for us.

• Intel Server Adapters: Which adapters support Intel iSCSI Remote Boot?
• Network Connectivity: Intel Ethernet User Guides
• A ZIP download of all of the guides.
• Once extracted the guides are accessed via HTML page.
• Network Connectivity: Intel® iSCSI Remote Boot: Intel® iSCSI Remote Boot troubleshooting and known issues help
• A list of links for troubleshooting iSCSI Remote Boot.
• Intel Support: Intel iSCSI Remote Boot
• The dedicated support page for the feature.

We will begin testing of this configuration once we have an HP ProLiant MicroServer set up to be the iSCSI Software Target.

Potentially, this setup would save the cost of two 15K or 10K SAS drives set up in each host node. The catch will be the cost savings versus performance costs. Once our testing has completed we will publish our results.

We are aiming this configuration test at our entry level 2 node commodity cluster which currently costs under $20K.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

iSCSI Initiator – Driver Causes Spontaneous Reboots BSOD STOP 0x0000000a

When things went sideways with the iSCSI Software Target setup yesterday our SBS 2011 VM started to spontaneously BSOD on a 0x0000000a.

When it started to happen the first place we checked when we logged in was the MiniDump folder since the server was set to create one.

Unfortunately, the first run through with the Crash Dump Analyzer (part of Windows Desktop Software Assurance MDOP) the result was misinterpreted by myself.

So, as can be seen by the above, the server went through a few BSOD and reboot cycles before it sunk in that the problem was with iSCSI Initiator and _not_ the Hyper-V SCSI bus. :(

Once we managed to get into the iSCSI Initiator’s control panel and disconnect the missing iSCSI target and drive we regained stability.

Chalk the miss up to not taking a step back, taking a deep breath or ten, and then subsequently looking at the situation again with fresh eyes . . . which was done a bit later. ;)

Moral of the story: Make sure that the iSCSI attached backup destinations are _offline_ in Disk Management on the VM before making any changes at the iSCSI Software Target server.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

iSCSI Target Update: Desktop Setup as “Server” Fail

Okay, so not long after yesterday’s post things started to go downhill big time.

The USB 3.0 ports and the docks started to break communication with each other randomly while the eSATA connections failed altogether.

So, it looks as though the desktop board setup will not work.

• MPECS Inc. Blog: A Tentative iSCSI Target Host System Setup
• MPECS Inc. Blog: Update on iSCSI Target via Desktop Hardware

StarTech Renesas USB 3.0 Card

We took the StarTech Renesas USB 3.0 card and installed it in one of our servers here in the shop. We configured the driver and connected two NexStar USB 3.0 docks to a NexStar USB 3.0 hub which was then connected to the USB 3.0 port.

We ran into the same funky behaviours with the NexStar SuperSpeed docks where we could not have more than two of them hooked in with live hard drives no matter what. before t The third drive connection would cycle in and out.

However, the two docks that have been connected to the StarTech card via USB 3.0 hub have been moving data non-stop since yesterday.

We have been moving the original iSCSI target VHDs from the drives we started with to a couple of large drives to consolidate the backups to two drive destinations.

The copy process runs at about 100MB/Second on average. We have moved Terabytes of data between the two docks without a hiccup as of yet.

Plus, we brought the iSCSI Software Target service online and hooked up our largest VHDs to their initiators to allow backups to run last night without a hitch.

StarTech USB 3.0 Drive Dock

As an alternative to the NexStar SuperSpeed docks we have ordered in one of these:

• StarTech 4 Bay eSATA USB 3.0 to SATA Hard Drive Docking Station for 2.5/3.5 HDD
  • P/N: SATDOCK4U3E

It will be interesting to see if the StarTech dock can plug directly into the USB 3.0 card and allow more than one drive to be connected without a USB 3.0 hub.

Conclusion

For now, it is looking like a real server will be needed as our dedicated iSCSI Software Target. :)

Does the HP ProLiant MicroServer count as a “real server”? Well, we will see.

The next step in this investigation is to drop a StarTech USB 3.0 card into one and run through these tests again. If things work out then we will have a very reasonably priced setup for our smaller clients.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Update on iSCSI Target via Desktop Hardware

A while back we published that we were testing a desktop board with Windows Server 2008 R2 Standard as a possible configuration to utilize the two eSATA and two USB 3.0 ports.

• MPECS Inc. Blog: A Tentative iSCSI Target Host System Setup

Here we are almost two weeks later with the setup providing backup targets for a number of different physical and virtual machines and things are moving along though not perfectly.

We had one NexStar USB 3.0 dock connected to a Vantec USB 3.0 hub which then connected to the USB 3.0 port on the motherboard and the other dock connected directly to a USB 3.0 port on the motherboard.

We set things up this way because there is some sort of conflict between the Vantec NexStar USB 3.0 hard drive dock and the USB 3.0 ports on the DQ67xx series Intel Desktop Boards (we have seen this with both DQ67SW and DQ67EP boards).

After our break this last weekend we came back to two of the four drives connected to the box being offline. We had moved things around prior to leaving to free up some space on the workbench however the connections were still online before we left.

The two drives that were offline were in one USB 3.0 dock and one eSATA dock. So, we could not pin the problem on either port set.

The eSATA drive turned out to be a cable that was not sitting quite right in the dock. Once the cable was pulled and re-seated at both ends the drive came online without a hitch.

The USB 3.0 drive however turned out to be the dock and the motherboard not playing nice anymore. We re-seated the cable with nothing happening. We power cycled the dock and sure enough the drive came online . . . and then it went offline, and then on, and then off, and so on.

Once we plugged the dock into the USB 3.0 hub that the other dock was already connected to the drive came online.

So, it is looking like while this setup may have been a good idea on paper the actual implementation is having poor results.

Now the kicker, we purchased a StarTech PEXUSB3S2 card (product site) as we had seen mention of the card being compatible with Windows Server 2008 RTM/R2.

The card arrived and we downloaded the driver:

It looks to be based on the same chip providing USB 3.0 connectivity on the Intel boards. :(

We will still plug the card in and run some tests with it on an actual server configuration. At least we can confirm that the NexStar USB 3.0 drive docks behave the same way when directly plugged into the card as far as the drive coming online and going offline in cycles.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Some Microsoft iSCSI Software Target 3.3 Resources

As we are delving into Microsoft’s free iSCSI Target software there are a number of key resources available to us for the product.

The download is available here:

• Microsoft Downloads: Microsoft iSCSI Software Target 3.3

There are a number of different resources available to us:

• TechNet: Microsoft iSCSI Software Target 3.3 Implementation Notes
• TechNet: Support Policies for Microsoft iSCSI Software Target 3.3 on Windows Server 2008 R2
• TechNet: Microsoft iSCSI Software Target 3.3 Known Issues and Updates
• Microsoft KB2535811: Supported and tested Microsoft iSCSI Software Target 3.3 limits
  • Note that this KB looks to be out of date as the TechNet Scalability Limits page has what looks to be a newer version of the KB content.

The following link is the the Microsoft iSCSI Software Target 3.3 Product Help:

• TechNet: Microsoft iSCSI Software Target 3.3 Product Help

Under Product Help we find a list of operating systems that it can be installed on. Unfortunately it looks as though Windows Storage Server 2008 R2 Essentials is not one of them.

iSCSI Target and Adapter Teaming

One of the notes in the KB and Scalability pages talk about network adapter teaming _not_ being supported.

• TechNet: Installing and Configuring Microsoft iSCSI Initiator

From that article we find:

Note

Use Microsoft MultiPath IO (MPIO) to manage multiple paths to iSCSI storage. Microsoft does not support teaming on network adapters that are used to connect to iSCSI-based storage devices. This is because the teaming software that is used in these types of configurations is not owned by Microsoft, and it is considered to be a non-Microsoft product. If you have an issue with network adapter teaming, contact your network adapter vendor for support. If you have contacted Microsoft Support, and they determine that teaming is the source of your problem, you might be required to remove the teaming from the configuration and/or contact the provider of the teaming software.

So, we see that the reason given here has to do with the teaming being provided by third parties.

So, off to Intel we go to see what they state:

• Intel Server Products Support: Is Adapter Teaming supported on any system which carries iSCSI traffic?

The answer is “No”. Unequivocally “No”. From the Intel support page:

Adapter Teaming using Intel's ANS is not supported for those ports transferring iSCSI traffic. This is due to limitations of the third-party iSCSI target/initiator software.

For load balancing and failover support in Microsoft Windows* operating systems, you can use Microsoft's* MPIO (MultiPath I/O). Check the Microsoft Initiator User Guide on how to setup MPIO.

If you are planning to use Microsoft MPIO on an Intel® Storage System, please ensure that your firmware supports multi path I/O.

Now, so far our test iSCSI Target box (previous post) has been running fine with the three NICs teamed in Load Balancing Mode. However, since we are running in an unsupported configuration we will look to see about how to configure MPIO on that box.

iSCSI Initiator

Once we have our iSCSI Target set up we need to connect to it!

The iSCSI Initiator Guide for Windows 7 and Windows Server 2008 R2 can be downloaded here:

• Microsoft Downloads: iSCSI Initiator Users Guide for Windows 7 and Windows Server 2008 R2

When working with Server Core R2 installations the iSCSI Initiator GUI can be started by the following command:

• iscsicpl.exe [Enter]

Server Core iSCSI Initiator GUI

Here is a blog post on enabling this functionality in Server Core 2008 RTM:

• Windows Storage Management Blog: How to add the iSCSI Initiator User Interface to Windows Server 2008 Server Core edition

iSCSI MPIO Configurations

One aspect that we will need to look into will be how to enable and configure multiple I/O paths to our iSCSI located storage if we were looking to use that storage for clustered VM VHD storage.

• Windows Storage Server Blog: Shared storage for Windows Failover Cluster with MPIO
• Off Campus Blog: Configuring iSCSI MPIO on Windows Server 2008 R2 (full)

For now we will be sticking with our DAS configurations, but there may be a need to look at iSCSI MPIO as an failover cluster storage alternate especially if this type of configuration brings the cost of a 2 node cluster down.

Conclusion

As we can see, there is a _lot_ of reading that needs to be done in order to understand the ins and outs of using Microsoft’s iSCSI Software Target. Plus, we need to check into our backup box’s configuration to figure out how to work iSCSI with multiple NIC ports.

KB2535811 Hat Tip: Susan Bradley

Our previous post: MPECS Inc. Blog: Microsoft Releases iSCSI Software Target Version 3.3 For Windows Server 2008 R2 . . . For Free!

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

A Tentative iSCSI Target Host System Setup

We are looking at a number of different system configurations to run Microsoft’s new iSCSI Target Software 3.3 on that will provide us with VM to external storage access.

Now, one of the drawbacks to using one server box with today’s hardware is that we are limited to USB 2.0 data speeds.

Our goal is to find a configuration that will work under high I/O loads over longer periods of time with at least four (4) externally connected hard disks at USB 3.0 and/or eSATA data speeds.

We have tried to run with PCI-E USB 3.0 host bus adapters but so far all are aimed at the consumer desktop space and thus we would not risk them in a server setting. We are looking into some that state that they have USB 3.0 support for Windows Server 2008 R2 but have not yet tested them.

So, tentatively, we are looking at the following configuration to be used as an iSCSI Target box _only_. Other than the external drives connected to the box that will be hooked into both standalone and cluster based VMs for backup the box will not be used.

• Intel Desktop Board DQ67EP
• Intel Core i5-2400S
• 4GB Kingston Non-ECC RAM
• 2x 250GB Seagate 2.5” SATA drives in RAID 1
• Antec ISK 300-150 case
• Intel dual Gigabit Server NIC configured for low profile use

Once we have the Win2K8 Standard R2 SP1 OS installed along with the necessary drivers we installed Microsoft’s free iSCSI Target software. We then teamed the three NICs using the PROSet utility in Load Balancing mode.

Note that the USB 3.0 driver does install correctly while the OS picks up the eSATA ports with built-in drivers. And, both eSATA and USB 3.0 ports support hot swap for the drives.

NOTE: We ran into the same weird behaviour with the USB 3.0 ports and the Vantec NexStar USB 3.0 drive docks where one connected fine but the other would not until we dropped a USB 3.0 hub in between the second NexStar drive dock and its port.

Microsoft iSCSI Software Target

We begin by running the downloaded file:

Once the extraction was complete our IE browser opened with:

From there we ran the install routine:

At one point during the install we were asked about the server’s update settings:

We left the setting where it was since our network is managed by WSUS which would pick up on the iSCSI Software Target’s presence.

Once the install was complete we opened the new MMC:

And we were greeted with:

We will run through a separate blog post on how to set up the target and connect the initiator in a separate post. The only quirk we ran into when setting up an iSCSI Target was that we were required to know the IQN for the Initiator ahead of time.

So far, we have run through creating a number of targets, connecting them, configuring the backup, and running the backups.

On the iSCSI Target Host we see:

On the guest that is running a backup to the iSCSI Target host we see:

The throughput shown above is to a target VHD located on an eSATA connected hard disk. So, the real throughput would be about 80MB/Second to 85MB/Second.

We will keep testing the configuration, but it looks like we have a winner here. The box configured without OS is under $1,000.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Hyper-V – Quick and Easy Network Storage Access via StarWind Free iSCSI

We had a need to back up a VM using network located storage during the migration we were running this weekend.

So, we downloaded StarWind iSCSI Target software that is free since the host was to be the source SBS 2003 box:

• StarWind Free iSCSI Target Software

It turned out that we needed to download and install the Microsoft iSCSI Software Initiator first:

• Microsoft iSCSI Software Initiator Version 2.08

Once the StarWind software was installed we were able to open the StarWind Console and connect to the StarWind iSCSI service:

• Default username and password:
• UN: root
• Pwd: starwind

For a backup destination we needed to do the following:

1. Right click on Targets and Add Target.
2. Set an Alias: SBS2011-BU-D0.
3. Storage type: Hard Disk.
4. Device type: Basic Virtual.
5. Device type: Image File device.
6. Device creation method: Create new virtual disk.
7. Choose a path: X:\Temporary Backups\SBS2011-BU-D0.img
• Size: 500GB (NOTE: Size created will take up that much on disk!)
8. Image File device parameters: Leave defaults.
9. Cache mode: Normal.
10. Next until done.

On the Hyper-V box we needed to enable the Microsoft iSCSI Initiator service.

Once done we were able to select the iSCSI target located on our source SBS 2003 box.

The target showed up as a new disk in Disk Management. We initialized it and then set it to Offline status. We then finally connect the storage to our VM using the Hyper-V SCSI bus.

We now had a temporary local backup destination to use after each migration step on the SBS 2011 VM.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Excellent How-To: Creating a SAN Using the Free Microsoft iSCSI Software Target 3.3

The Canadian IT Pro Connection blog has a series on how to get things going with the new iSCSI Target software from Microsoft.

• Canadian IT Pro Blog: Creating a SAN using Microsoft iSCSI Software Target 3.3

The How-To includes lots of good information on _why_ things are being set up the way they were. The article was written by fellow MVP Mitch Garvis who has an awesome ability to teach.

• Canadian IT Pro Blog: All for SAN and SAN for All!

The above article also written by Mitch gives a good explanation for the various types of shared storage and how they operate.

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer

Microsoft Releases iSCSI Software Target Version 3.3 For Windows Server 2008 R2 . . . For Free!

This is a pretty kewl bit of news since messing around with third party free iSCSI Target software is not always easy.

What do we use iSCSI Targets for?

Primarily for shared storage located on a device that is used by a Hyper-V Cluster and available across a LAN.

So, with a couple of no-name desktops and a small NAS that is iSCSI Target enabled we can build a lab based Hyper-V Cluster.

Since we already have a TechNet Plus subscription we have Windows Server 2008 R2 licensed for a lab based environment.

So, downloading the Microsoft iSCSI Software Target 3.3 and tying it into Windows Server 2008 R2 in a lab setting is pretty much straight forward as far as the licensing side of things go. Getting it installed, configured, and then a Hyper-V cluster up and running and Live Migration functioning as expected is a whole different ball game! ;)

A great candidate for Win2K8 R2 + iSCSI Software Target would the the HP MicroServer. We can drop an Intel PRO series dual or quad port server NIC to allow for some teaming if required, or even drop in a hardware RAID controller to bump up the performance on the disk subsystem.

• Microsoft Downloads: Microsoft iSCSI Software Target 3.3

After downloading the software have a look at the following:

• Microsoft iSCSI Software Target 3.3 Known Issues and Updates.

Jose’s blog post is an excellent resource:

• Jose Barreto’s Blog: Microsoft iSCSI Software Target 3.3 for Windows Server 2008 R2 available for public download

Hat Tip: Mikael Nystrom – The Deployment Bunny Blog - Microsoft iSCSI Software Target 3.3 for Windows Server 2008 R2 available for public download

Philip Elder
MPECS Inc.
Microsoft Small Business Specialists
Co-Author: SBS 2008 Blueprint Book

*Our original iMac was stolen (previous blog post). We now have a new MacBook Pro courtesy of Vlad Mazek, owner of OWN.

Windows Live Writer