13 Questions Answered on the Future of Hyper-V

13 Questions Answered on the Future of Hyper-V

What does the future hold for Hyper-V and its users? Technology moves fast so should Hyper-V admins be concerned about the future? Well, we don’t have a crystal ball to tell us what the future holds but we do have 3 industry experts and Microsoft MVPs to tell you what to expect. Following our hugely popular panel webinar 3 Emerging Technologies that will Change the Way you use Hyper-V we’ve decided to bring together all of the questions asked during both sessions (we hold 2 separate webinar sessions on the same topic to accommodate our European and American audiences) into one article with some extended answers to address the issue of what’s around the corner for Hyper-V and related technologies.

Let’s get started!

questions from the webinar 3 emerging technologies that will change the way you use hyper-v

The Questions

Question 1: Do you think IT Security is going to change as more and more workloads move into the cloud?

Answer: Absolutely! As long as we’re working with connected systems, no matter where they are located, we will always have to worry about security. 1 common misconception though is that just because a workload is housed inside of Microsoft Azure, doesn’t mean that it’s LESS secure. Public cloud platforms have been painstakingly setup from the ground up with the help of security experts in the industry. You’ll find that if best practices are followed, and rules of least access and just-in-time administration are followed, the public cloud is a highly secure platform.

Question 2: Do you see any movement to establish a global “law” of data security/restrictions that are not threatened by local laws (like the patriot act)?

Answer: Until all countries of the world are on the same page, I just don’t see this happening. The US treats data privacy in a very different way than the EU unfortunately. The upcoming General Data Protection Regulation (GDPR) coming in may of 2018 is a step in the right direction, but that only applies to the EU and data traversing the boundaries of the EU. It will certainly affect US companies and organizations, but nothing similar in nature is in the works there.

Question 3: In the SMB Space, where a customer may only have a single MS Essentials server and use Office 365, do you feel that this is still something that should move to the cloud?

Answer: I think the answer to that question depends greatly on the customer and the use case. As Didier, Thomas and I discussed in the webinar, the cloud is a tool, and you have to evaluate for each case, whether it makes sense or not to run that workload in the cloud. If for that particular customer, they could benefit from those services living in the cloud with little downside, then it may be a great fit. Again, it has to make sense, technically, fiscally, and operationally, before you can consider doing so.

Question 4: What exactly is a Container?

Answer: While not the same at all, it’s often easiest to see a container as a kind of ultra-stripped down VM. A container holds an ultra-slim OS image (In the case of Nano Server 50-60 MB), any supporting code framework, such as DotNet, and then whatever application you want to run within the container. They are not the same as a VM due to the fact that Windows containers all share the kernel of the underlying host OS. However, if you require further isolation, you can do so with Hyper-V containers, which allows you to run a container within an optimized VM so you can take advantage of Hyper-V’s isolation capabilities.

Question 5: On-Premises Computing is Considered to be a “cloud” now too correct?

Answer: That is correct! In my view, the term cloud doesn’t refer to a particular place, but to the new technologies and software-defined methods that are taking over datacenters today. So you can refer to your infrastructure on-prem as “private cloud”, and anything like Azure or AWS as “Public Cloud”. Then on top of that anything that uses both is referred to as “Hybrid Cloud”.

Question 6: What happens when my client goes to the cloud and they lose their internet service for 2 weeks?

Answer: The cloud, just like any technology solution, has its shortcomings that can be overcome if planned for properly. If you have mission critical service you’d like to host in the cloud, then you’ll want to research ways for the workload to be highly available. That would include a secondary internet connection from a different provider or some way to make that workload accessible from the on-prem location if needed. Regardless of where the workload is, you need to plan for eventualities like this.

Question 7: What Happened to Azure Pack?

Answer: Azure Pack is still around and usable, it will just be replaced by Azure stack at some point. In the meantime, there are integrations available that allow you to manage both solutions from your Azure Stack management utility.

Question 8: What about the cost of Azure Stack? What’s the entry point?

Answer: This is something of a difficult question. Ranges that I’ve heard range from 75k to 250k, depending on the vendor and the load-out. You’ll want to contact your preferred hardware vendor for more information on this question.

Question 9: We’re a hosting company, is it possible to achieve high levels of availability with Azure Stack?

Answer: Just like any technology solution, you can achieve the coveted 4 9s of availability. The question is how much money do you want to spend? You could do so with Azure stack and the correct supporting infrastructure. However, one other thing to keep in mind, your SLA is only as good as your supporting vendors as well. For example, if you sell 4 9s as an SLA, and the internet provider for your datacenter can only provide 99%, then you’ve already broken your SLA, so something to keep in mind there.

Question 10: For Smaller businesses running Azure Stack, should software vendors assume these businesses will look to purchase traditionally on-prem software solutions that are compatible with this? My company’s solution does not completely make sense for the public cloud, but this could bridge the gap. 

Answer: I think for most SMBs, Azure Stack will be fiscally out of reach. In Azure Stack you’re really paying for a “Cloud Platform”, and for most SMBs it will make more sense to take advantage of public Azure if those types of features are needed. that said, to answer your question, there are already vendors doing this. Anything that will deploy on public Azure using ARM will also deploy easily on Azure Stack.

Question 11: In Azure Stack, can I use any backup software and backup the VM to remote NAS storage or to AWS?

Answer: At release, there is no support for 3rd party backup solutions in Azure Stack. Right now there is a built-in flat file backup and that is it. I suspect that it will be opened up to third-party vendors at some point in time and it will likely be protected in much the same way as public Azure resources.

Question 12: How would a lot of these [Azure Stack] services be applied to the K-12 education market? There are lots of laws that require data to be stored in the same country. Yet providers often host in a different country. 

Answer: If you wanted to leverage a providers Azure stack somewhere, you would likely have to find one that actually hosts it in the geographical region you’re required to operate in. Many hosters will provide written proof of where the workload is hosted for these types of situations.

Question 13: I’m planning to move to public Azure, how many Azure cloud Instances would I need?

Answer: There is no hard set answer for this. It depends on the number of VMs/Applications and whether you run them in Azure as VMs or in Azure’s PaaS fabric. The Azure Pricing Calculator will give you an idea of VM sizes and what services are available.

Watch the webinar

Watch the altaro webinar now

Did you miss the webinar when it first went out? Has this blog post instilled a desire for you to rewatch the session again? Have no fear, we have set up an on-demand version for you to watch right now! Simply click on the link below to go the on-demand webinar page where you can watch a live recording of the webinar free.

WATCH NOW: 3 Emerging Technologies that will Change the Way you Use Hyper-V – Altaro On-demand Webinar

Join the Discussion: #FutureOfHyperV

Follow the hashtag #FutureOfHyperV on twitter and join the discussion about the impact of the emerging technologies discussed in our webinar along with other things that will shape the future of Hyper-V for years to come.

Did we miss something?

If you have a question on the future of Hyper-v or any of the 3 emerging technologies that were discussed in the webinar just post in the comments below and we will get straight back to you. Furthermore, if you asked a question during the webinar that you don’t see here, by all means, let us know in the comments section below and we will be sure to answer it here. Any follow-up questions are also very welcome – to feel free to let us know about that as well!

As always – thanks for reading!

How to Hot Add/Remove Virtual Network Adapters in Hyper-V 2016

How to Hot Add/Remove Virtual Network Adapters in Hyper-V 2016

Last week I showed you how to hot add/remove memory in Hyper-V 2016 and this week I’m covering another super handy new feature that system admins will also love. In fact, Hyper-V 2016 brought many fantastic features. Containers! It also added some features that indicate natural product maturation. On that list, we find “hot add/remove of virtual network adapters”. If that’s not obvious, it means that you can now add or remove virtual network adapters to/from running virtual machines.

Requirements for Hyper-V Hot Add/Remove of Virtual Network Adapters

To make hot add/remove of network adapters work in Hyper-V, you must meet these requirements:

  • Hypervisor must be 2016 version (Windows 10, Windows Server 2016, or Hyper-V Server 2016)
  • Virtual machine must be generation 2
  • To utilize the Device Naming feature, the virtual machine version must be at least 6.2. The virtual machine configuration version does not matter if you do not attempt to use Device Naming. Meaning, you can bring a version 5.0 virtual machine over from 2012 R2 to 2016 and hot add a virtual network adapter. A discussion on Device Naming will appear in a different article.

The guest operating system may need an additional push to realize that a change was made. I did not encounter any issues with the various operating systems that I tested.

How to Use PowerShell to Add or Remove a Virtual Network Adapter from a Running Hyper-V Guest

I always recommend PowerShell to work with second or higher network adapters to a virtual machine. Otherwise, they’re all called “Network Adapter”. Sorting that out can be unpleasant.

Adding a Virtual Adapter with PowerShell

Use Add-VMNetworkAdapter to add a network adapter to a running Hyper-V guest. That’s the same command that you’d use for an offline guest, as well. I don’t know why the authors chose the verb “Add” instead of “New”.

The above will work on a virtual machine with a configuration version of at least 6.2. If the virtual machine is set to a lower version, you get a rather confusing message that talks about DVD drives:

It does eventually get around to telling you exactly what it doesn’t like. You can avoid this error by not specifying the DeviceNaming parameter. If you’re scripting, you can avoid the parameter by employing splatting or by setting DeviceNaming to Off.

You can use any of the other parameters of Add-VMNetworkAdapter normally.

Removing a Virtual Adapter with PowerShell

To remove the adapter, use Remove-VMNetworkAdapter:

This is where things can get… interesting. Especially if you didn’t specify a unique name for the adapter. The Name parameter works like a search filter; it will remove any adapter that perfectly matches that name. So, if all of the virtual machine’s network adapters use the default name Network Adapter, and you specify Network Adapter for the Name parameter, then all of that VM’s adapters will be removed.

To address that issue, you’ll need to employ some cleverness. A quick ‘n’ dirty option would be to just remove all of the adapters, then add one. By default, that one adapter will pick up an IP from an available DHCP server. Since you can specify a static MAC address with the StaticMacAddress parameter of Add-VMNetworkAdapter, you can control that behavior with reservations.

You could also filter adapters by MAC address:

You could also use arrays to selectively remove items:

You could even use a loop to knock out all adapters after the first:

In my unscientific testing, virtual machine network adapters are always stored and retrieved in the order in which they were added, so the above script should always remove every adapter except the original. Based on the file format, I would expect that to always hold true. However, no documentation exists that outright supports that; use this sort of cleverness with caution.

I recommend naming your adapters to save a lot of grief in these instances.

How to Use the GUI to Add or Remove a Virtual Network Adapter from a Running Hyper-V Guest

These instructions work for both Hyper-V Manager and Failover Cluster Manager. Use the virtual machine’s Settings dialog in either tool.

Adding a Virtual Network Adapter in the GUI

Add a virtual network adapter to a running VM the same way that you add one to a stopped VM:

  1. On the VM’s Settings dialog, start on the Add Hardware page. The Network Adapter entry should be black, not gray. If it’s gray, then the VM is either Generation 1 or not in a valid state to add an adapter:
  2. Highlight Network Adapter and click Add.
  3. You will be taken to a screen where you can fill out all of the normal information for a network adapter. Set all items as desired.
  4. Once you’ve set everything to your liking, click OK to add the adapter and close the dialog or Apply to add the adapter and leave the dialog open.

Removing a Virtual Network Adapter in the GUI

As with adding an adapter, removing an adapter for a running virtual machine is performed the same way as adding one:

  1. Start on the Settings dialog for the virtual machine. Switch to the tab for the adapter that you wish to remove:
  2. Click the Remove button.
  3. The tab for the adapter to be removed will have all of its text crossed out. The dialog items for it will turn gray.
  4. Click OK to remove the adapter and close the dialog or Apply to remove the adapter and leave the dialog open. Click Cancel if you change your mind. For OK or Apply, a prompt will appear with a warning that you’ll probably disrupt network communications:

Hot Add/Remove of Hyper-V Virtual Adapters for Linux Guests

I didn’t invest a great deal of effort into testing, but this feature works for Linux guests with mixed results. A Fedora guest running on my Windows 10 system was perfectly happy with it:


OpenSUSE Leap… not so much:


But then, I added another virtual network adapter to my OpenSUSE system. This time, I remembered to connect it to a virtual switch before adding. It liked that much better:


So, the moral of the story: for Linux guests, always specify a virtual switch when hot adding a virtual network card. Connecting it afterward does not help.

Also notice that OpenSUSE Leap did not ever automatically configure the adapter for DHCP, whereas Fedora did. As I mentioned in the beginning of the article, you might need to give some environments an extra push.

Also, Leap seemed to get upset when I hot removed the adapter:


To save your eyes, the meat of that message says: “unable to send revoke receive buffer to netvsp”. I don’t know if that’s serious or not. The second moral of this story, then: hot removing network adapters might leave some systems in an inconsistent, unhappy state.

My Thoughts on Hyper-V’s Hot Add/Remove of Network Adapters Feature

Previous versions of Hyper-V did not have this feature and I never missed it. I wasn’t even aware that other hypervisors had it until I saw posts from people scrounging for any tiny excuse to dump hate on Microsoft. Sure, I’ve had a few virtual machines with services that benefited from multiple network adapters. However, I knew of that requirement going in, so I just built them appropriately from the beginning. I suppose that’s a side effect of competent administration. Overall, I find this feature to be a hammer desperately seeking a nail.

That said, it misses the one use that I might have: it doesn’t work for generation 1 VMs. As you know, a generation 1 Hyper-V virtual machine can only PXE boot from a legacy network adapter. The legacy network adapter has poor performance. I’d like to be able to remove that legacy adapter post-deployment without shutting down the virtual machine. That said, it’s very low on my wish list. I’m guessing that we’ll eventually be using generation 2 VMs exclusively, so the problem will handle itself.

During my testing, I did not find any problems at all using this feature with Windows guests. As you can see from the Linux section, things didn’t go quite as well there. Either way, I would think twice about using this feature with production systems. Network disruptions do not always behave exactly as you might think because networks often behave unexpectedly. Multi-homed systems often crank the “strange” factor up somewhere near “haunted”. Multi-home a system and fire up Wireshark. I can almost promise that you’ll see something that you didn’t expect within the first five minutes.

I know that you’re going to use this feature anyway, and that’s fine; that’s why it’s there. I would make one recommendation: before removing an adapter, clear its TCP/IP settings and disconnect it from the virtual switch. That gives the guest operating system a better opportunity to deal with the removal of the adapter on familiar terms.

95 Best Practices for Optimizing Hyper-V Performance

95 Best Practices for Optimizing Hyper-V Performance

We can never get enough performance. Everything needs to be faster, faster, faster! You can find any number of articles about improving Hyper-V performance and best practices, of course, unfortunately, a lot of the information contains errors, FUD, and misconceptions. Some are just plain dated. Technology has changed and experience is continually teaching us new insights. From that, we can build a list of best practices that will help you to tune your system to provide maximum performance.

How to optimize Hyper-V Performance

Philosophies Used in this Article

This article focuses primarily on performance. It may deviate from other advice that I’ve given in other contexts. A system designed with performance in mind will be built differently from a system with different goals. For instance, a system that tries to provide high capacity at a low price point would have a slower performance profile than some alternatives.

  • Subject matter scoped to the 2012 R2 and 2016 product versions.
  • I want to stay on target by listing the best practices with fairly minimal exposition. I’ll expand ideas where I feel the need; you can always ask questions in the comments section.
  • I am not trying to duplicate pure physical performance in a virtualized environment. That’s a wasted effort.
  • I have already written an article on best practices for balanced systems. It’s a bit older, but I don’t see anything in it that requires immediate attention. It was written for the administrator who wants reasonable performance but also wants to stay under budget.
  • This content targets datacenter builds. Client Hyper-V will follow the same general concepts with variable applicability.

General Host Architecture

If you’re lucky enough to be starting in the research phase — meaning, you don’t already have an environment — then you have the most opportunity to build things properly. Making good purchase decisions pays more dividends than patching up something that you’ve already got.

  1. Do not go in blind.
    • Microsoft Assessment and Planning Toolkit will help you size your environment: MAP Toolkit
    • Ask your software vendors for their guidelines for virtualization on Hyper-V.
    • Ask people that use the same product(s) if they have virtualized on Hyper-V.
  2. Stick with logo-compliant hardware. Check the official list: https://www.windowsservercatalog.com/
  3. Most people will run out of memory first, disk second, CPU third, and network last. Purchase accordingly.
  4. Prefer newer CPUs, but think hard before going with bleeding edge. You may need to improve performance by scaling out. Live Migration requires physical CPUs to be the same or you’ll need to enable CPU compatibility mode. If your environment starts with recent CPUs, then you’ll have the longest amount of time to be able to extend it. However, CPUs commonly undergo at least one revision, and that might be enough to require compatibility mode. Attaining maximum performance may reduce virtual machine mobility.
  5. Set a target density level, e.g. “25 virtual machines per host”. While it may be obvious that higher densities result in lower performance, finding the cut-off line for “acceptable” will be difficult. However, having a target VM number in mind before you start can make the challenge less nebulous.
  6. Read the rest of this article before you do anything.

Management Operating System

Before we carry on, I just wanted to make sure to mention that Hyper-V is a type 1 hypervisor, meaning that it runs right on the hardware. You can’t “touch” Hyper-V because it has no direct interface. Instead, you install a management operating system and use that to work with Hyper-V. You have three choices:

Note: Nano Server initially offered Hyper-V, but that functionality will be removed (or has already been removed, depending on when you read this). Most people ignore the fine print of using Nano Server, so I never recommended it anyway.

TL;DR: In absence of a blocking condition, choose Hyper-V Server. A solid blocking condition would be the Automatic Virtual Machine Activation feature of Datacenter Edition. In such cases, the next preferable choice is Windows Server in Core mode.

I organized those in order by distribution size. Volumes have been written about the “attack surface” and patching. Most of that material makes me roll my eyes. No matter what you think of all that, none of it has any meaningful impact on performance. For performance, concern yourself with the differences in CPU and memory footprint. The widest CPU/memory gap lies between Windows Server and Windows Server Core. When logged off, the Windows Server GUI does not consume many resources, but it does consume some. The space between Windows Server Core and Hyper-V Server is much tighter, especially when the same features/roles are enabled.

One difference between Core and Hyper-V Server is the licensing mechanism. On Datacenter Edition, that does include the benefit of Automatic Virtual Machine Activation (AVMA). That only applies to the technological wiring. Do not confuse it with the oft-repeated myth that installing Windows Server grants guest licensing privileges. The legal portion of licensing stands apart; read our eBook for starting information.

Because you do not need to pay for the license for Hyper-V Server, it grants one capability that Windows Server does not: you can upgrade at any time. That allows you to completely decouple the life cycle of your hosts from your guests. Such detachment is a hallmark of the modern cloud era.

If you will be running only open source operating systems, Hyper-V Server is the natural choice. You don’t need to pay any licensing fees to Microsoft at all with that usage. I don’t realistically expect any pure Linux shops to introduce a Microsoft environment, but Linux-on-Hyper-V is a fantastic solution in a mixed-platform environment. And with that, let’s get back onto the list.

Management Operating System Best Practices for Performance

  1. Prefer Hyper-V Server first, Windows Server Core second
  2. Do not install any software, feature, or role in the management operating system that does not directly aid the virtual machines or the management operating system. Hyper-V prioritizes applications in the management operating system over virtual machines. That’s because it trusts you; if you are running something in the management OS, it assumes that you really need it.
  3. Do not log on to the management operating system. Install the management tools on your workstation and manipulate Hyper-V remotely.
  4. If you must log on to the management operating system, log off as soon as you’re done.
  5. Do not browse the Internet from the management operating system. Don’t browse from any server, really.
  6. Stay current on mainstream patches.
  7. Stay reasonably current on driver versions. I know that many of my peers like to install drivers almost immediately upon release, but I can’t join that camp. While it’s not entirely unheard of for a driver update to bring performance improvements, it’s not common. With all of the acquisitions and corporate consolidations going on in the hardware space — especially networking — I feel that the competitive drive to produce quality hardware and drivers has entered a period of decline. In simple terms, view new drivers as a potential risk to stability, performance, and security.
  8. Join your hosts to the domain. Systems consume less of your time if they answer to a central authority.
  9. Use antivirus and intrusion prevention. As long you choose your anti-malware vendor well and the proper exclusions are in place, performance will not be negatively impacted. Compare that to the performance of a compromised system.
  10. Read through our article on host performance tuning.

Leverage Containers

In the “traditional” virtualization model, we stand up multiple virtual machines running individual operating system environments. As “virtual machine sprawl” sets in, we wind up with a great deal of duplication. In the past, we could justify that as a separation of the environment. Furthermore, some Windows Server patches caused problems for some software but not others. In the modern era, containers and omnibus patch packages have upset that equation.

Instead of building virtual machine after virtual machine, you can build a few virtual machines. Deploy containers within them. Strategies for this approach exceed the parameters of this article, but you’re aiming to reduce the number of disparate complete operating system environments deployed. With careful planning, you can reduce density while maintaining a high degree of separation for your services. Fewer kernels are loaded, fewer context switches occur, less memory contains the same code bits, fewer disk seeks to retrieve essentially the same information from different locations.

  1. Prefer containers over virtual machines where possible.


You can’t do a great deal to tune CPU performance in Hyper-V. Overall, I count that among my list of “good things”; Microsoft did the hard work for you.

  1. Follow our article on host tuning; pay special attention to C States and the performance power settings.
  2. For Intel chips, leave hyperthreading on unless you have a defined reason to turn it off.
  3. Leave NUMA enabled in hardware. On your VMs’ property sheet, you’ll find a Use Hardware Topology button. Remember to use that any time that you adjust the number of vCPUs assigned to a virtual machine or move it to a host that has a different memory layout (physical core count and/or different memory distribution).
    best pratices for optimizing hyper-v performance - settings NUMA configuration
  4. Decide whether or not to allow guests to span NUMA nodes (the global host NUMA Spanning setting). If you size your VMs to stay within a NUMA node and you are careful to not assign more guests than can fit solidly within each NUMA node, then you can increase individual VM performance. However, if the host has trouble locking VMs into nodes, then you can negatively impact overall memory performance. If you’re not sure, just leave NUMA at defaults and tinker later.
  5. For modern guests, I recommend that you use at least two virtual CPUs per virtual machine. Use more in accordance with the virtual machine’s performance profile or vendor specifications. This is my own personal recommendation; I can visibly detect the response difference between a single vCPU guest and a dual vCPU guest.
  6. For legacy Windows guests (Windows XP/Windows Server 2003 and earlier), use 1 vCPU. More will likely hurt performance more than help.
  7. Do not grant more than 2 vCPU to a virtual machine without just cause. Hyper-V will do a better job reducing context switches and managing memory access if it doesn’t need to try to do too much core juggling. I’d make exceptions for very low-density hosts where 2 vCPU per guest might leave unused cores. At the other side, if you’re assigning 24 cores to every VM just because you can, then you will hurt performance.
  8. If you are preventing VMs from spanning NUMA nodes, do not assign more vCPU to a VM than you have matching physical cores in a NUMA node (usually means the number of cores per physical socket, but check with your hardware manufacturer).
  9. Use Hyper-V’s priority, weight, and reservation settings with great care. CPU bottlenecks are highly uncommon; look elsewhere first. A poor reservation will cause more problems than it solves.


I’ve long believed that every person that wants to be a systems administrator should be forced to become conversant in x86 assembly language, or at least C. I can usually spot people that have no familiarity with programming in such low-level languages because they almost invariably carry a bizarre mental picture of how computer memory works. Fortunately, modern memory is very, very, very fast. Even better, the programmers of modern operating system memory managers have gotten very good at their craft. Trying to tune memory as a systems administrator rarely pays dividends. However, we can establish some best practices for memory in Hyper-V.

  1. Follow our article on host tuning. Most importantly, if you have multiple CPUs, install your memory such that it uses multi-channel and provides an even amount of memory to each NUMA node.
  2. Be mindful of operating system driver quality. Windows drivers differ from applications in that they can permanently remove memory from the available pool. If they do not properly manage that memory, then you’re headed for some serious problems.
  3. Do not make your CSV cache too large.
  4. For virtual machines that will perform high quantities of memory operations, avoid dynamic memory. Dynamic memory disables NUMA (out of necessity). How do you know what constitutes a “high volume”? Without performance monitoring, you don’t.
  5. Set your fixed memory VMs to a higher priority and a shorter startup delay than your Dynamic Memory VMs. This ensures that they will start first, allowing Hyper-V to plot an optimal NUMA layout and reduce memory fragmentation. It doesn’t help a lot in a cluster, unfortunately. However, even in the best case, this technique won’t yield many benefits.
  6. Do not use more memory for a virtual machine than you can prove that it needs. Especially try to avoid using more memory than will fit in a single NUMA node.
  7. Use Dynamic Memory for virtual machines that do not require the absolute fastest memory performance.
  8. For Dynamic Memory virtual machines, pay the most attention to the startup value. It sets the tone for how the virtual machine will be treated during runtime. For virtual machines running full GUI Windows Server, I tend to use a startup of either 1 GB or 2 GB, depending on the version and what else is installed.
  9. For Dynamic Memory VMs, set the minimum to the operating system vendor’s stated minimum (512 MB for Windows Server). If the VM hosts a critical application, add to the minimum to ensure that it doesn’t get choked out.
  10. For Dynamic Memory VMs, set the maximum to a reasonable amount. You’ll generally discover that amount through trial and error and performance monitoring. Do not set it to an arbitrarily high number. Remember that, even on 2012 R2, you can raise the maximum at any time.

Check the CPU section for NUMA guidance.


In the time that I’ve been helping people with Hyper-V, I don’t believe that I’ve seen anyone waste more time worrying about anything that’s less of an issue than networking. People will read whitepapers and forums and blog articles and novels and work all weekend to draw up intricately designed networking layouts that need eight pages of documentation. But, they won’t spend fifteen minutes setting up a network utilization monitor. I occasionally catch grief for using MRTG since it’s old and there are shinier, bigger, bolder tools, but MRTG is easy and quick to set up. You should know how much traffic your network pushes. That knowledge can guide you better than any abstract knowledge or feature list.

That said, we do have many best practices for networking performance in Hyper-V.

  1. Follow our article on host tuning. Especially pay attention to VMQ on gigabit and separation of storage traffic.
  2. If you need your network to go faster, use faster adapters and switches. A big team of gigabit won’t keep up with a single 10 gigabit port.
  3. Use a single virtual switch per host. Multiple virtual switches add processing overhead. Usually, you can get a single switch to do whatever you wanted multiple switches to do.
  4. Prefer a single large team over multiple small teams. This practice can also help you to avoid needless virtual switches.
  5. For gigabit, anything over 4 physical ports probably won’t yield meaningful returns. I would use 6 at the outside. If you’re using iSCSI or SMB, then two more physical adapters just for that would be acceptable.
  6. For 10GbE, anything over 2 physical ports probably won’t yield meaningful returns.
  7. If you have 2 10GbE and a bunch of gigabit ports in the same host, just ignore the gigabit. Maybe use it for iSCSI or SMB, if it’s adequate for your storage platform.
  8. Make certain that you understand how the Hyper-V virtual switch functions. Most important:
    • You cannot “see” the virtual switch in the management OS except with Hyper-V specific tools. It has no IP address and no presence in the Network and Sharing Center applet.
    • Anything that appears in Network and Sharing Center that you think belongs to the virtual switch is actually a virtual network adapter.
    • Layer 3 (IP) information in the host has no bearing on guests — unless you create an IP collision
  9. Do not create a virtual network adapter in the management operating system for the virtual machines. I did that before I understood the Hyper-V virtual switch, and I have encountered lots of other people that have done it. The virtual machines will use the virtual switch directly.
  10. Do not multi-home the host unless you know exactly what you are doing. Valid reasons to multi-home:
    • iSCSI/SMB adapters
    • Separate adapters for cluster roles. e.g. “Management”, “Live Migration”, and “Cluster Communications”
  11. If you multi-home the host, give only one adapter a default gateway. If other adapters must use gateways, use the old route command or the new New-NetRoute command.
  12. Do not try to use internal or private virtual switches for performance. The external virtual switch is equally fast. Internal and private switches are for isolation only.
  13. If all of your hardware supports it, enable jumbo frames. Ensure that you perform validation testing (i.e.: ping storage-ip -f -l 8000)
  14. Pay attention to IP addressing. If traffic needs to locate an external router to reach another virtual adapter on the same host, then traffic will traverse the physical network.
  15. Use networking QoS if you have identified a problem.
    • Use datacenter bridging, if your hardware supports it.
    • Prefer the Weight QoS mode for the Hyper-V switch, especially when teaming.
    • To minimize the negative side effects of QoS, rely on limiting the maximums of misbehaving or non-critical VMs over trying to guarantee minimums for vital VMs.
  16. If you have SR-IOV-capable physical NICs, it provides the best performance. However, you can’t use the traditional Windows team for the physical NICs. Also, you can’t use VMQ and SR-IOV at the same time.
  17. Switch-embedded teaming (2016) allows you to use SR-IOV. Standard teaming does not.
  18. If using VMQ, configure the processor sets correctly.
  19. When teaming, prefer Switch Independent mode with the Dynamic load balancing algorithm. I have done some performance testing on the types (near the end of the linked article). However, a reader commented on another article that the Dynamic/Switch Independent combination can cause some problems for third-party load balancers (see comments section).


When you need to make real differences in Hyper-V’s performance, focus on storage. Storage is slow. The best way to make storage not be slow is to spend money. But, we have other ways.

  1. Follow our article on host tuning. Especially pay attention to:
    • Do not break up internal drive bays between Hyper-V and the guests. Use one big array.
    • Do not tune the Hyper-V partition for speed. After it boots, Hyper-V averages zero IOPS for itself. As a prime example, don’t put Hyper-V on SSD and the VMs on spinning disks. Do the opposite.
    • The best ways to get more storage speed is to use faster disks and bigger arrays. Almost everything else will only yield tiny differences.
  2. For VHD (not VHDX), use fixed disks for maximum performance. Dynamically-expanding VHD is marginally, but measurably, slower.
  3. For VHDX, use dynamically-expanding disks for everything except high-utilization databases. I receive many arguments on this, but I’ve done the performance tests and have years of real-world experience. You can trust that (and run the tests yourself), or you can trust theoretical whitepapers from people that make their living by overselling disk space but have perpetually misplaced their copy of diskspd.
  4. Avoid using shared VHDX (2012 R2) or VHDS (2016). Performance still isn’t there. Give this technology another maturation cycle or two and look at it again.
  5. Where possible, do not use multiple data partitions in a single VHD/X.
  6. When using Cluster Shared Volumes, try to use at least as many CSVs as you have nodes. Starting with 2012 R2, CSV ownership will be distributed evenly, theoretically improving overall access.
  7. You can theoretically improve storage performance by dividing virtual machines across separate storage locations. If you need to make your arrays span fewer disks in order to divide your VMs’ storage, you will have a net loss in performance. If you are creating multiple LUNs or partitions across the same disks to divide up VMs, you will have a net loss in performance.
  8. For RDS virtual machine-based VDI, use hardware-based or Windows’ Hyper-V-mode deduplication on the storage system. The read hits, especially with caching, yield positive performance benefits.
  9. The jury is still out on using host-level deduplication for Windows Server guests, but it is supported with 2016. I personally will be trying to place Server OS disks on SMB storage deduplicated in Hyper-V mode.
  10. The slowest component in a storage system is the disk(s); don’t spend a lot of time worrying about controllers beyond enabling caching.
  11. RAID-0 is the fastest RAID type, but provides no redundancy.
  12. RAID-10 is generally the fastest RAID type that provides redundancy.
  13. For Storage Spaces, three-way mirror is fastest (by a lot).
  14. For remote storage, prefer MPIO or SMB multichannel over multiple unteamed adapters. Avoid placing this traffic on teamed adapters.
  15. I’ve read some scattered notes that say that you should format with 64 kilobyte allocation units. I have never done this, mostly because I don’t think about it until it’s too late. If the default size hurts anything, I can’t tell. Someday, I’ll remember to try it and will update this article after I’ve gotten some performance traces. If you’ll be hosting a lot of SQL VMs and will be formatting their VHDX with 64kb AUs, then you might get more benefit.
  16. I still don’t think that ReFS is quite mature enough to replace NTFS for Hyper-V. For performance, I definitely stick with NTFS.
  17. Don’t do full defragmentation. It doesn’t help. The minimal defragmentation that Windows automatically performs is all that you need. If you have some crummy application that makes this statement false, then stop using that application or exile it to its own physical server. Defragmentation’s primary purpose is to wear down your hard drives so that you have to buy more hard drives sooner than necessary, which is why employees of hardware vendors recommend it all the time. If you have a personal neurosis that causes you pain when a disk becomes “too” fragmented, use Storage Live Migration to clear and then re-populate partitions/LUNs. It’s wasted time that you’ll never get back, but at least it’s faster. Note: All retorts must include verifiable and reproducible performance traces, or I’m just going to delete them.


For real performance, don’t cluster virtual machines. Use fast internal or direct-attached SSDs. Cluster for redundancy, not performance. Use application-level redundancy techniques instead of relying on Hyper-V clustering.

In the modern cloud era, though, most software doesn’t have its own redundancy and host clustering is nearly a requirement. Follow these best practices:

  1. Validate your cluster. You may not need to fix every single warning, but be aware of them.
  2. Follow our article on host tuning. Especially pay attention to the bits on caching storage. It includes a link to enable CSV caching.
  3. Remember your initial density target. Add as many nodes as necessary to maintain that along with sufficient extra nodes for failure protection.
  4. Use the same hardware in each node. You can mix hardware, but CPU compatibility mode and mismatched NUMA nodes will have at least some impact on performance.
  5. For Hyper-V, every cluster node should use a minimum of two separate IP endpoints. Each IP must exist in a separate subnet. This practice allows the cluster to establish multiple simultaneous network streams for internode traffic.
    • One of the addresses must be designated as a “management” IP, meaning that it must have a valid default gateway and register in DNS. Inbound connections (such as your own RDP and PowerShell Remoting) will use that IP.
    • None of the non-management IPs should have a default gateway or register in DNS.
    • One alternative IP endpoint should be preferred for Live Migration. Cascade Live Migration preference order through the others, ending with the management IP. You can configure this setting most easily in Failover Cluster Manager by right-clicking on the Networks node.
    • Further IP endpoints can be used to provide additional pathways for cluster communications. Cluster communications include the heartbeat, cluster status and update messages, and Cluster Shared Volume information and Redirected Access traffic.
    • You can set any adapter to be excluded from cluster communications but included in Live Migration in order to enforce segregation. Doing so generally does not improve performance, but may be desirable in some cases.
    • You can use physical or virtual network adapters to host cluster IPs.
    • The IP for each cluster adapter must exist in a unique subnet on that host.
    • Each cluster node must contain an IP address in the same subnet as the IPs on other nodes. If a node does not contain an IP in a subnet that exists on other nodes, then that network will be considered “partitioned” and the node(s) without a member IP will be excluded from that network.
    • If the host will connect to storage via iSCSI, segregate iSCSI traffic onto its own IP(s). Exclude it/them from cluster communications and Live Migration. Because they don’t participate in cluster communications, it is not absolutely necessary that they be placed into separate subnets. However, doing so will provide some protection from network storms.
  6. If you do not have RDMA-capable physical adapters, Compression usually provides the best Live Migration performance.
  7. If you do have RDMA-capable physical adapters, SMB usually provides the best Live Migration performance.
  8. I don’t recommend spending time tinkering with the metric to shape CSV traffic anymore. It utilizes SMB, so the built-in SMB multi-channel technology can sort things out.

Virtual Machines

The preceding guidance obliquely covers several virtual machine configuration points (check the CPU and the memory sections). We have a few more:

  1. Don’t use Shielded VMs or BitLocker. The encryption and VMWP hardening incur overhead that will hurt performance. The hit is minimal — but this article is about performance.
  2. If you have 1) VMs with very high inbound networking needs, 2) physical NICs >= 10GbE, 3) VMQ enabled, 4) spare CPU cycles, then enable RSS within the guest operating systems. Do not enable RSS in the guest OS unless all of the preceding are true.
  3. Do not use the legacy network adapter in Generation 1 VMs any more than absolutely necessary.
  4. Utilize checkpoints rarely and briefly. Know the difference between standard and “production” checkpoints.
  5. Use time synchronization appropriately. Meaning, virtual domain controllers should not have the Hyper-V time synchronization service enabled, but all other VMs should (generally speaking). The hosts should pull their time from the domain hierarchy. If possible, the primary domain controller should be pulling from a secured time source.
  6. Keep Hyper-V guest services up-to-date. Supported Linux systems can be updated via kernel upgrades/updates from their distribution repositories. Windows 8.1+ and Windows Server 2012 R2+ will update from Windows Update.
  7. Don’t do full defragmentation in the guests, either. Seriously. We’re administering multi-spindle server equipment here, not displaying a progress bar to someone with a 5400-RPM laptop drive so that they feel like they’re accomplishing something.
  8. If the virtual machine’s primary purpose is to run an application that has its own replication technology, don’t use Hyper-V Replica. Examples: Active Directory and Microsoft SQL Server. Such applications will replicate themselves far more efficiently than Hyper-V Replica.
  9. If you’re using Hyper-V Replica, consider moving the VMs’ page files to their own virtual disk and excluding it from the replica job. If you have a small page file that doesn’t churn much, that might cost you more time and effort than you’ll recoup.
  10. If you’re using Hyper-V Replica, enable compression if you have spare CPU but leave it disabled if you have spare network. If you’re not sure, use compression.
  11. If you are shipping your Hyper-V Replica traffic across an encrypted VPN or keeping its traffic within secure networks, use Kerberos. SSL en/decryption requires CPU. Also, the asymmetrical nature of SSL encryption causes the encrypted data to be much larger than its decrypted source.


You must monitor your systems. Monitoring is not and has never been, an optional activity.

  1. Be aware of Hyper-V-specific counters. Many people try to use Task Manager in the management operating system to gauge guest CPU usage, but it just doesn’t work. The management operating system is a special-case virtual machine, which means that it is using virtual CPUs. Its Task Manager cannot see what the guests are doing.
  2. Performance Monitor has the most power of any built-in tool, but it’s tough to use. Look at something like Performance Analysis of Logs (PAL) tool, which understands Hyper-V.
  3. In addition to performance monitoring, employ state monitoring. With that, you no longer have to worry (as much) about surprise events like disk space or memory filling up. I like Nagios, as regular readers already know, but you can select from many packages.
  4. Take periodic performance baselines and compare them to earlier baselines


If you’re able to address a fair proportion of points from this list, I’m sure you’ll see a boost in Hyper-V performance. Don’t forget this list is not exhaustive and I’ll be adding to it periodically to ensure it’s as comprehensive as possible however if you think there’s something missing, let me know in the comments below and you may see the number 95 increase!

Get Involved on twitter: #How2HyperV

Get involved on twitter where we will be regularly posting excerpts from this article and engaging the IT community to help each other improve our use of Hyper-V. Got your own Hyper-V tips or tricks for boosting performance? Use the hashtag #How2HyperV when you tweet and share your knowledge with the world!

#How2HyperV Tweets

How to Perform Hyper-V Storage Migration

How to Perform Hyper-V Storage Migration

New servers? New SAN? Trying out hyper-convergence? Upgrading to Hyper-V 2016? Any number of conditions might prompt you to move your Hyper-V virtual machine’s storage to another location. Let’s look at the technologies that enable such moves.

An Overview of Hyper-V Migration Options

Hyper-V offers numerous migration options. Each has its own distinctive features. Unfortunately, we in the community often muck things up by using incorrect and confusing terminology. So, let’s briefly walk through the migration types that Hyper-V offers:

  • Quick migration: Cluster-based virtual machine migration that involves placing a virtual machine into a saved state, transferring ownership to another node in the same cluster, and resuming the virtual machine. A quick migration does not involve moving anything that most of us consider storage.
  • Live migration: Cluster-based virtual machine migration that involves transferring the active state of a running virtual machine to another node in the same cluster. A Live Migration does not involve moving anything that most of us consider storage.
  • Storage migration: Any technique that utilizes the Hyper-V management service to relocate any file-based component that belongs to a virtual machine. This article focuses on this migration type, so I won’t expand any of those thoughts in this list.
  • Shared Nothing Live Migration: Hyper-V migration technique between two hosts that does not involve clustering. It may or may not include a storage migration. The virtual machine might or might not be running. However, this migration type always includes ownership transfer from one host to another.

It Isn’t Called Storage Live Migration

I have always called this operation “Storage Live Migration”. I know lots of other authors call it “Storage Live Migration”. But, Microsoft does not call it “Storage Live Migration”. They just call it “Storage Migration”. The closest thing that I can find to “Storage Live Migration” in anything from Microsoft is a 2012 TechEd recording by Benjamin Armstrong. The title of that presentation includes the phrase “Live Storage Migration”, but I can’t determine if the “Live” just modifies “Storage Migration” or if Ben uses it as part of the technology name. I suppose I could listen to the entire hour and a half presentation, but I’m lazy. I’m sure that it’s a great presentation, if anyone wants to listen and report back.

Anyway, does it matter? I don’t really think so. I’m certainly not going to correct anyone that uses that phrase. However, the virtual machine does not necessarily need to be live. We use the same tools and commands to move a virtual machine’s storage whether it’s online or offline. So, “Storage Migration” will always be a correct term. “Storage Live Migration”, not so much. However, we use the term “Shared Nothing Live Migration” for virtual machines that are turned off, so we can’t claim any consistency.

What Can Be Moved with Hyper-V Storage Migration?

When we talk about virtual machine storage, most people think of the places where the guest operating system stores its data. That certainly comprises the physical bulk of virtual machine storage. However, it’s also only one bullet point on a list of multiple components that form a virtual machine.

Independently, you can move any of these virtual machine items:

  • The virtual machine’s core files (configuration in xml or .vmcx, .bin, .vsv, etc.)
  • The virtual machine’s checkpoints (essentially the same items as the preceding bullet point, but for the checkpoint(s) instead of the active virtual machine)
  • The virtual machine’s second-level paging file location. I have not tested to see if it will move a VM with active second-level paging files, but I have no reason to believe that it wouldn’t
  • Virtual hard disks attached to a virtual machine
  • ISO images attached to a virtual machine

We most commonly move all of these things together. Hyper-V doesn’t require that, though. Also, we can move all of these things in the same operation but distribute them to different destinations.

What Can’t Be Moved with Hyper-V Storage Migration?

In terms of storage, we can move everything related to a virtual machine. But, we can’t move the VM’s active, running state with Storage Migration. Storage Migration is commonly partnered with a Live Migration in the operation that we call “Shared Nothing Live Migration”. To avoid getting bogged down in implementation details that are more academic than practical, just understand one thing: when you pick the option to move the virtual machine’s storage, you are not changing which Hyper-V host owns and runs the virtual machine.

More importantly, you can’t use any Microsoft tool-based technique to separate a differencing disk from its parent. So, if you have an AVHDX (differencing disk created by the checkpointing mechanism) and you want to move it away from its source VHDX, Storage Migration will not do it. If you instruct Storage Migration to move the AVHDX, the entire disk chain goes along for the ride.

Uses for Hyper-V Storage Migration

Out of all the migration types, storage migration has the most applications and special conditions. For instance, Storage Migration is the only Hyper-V migration type that does not always require domain membership. Granted, the one exception to the domain membership rule won’t be very satisfying for people that insist on leaving their Hyper-V hosts in insecure workgroup mode, but I’m not here to please those people. I’m here to talk about the nuances of Storage Migration.

Local Relocation

Let’s start with the simplest usage: relocation of local VM storage. Some situations in this category:

  • You left VMs in the default “C:\ProgramData\Microsoft\Windows\Hyper-V” and/or “C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks” locations and you don’t like it
  • You added new internal storage as a separate volume and want to re-distribute your VMs
  • You have storage speed tiers but no active management layer
  • You don’t like the way your VMs’ files are laid out
  • You want to defragment VM storage space. It’s a waste of time, but it works.

Network Relocation

With so many ways to do network storage, it’s nearly a given that we’ll all need to move a VHDX across ours at some point. Some situations:

  • You’re migrating from local storage to network storage
  • You’re replacing a SAN or NAS and need to relocate your VMs
  • You’ve expanded your network storage and want to redistribute your VMs

Most of the reasons listed under “Local Relocation” can also apply to network relocation.

Cluster Relocation

We can’t always build our clusters perfectly from the beginning. For the most part, a cluster’s relocation needs list will look like the local and network lists above. A few others:

  • Your cluster has new Cluster Shared Volumes that you want to expand into
  • Existing Cluster Shared Volumes do not have a data distribution that does not balance well. Remember that data access from a CSV owner node is slightly faster than from a non-owner node

The reasons matter less than the tools when you’re talking about clusters. You can’t use the same tools and techniques to move virtual machines that are protected by Failover Clustering under Hyper-V as you use for non-clustered VMs.

Turning the VM Off Makes a Difference for Storage Migration

You can perform a very simple experiment: perform a Storage Migration for a virtual machine while it’s on, then turn it off and migrate it back. The virtual machine will move much more quickly while it’s off. This behavior can be explained in one word: synchronization.

When the virtual machine is off, a Storage Migration is essentially a monitored file copy. The ability of the constituent parts to move bits from source to destination sets the pace of the move. When the virtual machine is on, all of the rules change. The migration is subjected to these constraints:

  • The virtual machine’s operating system must remain responsive
  • Writes must be properly captured
  • Reads must occur from the most appropriate source

Even if the guest operating does not experience much activity during the move, that condition cannot be taken as a constant. In other words, Hyper-V needs to be ready for it to start demanding lots of I/O at any time.

So, the Storage Migration of a running virtual machine will always take longer than the Storage Migration of a virtual machine in an off or saved state. You can choose the convenience of an online migration or the speed of an offline migration.

Note: You can usually change a virtual machine’s power state during a Storage Migration. It’s less likely to work if you are moving across hosts.

How to Perform Hyper-V Storage Migration with PowerShell

The nice thing about using PowerShell for Storage Migration: it works for all Storage Migration types. The bad thing about using PowerShell for Storage Migration: it can be difficult to get all of the pieces right.

The primary cmdlet to use is Move-VMStorage. If you will be performing a Shared Nothing Live Migration, you can also use Move-VM. The parts of Move-VM that pertain to storage match Move-VMStorage. Move-VM has uses, requirements, and limitations that don’t pertain to the topic of this article, so I won’t cover Move-VM here.

A Basic Storage Migration in PowerShell

Let’s start with an easy one. Use this when you just want all of a VM’s files to be in one place:

This will move the virtual machine named testvm so that all of its components reside under the C:\LocalVMs folder. That means:

  • The configuration files will be placed in C:\LocalVMs\Virtual Machines
  • The checkpoint files will be placed in C:\LocalVMs\Snapshots
  • The VHDXs will be placed in C:\LocalVMs\Virtual Hard Disks
  • Depending on your version, an UndoLog Configuration folder will be created if it doesn’t already exist. The folder is meant to contain Hyper-V Replica files. It may be created even for virtual machines that aren’t being replicated.

Complex Storage Migrations in PowerShell

For more complicated move scenarios, you won’t use the DestinationStoragePath parameter. You’ll use one or more of the individual component parameters. Choose from the following:

  • VirtualMachinePath: Where to place the VM’s configuration files.
  • SnapshotFilePath: Where to place the VM’s checkpoint files (again, NOT the AVHDXs!)
  • SmartPagingFilePath: Where to place the VM’s smart paging files
  • Vhds: An array of hash tables that indicate where to place individual VHD/X files.

Some notes on these items:

  • You are not required to use all of these parameters. If you do not specify a parameter, then its related component is left alone. Meaning, it doesn’t get moved at all.
  • If you’re trying to use this to get away from those auto-created Virtual Machines and Snapshots folders, it doesn’t work. They’ll always be created as sub-folders of whatever you type in.
  • It doesn’t auto-create a Virtual Hard Disks folder.
  • If you were curious whether or not you needed to specify those auto-created subfolders, the answer is: no. Move-VMStorage will always create them for you (unless they already exist).
  • The VHDs hash table is the hardest part of this whole thing. I’m usually a PowerShell-first kind of guy, but even I tend to go to the GUI for Storage Migrations.

The following will move all components except VHDs, which I’ll tackle in the next section:

Move-VMStorage’s Array of Hash Tables for VHDs

The three …FilePath parameters are easy: just specify the path. The Vhds parameter is tougher. It is one or more hash tables inside an array.

First, the hash tables. A hash table is a custom object that looks like an array, but each entry has a unique name. The hash tables that Vhds expects have a SourceFilePath entry and a DestinationFilePath entry. Each must be fully-qualified for a file. A hash table is contained like this: @{ }. The name of an entry and its value are joined with an =. Entries are separated by a ; So, if you want to move the VHDX named svtest.vhdx from \\svstore\VMs to C:\LocalVMs\testvm, you’d use this hash table:

Reading that, you might ask (quite logically): “Can I change the name of the VHDX file when I move it?” The answer: No, you cannot. So, why then do you need to enter the full name of the destination file? I don’t know!

Next, the arrays. An array is bounded by @( ). Its entries are separated by commas. So, to move two VHDXs, you would do something like this:

I broke that onto multiple lines for legibility. You can enter it all on one line. Note where I used parenthesis and where I used curly braces.

Tip: To move a single VHDX file, you don’t need to do the entire array notation. You can use the first example with Vhds.

A Practical Move-VMStorage Example with Vhds

If you’re looking at all that and wondering why you’d ever use PowerShell for such a thing, I have the perfect answer: scripting. Don’t do this by hand. Use it to move lots of VMs in one fell swoop. If you want to see a plain example of the Vhds parameter in action, the Get-Help examples show one. I’ve got a more practical script in mind.

The following would move all VMs on the host. All of their config, checkpoint, and second-level paging files will be placed on a share named “\\vmstore\slowstorage”. All of their VHDXs will be placed on a share named “\\vmstore\faststorage”. We will have PowerShell deal with the source paths and file names.

I used splatting for the parameters for two reasons: 1, legibility. 2, to handle VMs without any virtual hard disks.

How to Perform Hyper-V Storage Migration with Hyper-V Manager

Hyper-V Manager can only be used for non-clustered virtual machines. It utilizes a wizard format. To use it to move a virtual machine’s storage:

  1. Right-click on the virtual machine and click Move.
  2. Click Next on the introductory page.
  3. Change the selection to Move the virtual machine’s storage (the same storage options would be available if you moved the VM’s ownership, but that’s not part of this article)
  4. Choose how to perform the move. You can move everything to the same location, you can move everything to different locations, or you can move only the virtual hard disks.
  5. What screens you see next will depend on what you chose. We’ll cover each branch.

If you opt to move everything to one location, the wizard will show you this simple page:


If you choose the option to Move the virtual machine’s data to different locations, you will first see this screen:


For every item that you check, you will be given a separate screen where you indicate the desired location for that item. The wizard uses the same screen for these items as it does for the hard-disks only option. I’ll show its screen shot next.

If you choose Move only the virtual machine’s virtual hard disks, then you will be given a sequence of screens where you instruct it where to move the files. These are the same screens used for the individual components from the previous selection:


After you make your selections, you’ll be shown a summary screen where you can click Finish to perform the move:


How to Perform Hyper-V Storage Migration with Failover Cluster Manager

Failover Cluster Manager uses a slick single-screen interface to move storage for cluster virtual machines. To access it, simply right-click a virtual machine, hover over Move, and click Virtual Machine Storage. You’ll see the following screen:


If you just want to move the whole thing to one of the display Cluster Shared Volumes, just drag and drop it down to that CSV in the Cluster Storage heading at the lower left. You can drag and drop individual items or the entire VM. The Destination Folder Path will be populated accordingly.

As you can see in mine, I have all of the components except the VHD on an SMB share. I want to move the VHD to be with the rest. To get a share to show up, click the Add Share button. You’ll get this dialog:


The share will populate underneath the CSVs in the lower left. Now, I can drag and drop that file to the share. View the differences:


Once you have the dialog the way that you like it, click Start.

8 Most Important Announcements from Microsoft Ignite 2017

8 Most Important Announcements from Microsoft Ignite 2017

Last week saw us close the door on Microsoft Ignite 2017, and while the conference came and went in a blur, there was no lack of information or amazing reveals from Microsoft. While this conference serves as a great way to stay informed on all the new things that Microsoft is working on, I also find that it is a good way to get an overall sense of the company’s overall direction. With that in mind, I wanted to not only talk about some of my favorite reveals from the week but also discuss my take on Microsoft’s overall direction.

Microsoft Ignite 2017 - most important announcements

My take on the week from an Infrastructure Engineering Perspective

To put things simply….. things are changing, and they’re changing in a big way. I’ve had this gut feeling stirring for some time that the way we work with VMs and virtualization was changing, and the week of Ignite was a major confirmation of that. This is not to mention the continued shift from the on-premise model we’re used to, to the new cloud (Public, Private, and Hybrid) model that things are moving too.

It’s very clear that Microsoft is adopting what I would call the “Azure-Everywhere” approach. Sure, you’ve always been able to consume Azure using what Microsoft has publicly available, but things really changed when Azure Stack is put into the mix. Microsoft Azure Stack (MAS) is officially on the market now, and the idea of having MAS in datacenters around the world is an interesting prospect. What I find so interesting about it, is the fact that management of MAS onsite is identical to managing Azure. You use Azure Resource Manager and the same collection of tools to manage both. Pair that with the fact that Hyper-V is so abstracted and under-the-hood in MAS that you can’t even see it, and you’ve got a recipe for major day-to-day changes for infrastructure administrators.

Yes, we’ve still got Windows Server 2016, and the newly announced Honolulu management utility, but If I look out 5, or even 10 years, I’m not sure I see us working with Windows Server anymore in the way that we do so today. I don’t think VM usage will be as prevalent then as it is today either. After last week, I firmly believe that containers will be the “new virtual machine”. I think VMs will stay around for legacy workloads, and for workloads that require additional layers of isolation, but after seeing containers in action last week, I’m all in on that usage model.

We used to see VMs as this amazing cost-reducing technology, and it was for a long time. However, I saw containers do to VMs, what VMs did to physical servers. I attended a session on moving workloads to a container based model, and MetLife was on stage talking about moving some of their infrastructure to containers. In doing so they achieved:

  • -70% reduction in the number of VMs in the environment
  • -67% reduction in needed CPU cores
  • -66% reduction in overall cost of ownership

Those are amazing numbers that nobody can ignore. Given this level of success with containers, I see the industry moving to that deployment model from VMs over the next several years. As much as it pains me to say it, virtual machines are starting to look very “legacy”, and we all need to adjust our skill sets accordingly.

Big Reveals

As you know, Ignite is that time of year where Microsoft makes some fairly large announcements, and below I’ve compiled a list of some of my favorite. While this is by no means a comprehensive list,  but I feel these represent what our readers would find most interesting. Don’t agree? That’s fine! Just let me know what you think were the most important announcements in the comments. Let’s get started.

8. New Azure Exams and Certifications!

With new technologies, come new things to learn, and as such there are 3 new exams on the market today for Azure Technologies.

  • Azure Stack Operators – Exam 537: Configuring and Operating a Hybrid Cloud with Microsoft Azure Stack
  • For Azure Solutions Architects – Exam 539: Managing Linux Workloads on Azure
  • For Azure DevOps – Exam 538: Implementing Microsoft Azure DevOps Solutions

If you’re interested in pursuing any of these (Which I would Highly Recommend) then you can get more information on them at this link.

7. SQL Server 2017 is now Available

Normally I wouldn’t make much of a fuss about SQL Server as I’m not much of a SQL guy myself, but Microsoft did something amazing with this release. SQL Server 2017 will run on Windows, Linux, and inside of Docker Containers. Yes, you read correctly. SQL Server 2017 will run on Linux and inside of docker containers, which opens up a whole new avenue of providing SQL workloads. Exciting times indeed!

6. Patch Management from the Azure Portal

Ever wanted to have WSUS available from the Azure portal? Now you have it. You can easily view, and deploy patches for your Azure based workloads directly from the Azure portal. This includes Linux VMs as well, which is great news as more and more admins are finding themselves managing Linux workloads these days!

5. PowerShell now Available in Azure CLI.

When Azure CLI was announced and released, many people were taken aback at the lack of PowerShell support. This was done for a number of reasons that I won’t get into in this article, but regardless, it has been added in now. It is now possible with Azure CLI to deploy a VM with a single PowerShell cmdlet and more. So, get those scripts ready!

4. Azure File Sync in Preview

I know many friends and colleagues that have been waiting for something like this. You can essentially view this as next-generation DFS. (Though it doesn’t use the same technology). It essentially allows you to sync your on-premise file servers with an Azure Files account for distributed access to the stored information around the globe.

3. Quality of Life Improvements for Windows Containers

While there were no huge reveals in the container space, Windows Server 1709 was announced and contains a lot of improvements and optimizations for running containers on Windows Server. This includes things like smaller images and support for Linux Containers running on Windows Server. I did an Interview with Taylor Brown from the Containers team, which you can view below for more information.

2. Nested Virtualization in Azure for Production Workloads

Yes, I know, nested virtualization in Azure has been announced for some time. However, what I found different was Microsoft’s Insistence that it could also be used for production workloads. During Scott Guthrie’s Keynote, Corey Sanders actually demonstrated the use of the M-Series (Monster) VM in Azure being used to host production workloads with nested VMs. While not ideal in every scenario obviously, this is simply another tool that we have at our disposal for added flexibility in our day-to-day operations.

If you’re interested, I actually interviewed Rick Claus from the Azure Compute team about this. That Interview can be seen below

1. Project Honolulu

This one is for the folks that are still strictly interested in only the on-prem stuff. Microsoft revealed and showed us the new Project Honolulu management utility for on-premise workloads. Honolulu, takes the functionality of all the management tools and MMC snap-ins that we’ve been using for years and packages them up into a nice easy to use web-UI. It’s worth a look if you haven’t seen it yet. We even have a nice article on our blog about it if you’re interested in reading more!


As I mentioned, this was by no means a comprehensive list, but we’ll be talking about items (From this list and some not mentioned) from Ignite on our blogs for some time. So, be sure to keep an eye on our blog if you’re interested in more information.

Additionally, if you attended Microsoft Ignite, and you saw a feature or product you think is amazing that is not listed above, be sure to let us know in the comments section below!

Hyper-V 2016 Host Mode: GUI vs Core

Hyper-V 2016 Host Mode: GUI vs Core

Choice is a good thing, right? Well… usually. Sometimes, choice is just confusing. With most hypervisors, you get what you get. With Hyper-V, you can install in three different ways, and that’s just for the server hypervisor. In this article, we’ll balance the pros and cons of your options with the 2016 SKUs.

Server Deployment Options for Hyper-V

As of today, you can deploy Hyper-V in one of four packages.

Nano Server

When 2016 initially released, it brought a completely new install mode called “Nano”. Nano is little more than the Windows Server kernel with a tiny handful of interface bits attached. You then plug in the roles and features that you need to get to the server deployment that you want. I was not ever particularly fond of the idea of Hyper-V on Nano for several reasons, but none of them matter now. Nano Server is no longer supported as a Hyper-V host. It currently works, but that capability will be removed in the next iteration. Part of the fine print about Nano that no one reads includes the requirement that you keep within a few updates of current. So, you will be able to run Hyper-V on Nano for a while, but not forever.

If you currently use Nano for Hyper-V, I would start plotting a migration strategy now. If you are considering Nano for Hyper-V, stop.

Hyper-V Server

Hyper-V Server is the product name given to the free distribution vehicle for Hyper-V. You’ll commonly hear it referred to as “Hyper-V Core”, although that designation is both confusing and incorrect. You can download Hyper-V Server as a so-called “evaluation”, but it never expires.

A word of advice: Hyper-V Server includes a legally-binding license agreement. Violation of that licensing agreement subjects you to the same legal penalties that you would face for violating the license agreement of a paid operating system. Hyper-V Server’s license clearly dictates that it can only be used to host and maintain virtual machines. You cannot use it as a file server or a web server or anything else. Something that I need to make extremely clear: the license agreement does not provide special allowances for a test environment. I know of a couple of blog articles that guide you to doing things under the guise of “test environment”. That’s not OK. If it’s not legal in a production environment, it doesn’t magically become legal in a test environment.

Windows Server Core

When you boot to the Windows Server install media, the first listed option includes “Core” in the name. That’s not an accident; Microsoft wants you to use Core mode by default. Windows Server Core excludes the primary Windows graphical interface and explorer.exe. Some people erroneously believe that means that no graphical applications can be run at all. Applications that use the Explorer rendering engine will not function (such as MMC), but the base Windows Forms libraries and mechanisms exist.

Windows Server with GUI

I doubt that the GUI mode of Windows Server needs much explanation. You have the same basic graphical interface as Windows 10 with some modifications that make it more appropriate for a server environment. When you install from 2016 media, you will see this listed as (Desktop Experience).

The Pros and Cons of the Command-line and Graphical Modes for Hyper-V

I know that things would be easier if I would just tell you what to do. If I knew you and knew your environment, I might do that. I prefer giving you the tools and knowledge to make decisions like this on your own, though. So, we’ll complement our discussion with a pros and cons list of each option. After the lists, I’ll cover some additional guidelines and points to consider.

Hyper-V Server Pros and Cons

If you skipped the preamble, remember that “Hyper-V Server” refers to the completely free SKU that you can download at any time.

Pros of Hyper-V Server:

  • Never requires a licensing fee
  • Never requires activation
  • Smallest deployment
  • Smallest “surface area” for attacks
  • Least memory usage by the management operating system
  • Fewest patch needs
  • Includes all essential features for running Hyper-V (present, not necessarily enabled by default):
    • Hyper-V hypervisor
    • Hyper-V PowerShell interface
    • Cluster membership
    • Domain membership
    • Hyper-V Replica membership
    • Remote Desktop Virtual Host role for VDI deployments
    • RemoteFx (automatic with RDVH role)

Cons of Hyper-V Server:

  • Cannot provide Automatic Virtual Machine Activation
  • Cannot provide deduplication features
  • Impossible to enable the Windows Server GUI
  • Software manufacturers may refuse to support their software on it
  • Third-party support operations, such as independent consulting firms, may not have any experience with it
  • Switching to Windows Server requires a complete reinstall
  • Difficult to manage hardware

Hyper-V in Windows Server Core Pros and Cons

If you’ve seen the term “Hyper-V Core”, that probably means “Hyper-V Server”. This section covers the authentic Windows Server product installed in Core mode.

Pros of Windows Server Core for Hyper-V:

  • Microsoft recommends Windows Server Core for Hyper-V
  • Receives feature updates on the quickest schedule (look toward the bottom of the link in the preceding bullet)
  • Comparable deployment size to Hyper-V Server
  • Comparable surface area to Hyper-V Server
  • Comparable memory usage to Hyper-V Server
  • Comparable patch requirements to Hyper-V Server
  • Allows almost all roles and features of Windows Server
  • Can provide Automatic Activation for Windows Server in VMs (Datacenter Edition only)

Cons of Windows Server Core for Hyper-V:

  • Impossible to enable the Windows Server GUI
  • Must be licensed and activated
  • Upgrading to the next version requires paying for that version’s license, even if you will wait to deploy newer guests
  • Software manufacturers may refuse to support their software on it
  • Third-party support operations, such as independent consulting firms, may not have any experience with it
  • Difficult to manage hardware

Hyper-V in Windows Server GUI Pros and Cons

We saved what many consider the “default” option for last.

Pros of Windows Server with GUI for Hyper-V:

  • Familiar Windows GUI
  • More tools available, both native and third party
  • Widest support from software manufacturers and consultants
  • Easiest hardware management
  • Valid environment for all Windows Server roles, features, and software
  • Can provide Automatic Activation for Windows Server in VMs (Datacenter Edition only)

Cons of Windows Server with GUI for Hyper-V:

  • Familiarity breeds contempt
  • Slowest feature roll-out cycle (see the bottom of this article)
  • Largest attack surface, especially with explorer.exe
  • Largest deployment size
  • Largest memory usage
  • Largest patch requirements
  • Must be licensed and activated
  • Upgrading to the next version requires paying for that version’s license, even if you will wait to deploy newer guests

Side-by-Side Comparison of Server Modes for Hyper-V

Two items appear in every discussion of this topic: disk space and memory usage. I thought that it might be enlightening to see the real numbers. So, I built three virtual machines running Hyper-V in nested mode. The first contains Hyper-V Server, the second contains Windows Server Datacenter Edition in Core mode, and the third contains Windows Server Datacenter Edition in GUI mode. I have enabled Hyper-V in each of the Windows Server systems and included all management tools and subfeatures ( Add-WindowsFeature -Name Hyper-V -IncludeAllSubFeature -IncludeManagementTools). All came from the latest MSDN ISOs. None are patched. None are on the network.

Disk Usage Comparison of the Three Modes

I used the following PowerShell command to determine the used space: '{0:N0}' -f (Get-WmiObject -Class Win32_LogicalDisk | ? DeviceId -eq 'C:' | % {$_.Size - $_.FreeSpace}).

Deployment Mode Used Disk Space (bytes)
Hyper-V Server 2016 6,044,270,592
Windows Server 2016 Datacenter Edition in Core mode 7,355,858,944
Windows Server 2016 Datacenter Edition in GUI mode 10,766,614,528

For shock value, the full GUI mode of Windows Server adds 78% space utilization above Hyper-V Server 2016 and 46% space utilization above Core mode. That additional space amounts to less than 5 gigabytes. If 5 gigabytes will make or break your deployment, you’ve got other issues.

Memory Usage Comparison of the Three Modes

We’ll start with Task Manager while logged on:


These show what we expect: Hyper-V Server uses the least amount of memory, Windows Server Core uses a bit more, and Windows Server with GUI uses a few ticks above both. However, I need to point out that these charts show a more dramatic difference than you should encounter in reality. Since I’m using nested VMs to host my sample systems, I only gave them 2 GB total memory apiece. The consumed memory distance between Hyper-V Server and Windows Server with GUI weighs in at a whopping .3 gigabytes. If that number means a lot to you in your production systems, then you’re going to have other problems.

But that’s not the whole story.

Those numbers were taken from Task Manager while logged on to the systems. Good administrators log off of servers as soon as possible. What happens, then, when we log off? To test that, I had to connect each VM to the network and join the domain. I then ran: Get-WmiObject Win32_OperatingSystem | select FreePhysicalMemory with the ComputerName switch against each of the hosts. Check out the results:

Deployment Mode Free Memory (MB)
Hyper-V Server 2016 1,621,148
Windows Server 2016 Datacenter Edition in Core mode 1,643,060
Windows Server 2016 Datacenter Edition in GUI mode 1,558,744

Those differences aren’t so dramatic, are they? Windows Server Core even has a fair bit more free memory than Hyper-V Server… at that exact moment in time. If you don’t have much background in memory management, especially in terms of operating systems, then keep in mind that memory allocation and usage can seem very strange.

The takeaway: memory usage between all three modes is comparable when they are logged off.

Hyper-V and the “Surface Area” Argument

Look at the difference in consumed disk sizes between the three modes. Those extra bits represent additional available functionality. Within them, you’ll find things such as Active Directory Domain Services and IIS. So, when we talk about choosing between these modes, we commonly point out that all of these things add to the “attack surface”. We try to draw the conclusion that using a GUI-less system increases security.

First part: Let’s say that a chunk of malware injects itself into one of the ADDS DLLs sitting on your Windows Server host running Hyper-V. What happens if you never enable ADDS on that system? Well, it’s infected, to be sure. But, in order for any piece of malware to cause any harm, something eventually needs to bring it into memory and execute it. But, you know that you’re not supposed to run ADDS on a Hyper-V host. Philosophical question: if malware attacks a file and no one ever loads it, is the system still infected? Hopefully, you’ve got a decent antimalware system that will eventually catch and clean it, so you should be perfectly fine.

On one hand, I don’t want to downplay malware. I would never be comfortable with any level of infection on any system. On the other hand, I think common sense host management drastically mitigates any concerns. I don’t believe this is enough of a problem to carry a meaningful amount of weight in your decision.

Second part: Windows Server runs explorer.exe as its shell and includes Internet Explorer. Attackers love those targets. You can minimize your exposure by, you know, not browsing the Internet from a server, but you can’t realistically avoid using explorer.exe on a GUI system. However, as an infrastructure system, you should be able to safely instruct your antimalware system to keep a very close eye on Explorer’s behavior and practice solid defensive techniques to prevent malware from reaching the system.

Overall takeway from this section: Explorer presents the greatest risk. Choose the defense-in-depth approach of using Hyper-V Server or Windows Server Core, or choose to depend on antimalware and safe operating techniques with the Windows Server GUI.

Hyper-V and the Patch Frequency Non-Issue

Another thing that we always try to bring into these discussions is the effect of monthly patch cycles. Windows Server has more going on than Hyper-V Server, so it gets more patches. From there, we often make the argument that more patches equals more reboots.

A little problem, though. Let’s say that Microsoft releases twelve patches for Windows Server and only two apply to Hyper-V Server. One of those two patches requires a reboot. In that case, both servers will reboot. One time. So, if we get hung up on downtime over patches, then we gain nothing. I believe that, in previous versions, the downtime math did favor Hyper-V Server a few times. However, patches are now delivered in only a few omnibus packages instead of smaller targeted patches. So, I suspect that we will no longer be able to even talk about reboot frequency.

One part of the patching argument remains: with less to patch, fewer things can go wrong from a bad patch. However, this argument faces the same problem as the “surface area” non-issue. What are you using on your Windows Server system that you wouldn’t also use on a Hyper-V Server system? If you’re using your Windows Server deployment correctly, then your patch risks should be roughly identical.

Most small businesses will patch their Hyper-V systems via automated processes that occur when no one is around. Larger businesses will cluster Hyper-V hosts and allow Cluster Aware Updating to prevent downtime.

Overall takeaway from this section: patching does not make a convincing argument in any direction.

Discussions: Choosing Between Core and GUI for Hyper-V

Now you’ve seen the facts. You’ve seen a few generic arguments for the impact level of two of those facts. If you still don’t know what to do, that’s OK. Let’s look at some situational points.

A Clear Case for Hyper-V on Windows Server Full GUI

If you’re in a small environment with only a single physical server, go ahead and use the full GUI.

Why? Some reasons:

  • It is not feasible to manage Hyper-V without any GUI at all. I advocate for PowerShell usage as strongly as anyone else, but sometimes the GUI is a better choice. In a multi-server environment, you can easily make a GUI-less system work because you have at least one GUI-based management system somewhere. Without that, GUI-less demands too much.
  • The world has a shortage of Windows Server administrators that are willing and able to manage a GUI-less system. You will have difficulty hiring competent help at a palatable price.
  • Such a small shop will not face the density problems that justify the few extra resources saved by the GUI-less modes.
  • The other issues that I mentioned are typically easier to manage in a small environment than in a large environment.
  • A GUI system will lag behind Core in features, but Hyper-V is quite feature-complete for smaller businesses. You probably won’t miss anything that really matters to you.
  • If you try Hyper-V Server or Windows Server Core and decide that you made a mistake, you have no choice but to reinstall. If you install the GUI and don’t want to use it, then don’t use it — switch to remote management practices. You won’t miss out on anything besides the faster feature release cycle.

We can make some very good arguments for a GUI-less system, but none are strong enough to cause crippling pain for a small business. When the GUI fits, use it.

A Clear Case for Hyper-V Server

Let’s switch gears completely. Let’s say that:

  • You’re a PowerShell whiz
  • You’re a cmd whiz
  • You run a lot of Linux servers
  • Your Windows Servers (if any) are all temporary testing systems

Hyper-V Server will suit you quite well.

Everyone Else

If you’re somewhere in the middle of the above two cases, I think that Microsoft’s recommendation of Windows Server Core with Hyper-V fits perfectly. The parts that stand out to me:

  • Flexibility: Deduplication has done such wonders for me in VDI that I’m anxious to see how I can apply it to server loads. In 2012 R2, server guests were specifically excluded; VDI only. Server 2016 maintains the same wording in the feature setup, but I can’t find a comparable statement saying that server usage is verboten in 2016. I could also see a case for building a nice VM management system in ASP.Net and hosting it locally with IIS — you can’t do that in Hyper-V Server.
  • Automatic Virtual Machine Activation. Who loves activation? Nobody loves activation! Let the system deal with that.
  • Security by terror: Not all server admins are created equally. I find that the really incompetent ones won’t even log on to a Server Core/Hyper-V Server system. That means that they won’t put them at risk.
  • Remote management should be the default behavior. If you don’t currently practice remote management, there’s no time like the present! You can dramatically reduce the security risk to any system by never logging on to its console, even by RDP.

You can manage Hyper-V systems from a Windows 10 desktop with RSAT. It’s not entirely without pain, though:

  • Drivers! Ouch! Microsoft could help us out by providing a good way to use Device Manager remotely. We should not let driver manufacturers off the hook easily, though. Note: Honolulu is coming to reduce some of that pain.
  • Not everyone that requires the GUI is an idiot. Some of them just haven’t yet learned. Some have learned their way around PowerShell but don’t know how to use it for Hyper-V. You like taking vacations sometimes, don’t you?
  • Crisis mode when you don’t know what’s wrong can be a challenge. It’s one thing to keep the top spinning; it’s another to get it going when you can’t see what’s holding it down. However, these problems have solutions. It’s a pain, but a manageable one.

I’m not here to make the decision for you. You now have enough information to make an informed decision.

Project ‘Honolulu’: What you need to know

Project ‘Honolulu’: What you need to know

The biggest problem with Hyper-V isn’t Hyper-V at all. It’s the management experience. We’ve all had our complaints about that, so I don’t think a rehash is necessary. Thing is, Hyper-V is far from alone. Microsoft has plenty of management issues across its other infrastructure roles and features as well. Enter Project ‘Honolulu’: an attempt to unify and improve the management experience for Microsoft’s infrastructure offerings.

Before I get very far into this, I want one thing to be made abundantly clear: the Honolulu Project is barely out of its infancy. As I write this, it is exiting private preview. The public beta bits aren’t even published yet.

With that said, unless many things change dramatically between now and release, this is not the Hyper-V management solution that you have been waiting for. At its best, it has a couple of nice touches. In a few cases, it is roughly equivalent to what we have now. For most things, it is worse than what we have available today. I hate to be so blunt about it because I believe that Microsoft has put a great deal of effort into Honolulu. However, I also feel like they haven’t been paying much attention to the complaints and suggestions the community has made regarding the awful state of Hyper-V management tools.

What is Project ‘Honolulu’

When you look at Honolulu, it will appear something like an Azure-ified Server Manager. It adopts the left-to-right layouts that the Azure tools use, as opposed to the up-and-down scrolling that we humans and our mice are accustomed to.

Thou shalt not use Honolulu in a window

Thou shalt not use Honolulu in a window

This sort of thing is normative for the Azure tools. If you have a 50″ 4k screen and nothing else to look at, I’m sure that it looks wonderful. If you are using VMConnect or one of those lower resolution slide-out monitors that are still common in datacenters, then you might not enjoy the experience. And yes, the “<” icon next to Tools means that you can collapse that panel entirely. It doesn’t help much. I don’t know when it became passé for columns to be resizable and removable. Columns should be resizable and removable.

As you see it in that screenshot, Honolulu is running locally. It can also run in a gateway mode on a server. You can then access it from a web browser from other systems and devices.

Requirements for Running Project ‘Honolulu’

For the Honolulu Project itself, you can install on:

  • Windows 10
  • Windows Server 2012 through 2016

On a Windows 10 desktop or a Server 2012 system, it will only be accessible locally.

If you install on a server 2012 R2 through 2016 SKU, it will operate in the aforementioned gateway mode. You just open a web browser to that system on whatever port you configure, ex: https://managementmentsystem:6516. You will be prompted for credentials.

When you provide credentials to Honolulu, the systems that you connect to will be associated with your account. If you connect to Honolulu with a different user account, it will not display any of the servers that were chosen under a different account. Each need to be set up separately. You can import lists to reduce the pain.

Note: As it stands right now, I cannot get Honolulu work on a 2012 R2 system. It will open, but then refuses to connect to any server in my organization. I am actively working on this problem and will report back if a solution can be found. That’s one of the dangers of using early software, not a lifelong condemnation of the product.

Requirements for Targets of Honolulu

The target system(s) must be a Server SKU 2012 through 2016. It/they must have Windows Management Framework 5 or higher loaded. The easiest way to tell is by opening a PowerShell prompt and running $PSVersionTable. The PowerShell version and the Windows Management Framework version will always be the same. It also helps if you can verify that you can connect from the management system to the target with Enter-PSSession.

The following screenshot shows an example. I first tested that my management system has the correct version. Then I connected to my target and checked the WMF version there. I should have no problems setting up the first system to run Project Honolulu to connect to the second system.

running project honolulu

If you are running all of the systems in the same domain, then this will all “just work”. I’m not sure yet how cross-domain authentication works. If you’ve decided that security is unimportant and you’re running your Hyper-V host(s) in workgroup mode, then you will need to configure TrustedHosts on the gateway system.

Requirements for Viewing Project ‘Honolulu’

Honolulu presents its views via HTML 5 web pages. Edge and Chrome work well. Internet Explorer doesn’t work at all:


I think it will be interesting to see how that plays out in the enterprise. Windows 10 isn’t exactly the best corporate player, so several organizations are hanging on to Windows 7. Others are moving to Windows 10, but opting for the Long-Term Servicing Branch (LTSB). LTSB doesn’t include Edge. So, is Microsoft (inadvertently?) pushing people toward Google Chrome?

Connecting to a Target Server in Honolulu

When you first start up Honolulu, you have little to look at:


Click the + Add link to get started adding systems. Warning: If you’re going to add clusters, do that following the instructions in the next section. Only follow this for stand-alone hosts.

Type the name of a system to connect to, and it will automatically start searching. Hopefully, it will find the target. You can click the Submit button whether it can find it or not.

A working system:


A non-working system:


As you can see in the links, you can also Import Servers. For this, you need to supply a text file that contains a list of target servers.


Connecting to a Target Cluster in Honolulu

Honolulu starts out in “Server Manager” mode, so it will only connect to servers. If you try to connect it to a failover cluster in Server Manager mode, it will pick up the owning node instead. In order to connect to a failover cluster, you need to switch the mode.

At the top of the window, find the Server Manager heading. Drop that down and select Failover Cluster Manager.


Now, add clusters with the + Add button. When it detects the cluster, it will also prompt you to add the nodes as members of Server Manager:


Windows Management Framework Error for Honolulu

As mentioned in the beginning, every target system needs to have at least Windows Management Framework version 5 installed. If a target system does not meet that requirement, Honolulu will display that status:


The Really Quick Tour for Honolulu

I focus on Hyper-V and I’m certain that dozens of other Honolulu articles are already published (if not more). So, let’s burn through the non-Hyper-V stuff really fast.

Right-click doesn’t do anything useful anywhere in Honolulu. Train yourself to use only the left mouse button.

Server Manager has these sections:

  • Overview: Shows many of the things that you can see in Computer Properties. Also has several real-time performance charts, such as CPU and memory. For 2016+ you can see disk statistics. I like this page in theory, but the execution is awful. It assumes that you always want to see the basic facts about a host no matter what and that you have a gigantic screen resolution. My VMConnect screen is set to 1366×768 and I can’t even see a single performance chart in its entirety:
  • Certificates: No more dealing with all the drama of manually adding the certificates snap-in! Also, you can view the computer and user certificates at the same time! Unfortunately, it doesn’t look like you can request a new certificate, but most other functionality seems to be here.
  • Devices: You can now finally see the devices installed on a Server Core/Hyper-V Server installation. You can’t take any action except Disable, unfortunately. It’s still better than what we had.
  • Events: Event Viewer, basically.
  • Files: Mini-File Explorer in your browser! You can browse the directory structure and upload/download files. You can view properties, but you can’t do anything with shares or permissions.
  • Firewall: Covers the most vital parts of firewall settings (profile en/disabling and rule definitions).
  • Local Users and Groups: Add and remove local user accounts. Add them to or remove them from groups. You cannot add or delete local groups. Adding a user to a group is completely free-text; no browsing. Also, if you attempt to add a user that doesn’t exist, you get a confirmation message that tells you that it worked, but the field doesn’t populate.
  • Network: View the network connections and set basic options for IPv4 and IPv6.
  • Processes: Mostly like Task Manager. Has an option to Create Process Dump.
  • Registry: Nifty registry editor; includes Export and Import functions. Very slow, though; personally I’d probably give up and use regedit.exe for as long as I’m given a choice.
  • Roles and Features: Mostly what you expect. No option for alternate install sources, though, so you won’t be using it to install .Net 3.5. Also, I can’t tell how to discard accidental changes. No big deal if you only accidentally checked a single item. For some reason, clicking anywhere on a line toggles the checked/not checked state, so you can easily change something without realizing that you did it.
  • Services: Interface for installed services. Does not grant access to any advanced settings for a service (like the extra tabs on the SNMP Service). Also does not recognize the Delayed Start modifier for Automatic services. I would take care to only use this for Start and Stop functions.
  • Storage: Works like the Storage part of the Files and Storage Services section in Server Manager. Like the preceding sections, includes most of the same features as its real Server Manager counterpart, but not all.
  • Storage Replica: I’m not using Storage Replica anywhere so I couldn’t gauge this one. Requires a special setup.
  • Virtual Machines and Virtual Switches: These two sections will get more explanation later.
  • Windows Update: Another self-explanatory section. This one has most of the same functionality as its desktop counterpart, although it has major usability issues on smaller screens. The update list is forced to yield space to the restart scheduler, which consumes far more screen real estate than it needs to do its job.

Virtual Switches in Honolulu

Alphabetically, this comes after Virtual Machines, but I want to get it out of the way first.

The Virtual Switches section in Project ‘Honolulu’ mostly mimics the virtual switch interface in Hyper-V Manager. So, it gets props for being familiar. It takes major dings for duplicating Hyper-V Manager’s bad habits.

First, the view:



  • New Virtual Switch
  • Delete Virtual Switch
  • Rename Virtual Switch
  • Modify some settings of a virtual switch

The Settings page (which I had to stitch together because it successfully achieves the overall goal of wasting maximal space):


The New Virtual Switch screen looks almost identical, except that it’s in a sidebar so it’s not quite as wide.

Notes on Honolulu’s virtual switch page:

  • Copies Hyper-V Manager’s usage of the adapter’s cryptic Description field instead of its name field.
  • If you look in the Network Adapter setting on the Settings for vSwitch screenshot and then compare it to the overview screen shot, you should notice something: It didn’t pick the team adapter that I really have my vSwitch on. Also, you can’t choose the team adapter. I didn’t tinker with that because I didn’t want to break my otherwise functional system, but not being able to connect a virtual switch to a team is a non-starter for me.
  • Continues to use the incorrect and misleading “Share” terminology for “Shared with Management OS” and “Allow management OS to share this network adapter”. Hey Microsoft, how hard would it really be to modify those to say “Used by Management OS” and “Allow management OS to use this virtual switch”?
  • No VLAN settings.
  • No SR-IOV settings.
  • No Switch-Embedded Teaming settings
  • No options for controlling management OS virtual NICs beyond the first one

Virtual Machines in Honolulu

All right, this is why we’re here! Make sure that you’re over something soft or the let-down might sting.

Virtual Machine Overview

The overview is my favorite part, although it also manifests the wasteful space usage that plagues this entire tool. Even on a larger resolution, it’s poorly made. However, I like the information that it displays, even if you need to scroll a lot to see it all.

At the top, you get a quick VM count and a recap of recent events:


Even though I like the events being present, that tiny list will be mostly useless on an environment of any size. Also, it might cause undue alarm. For instance, those errors that you see mean that Dynamic Memory couldn’t expand any more because the VMs had reached their configured maximum. You can’t see that here because it needs two inches of whitespace padding to its left and right.

You can also see the Inventory link. We’ll come back to that after the host resources section.

Virtual Machine Host Resource Usage

I mostly like the resource view. Even on my 1366×768 VMConnect window, I have enough room to fit the CPU and memory charts side-by-side. But, they’re stacked and impossible to see together. I’ve stitched the display for you to see what it could look like with a lot of screen to throw at it:


Virtual Machine Inventory

Back at the top of the Virtual Machines page, you can find the Inventory link. That switches to a page where you can see all of the virtual machines:


That doesn’t look so bad, right? My primary complaint with the layout is that I believe that the VM’s name should be prioritized. I’d rather have an idea of the VM’s name as opposed to the Heart Beat or Protected statuses, if given a choice.

My next complaint is that, even at 1366×768, which is absolutely a widescreen resolution, the elements have some overrun. If I pick a VM that’s on, I must be very careful when trying to access the More menu so that I don’t inadvertently Shutdown the guest instead:


What’s on that More menu? Here you go:


That’s for a virtual machine that’s turned on. No, your eyes are not deceiving you. You cannot modify any of the settings of a virtual machine while it is running. Power states and checkpoints are the limit.

I don’t know what Protected means. It’s not about being shielded or clustered. I suppose it means that it’s being backed up to Azure? If you’re not using Azure backup then this field just wastes even more space.

Virtual Machine Settings

If you select a virtual machine that’s off, you can then modify its settings. I elected not to take all of those screenshots. Fitting with the general Honolulu motif, they waste a great deal of space and present less information than Hyper-V Manager. These setting groupings are available:

  • General: The VM’s name, notes, automatic start action, automatic stop action, and automatic critical state action
  • Memory: Startup amount, Dynamic Memory settings, buffer, and weight
  • Processors: Number only. No NUMA, compatibility mode, reservation, or weight settings
  • Disks: I could not get the disks tab to load for any virtual machine on any host, whether 2012 R2 or 2016. It just shows the loading animation
  • Networks: Virtual switch connection, VLAN, MAC (including spoofing), and QoS. Nothing about VMQ, IOV, IPSec, DHCP Guard, Router Guard, Protected Network, Mirroring, Guest Teaming, or Consistent Device Naming
  • Boot Order: I could not get this to load for any virtual machine.

Other Missing Hyper-V Functionality in Honolulu

A criticism that we often level at Hyper-V Manager is just how many settings it excludes. If we only start from there, Project ‘Honolulu’ excludes even more.

Features available in Hyper-V Manager that Honolulu does not expose:

  • Hyper-V host settings — any of them. Live Migration adapters, Enhanced Session Mode, RemoteFX GPUs, and default file locations
  • No virtual SAN manager. Personally, I can live with that, since people need to stop using pass-through disks anyway. But, there are some other uses for this feature and it still works, so it makes the list of Honolulu’s missing features.
  • Secure boot
  • VM Shielding
  • Virtual TPM
  • Virtual hardware add/remove
  • Indication of VM Generation
  • Indication/upgrade of VM version
  • Shared Nothing Live Migration (intra-cluster Live Migration does work; see the Failover Clustering section below)
  • Storage (Live) Migration
  • Hyper-V Replica
  • Smart page file

Except for the automatic critical action setting, I did not find anything in Project ‘Honolulu’ that isn’t in Hyper-V Manager. So, don’t look here for nested VM settings or anything like that.

Failover Clustering for Hyper-V in Honolulu

Honolulu’s Failover Cluster Manager is even more of a letdown than Hyper-V. Most of the familiar tabs are there, but it’s almost exclusively read-only. However, we Hyper-V administrators get the best of what it can offer.

If you look on the Roles tab, you can find the Move action. That initiates a Quick or Live Migration:


Unfortunately, it forces you to pick a destination host. In a small cluster like mine, no big deal. In a big cluster, you’d probably like the benefit of the automatic selector. You can’t even see what the other nodes’ load levels look like to help you to decide.

Other nice features missing from Honolulu’s Failover Cluster Manager:

  • Assignment, naming, and prioritizing of networks
  • Node manipulation (add/evict)
  • Disk manipulation (add/remove cluster disk, promote/demote Cluster Shared Volume, CSV ownership change)
  • Quorum configuration
  • Core resource failover
  • Cluster validation. The report is already in HTML, so even if this tool can’t run validation, it would be really nice if it could display the results of one

Showstopping Hyper-V Issues in Project ‘Honolulu’

Pay attention to the dating of this article, as all things can change. As of this writing, these items prevent me from recommending Honolulu:

  • No settings changes for running virtual machines. The Hyper-V team has worked very hard to allow us to change more and more things while the virtual machine is running. Honolulu negates all of that work, and more.
  • No Hyper-V switch on a team NIC
  • No VMConnect (console access). If you try to connect to a VM, it uses RDP. I use a fair number of Linux guests. Microsoft has worked hard to make it easy for me to use Linux guests. For Windows guests, RDP session cuts out the pre-boot portions that we sometimes need to see.
  • No host configuration

Any or all of these things might change between now and release. I’ll be keeping up with this project in hopes of being able to change my recommendation.

The Future of Honolulu

I need to stress, again, that Honolulu is just a baby. Yes, it needs a lot of work. My general take on it, though, is that it’s beginning life by following in the footsteps of the traditional Server Manager. The good: it tries to consolidate features into a single pane of glass. The bad: it doesn’t include enough. Sure, you can use Server Manager/Honolulu to touch all of your roles and features. You can’t use it as the sole interface to manage any of them, though. As-is, it’s a decent overview tool, but not much more.

Where Honolulu goes from here is in all of our hands. I’m writing this article a bit before the project goes into public beta, so you’re probably reading it at some point afterward. Get the bits, set it up, and submit your feedback. Be critical, but be nice. Designing a functional GUI is hard. Designing a great GUI is excruciatingly difficult. Don’t make it worse with cruel criticism.

[the_ad_group id=”229″]

Why Your Hyper-V PowerShell Commands Don’t Work (and how to fix them)

Why Your Hyper-V PowerShell Commands Don’t Work (and how to fix them)

I occasionally receive questions about Hyper-V-related PowerShell cmdlets not working as expected. Sometimes these problems arise with the module that Microsoft provides; other times, they manifest with third-party tools. Even my own tools show these symptoms. Most GUI tools are developed to avoid the problems that plague the command line, but the solutions aren’t always perfect.

The WMI Foundation

All tools, graphical or command-line, eventually work their way back to the only external interface that Hyper-V provides: its WIM/CIM provider. CIM stands for “Common Information Model”. The Distributed Management Task Force (DMTF) maintains the CIM standard. CIM defines a number of interfaces pertaining to management. Anyone can write CIM-conforming modules to work with their systems. These modules allow users, applications, and services to retrieve information and/or send commands to the managed system. By leveraging CIM, software and hardware manufacturers can provide APIs and controls with predictable, standardized behavior.

Traditionally, Microsoft has implemented CIM via Windows Management Instrumentation (WMI). Many WMI instructions involved VBS or WMIC. As PowerShell gained popularity, WMI also gained popularity due to the relative ease of Get-WmiObject. Depending on where you look in Microsoft’s vast documentation, you might see pushes away from the Microsoft-specific WMI implementation toward the more standard CIM corollaries. Get-CimInstance provides something of an analog to Get-WmiObject, but they are not interchangeable.

For any of this to ever make any sense, you need to understand one thing: anyone can write a CIM/WMI provider. The object definitions and syntax of a provider all descend from the common standard, but they do nothing more than establish the way an interface should look. The provider’s developer determines how it all functions behind the scenes.

Why Hyper-V PowerShell Cmdlets May Not Work

Beyond minor things like incorrect syntax and environmental things like failed hardware, two common reasons prevent these tools from functioning as expected.

The Hyper-V Security Model

I told you all that about WMI so that this part would be easier to follow. The developers behind the Hyper-V WMI provider decide how it will react to any given WMI/CIM command that it receives. Sometimes, it chooses to have no reaction at all.

Before I go too far, I want to make it clear that no documentation exists for the security model in Hyper-V’s WMI provider. I ran into some issues with WMI commands not working the way that I expected. I opened a case with Microsoft, and it wound up going all the way to the developers. The answer that came back pointed to the internal security coding of the module. In other words, I was experiencing a side effect of designed behavior. So, I asked if they would give me the documentation on that — basically, anything on what caused that behavior. I was told that it doesn’t exist. They obviously don’t have any externally-facing documentation, but they don’t have anything internal, either. So, everything that you’re going to see in this article originates from experienced (and repeatable) behavior. No insider secrets or pilfered knowledge were used in the creation of this material.

Seeing Effects of the Hyper-V Security Model in Action

Think about any “Get” PowerShell cmdlet. What happens when you run a “Get” against objects that don’t exist? For example, what happens when I run Get-Job when no jobs are present?


Nothing! That’s what happens. You get nothing. So, you learn to interpret “I got nothing” to mean “no objects of that type exist”.

So, if I run Get-VM and get nothing (2012/R2):


That means that the host has no virtual machines, right?

But wait:

Hyper-V Powershell commands help

What happened? A surprise Live Migration?

Look at the title bars carefully. The session on the left was started normally. The session on the right was started by using Run as administrator.

The PowerShell behavior has changed in 2016:


The PowerShell cmdlets that I tried now show an appropriate error message. However, only the PowerShell module has been changed. The WMI provider behaves as it always has:


To clarify that messy output, I ran gwmi -Namespace root\virtualization\v2 -Class Msvm_ComputerSystem -Filter 'Caption="Virtual Machine"' as a non-privileged user and the system gave no output. That window overlaps another window that contains the output from Get-VM in an elevated session.

Understanding the Effects of the Hyper-V Security Model

When we don’t have permissions to do something, we expect that the system will alert us. If we try to open a file, we get a helpful error message explaining why the system can’t allow it. We’ve all had that experience enough times that we’ve been trained to expect a red flag. The Hyper-V WMI provider does not exhibit that expected behavior. I have never attempted to program a WMI provider myself, so I don’t want to pass any judgment. I noticed that the MSCluster namespace acts the same way, so it may be something inherent to CIM/WMI that the provider authors have no control over.

In order for a WMI query to work against Hyper-V’s provider, you must be running with administrative privileges. Confusingly, “being a member of the Administrators group” and “running with administrative privileges” are not always the same thing. When working with the Hyper-V provider on the local system, you must always ensure that you run with elevated privileges (Run as administrator) — even if you log on with an administrative account. Remote processes don’t have that problem.

The administrative requirement presents another stumbling block: you cannot create a permanent WMI event watcher for anything in the Hyper-V provider. Permanent WMI registration operates anonymously; the Hyper-V provider requires confirmed administrative privileges. As with everything else, no errors are thrown. Permanent WMI watchers simply do not function.

The takeaway: when you unexpectedly get no output from a Hyper-V-related PowerShell command, you most likely do not have sufficient permissions. Because the behavior bubbles up from the bottom-most layer (CIM/WMI), the problem can manifest in any tool.

The Struggle for Scripters and Application Developers

People sometimes report that my tools don’t work. For example, I’ve been told that my KVP processing stack doesn’t do anything. Of course, the tool works perfectly well — as long as it has the necessary privileges. So, why didn’t I write that, and all of my other scripts, to check their privilege? Because it’s really hard, that’s why.

With a bit of searching, you’ll discover that I could just insert #requires -RunAsAdministrator at the top of all my scripts. Problem solved, right? Well, no. Sure, it would “fix” the problem when you run the script locally. But, sometimes you’ll run the script remotely. What happens if:

  • … you run the script with an account that has administrative privileges on the target host but not on the local system?
  • … you run the script with an account that has local administrative privileges but only user privileges on the target host?

The answer to both: the actual outcome will not match your desired outcome.

I would need to write a solution that:

  • Checks to see if you’re running locally (harder than you might think!)
  • Checks that you’re a member of the local administrators
  • If you’re running locally, checks if your process token has administrative privileges

That’s not too tough, right? No, it’s not awful. Unfortunately, that’s not the end of it. What if you’re running locally, but invoke PowerShell Remoting with -ComputerName or Enter-PSSession or Invoke-Command? Then the entire dynamic changes yet again, because you’re not exactly remote but you’re not exactly local, either.

I’ve only attempted to fully solve this problem one time. My advanced VM settings editor includes layers of checks to try to detect all of these conditions. I spent quite a bit of time devising what I hoped would be a foolproof way to ensure that my application would warn you of insufficient privileges. I still get messages telling me that it doesn’t show any virtual machines.

I get better mileage by asking you to run my tools properly.

How to Handle the Hyper-V WMI Provider’s Security

Simply put, always ensure that you are running with the necessary privileges. If you are working locally, open PowerShell with elevated permissions:


If running remotely, always ensure that the account that you use has the necessary permissions. If your current local administrator account does not have the necessary permissions on the target system, invoke PowerShell (or whatever tool you’re using) by [Shift]+right-clicking the icon and selecting Run as different user:


What About the “Hyper-V Administrators” Group?

Honestly, I do not deal with this group often. I don’t understand why anyone would be a Hyper-V Administrator but not a host administrator. I believe that a Hyper-V host should not perform any other function. Trying to distinguish between the two administrative levels gives off a strong “bad plan” odor.

That said, I’ve seen more than a few reports that membership in Hyper-V Administrators does not work as expected. I have not tested it extensively, but my experiences corroborate those reports.

The Provider Might Not Be Present

All this talk about WMI mostly covers instances when you have little or no output. What happens when you have permissions, yet the system throws completely unexpected errors? Well, many things could cause that. I can’t make this article into a comprehensive troubleshooting guide, unfortunately. However, you can be certain of one thing: you cannot tell Hyper-V to carry out an action if Hyper-V is not running!

Let’s start with an obvious example. I ran Get-VM on a Windows 10 system without Hyper-V:


Nice, clear error, right? 2012 R2/Win 8.1 have a slightly different message.

Things change a bit when using the VHD cmdlets. I don’t have any current screenshots to show you because the behavior changed somewhere along the way… perhaps with Update 1 for Windows Server 2012 R2. Windows Vista/Server 2008 and later include a native driver for mounting and reading/writing VHD files. Windows 8/Server 2012 and later include a native driver for mounting and reading/writing VHDX files. However, only Hyper-V can process any of the VHD cmdlets. Get-VHD, New-VHD, Optimize-VHD, Resize-VHD, and Set-VHD require a functioning installation of Hyper-V. Just installing the Hyper-V PowerShell module won’t do it.

Currently, all of these cmdlets will show the same or a similar message to the one above. However, older versions of the cmdlets give a very cryptic message that you can’t do much with.

How to Handle a Missing Provider

This seems straightforward enough: only run cmdlets from Hyper-V module against a system with a functioning installation of Hyper-V. You can determine which functions it owns with:

When running them from a system that doesn’t have Hyper-V installed, use the ComputerName parameter.

Further Troubleshooting

With this article, I wanted to knock out two very simple reasons that Hyper-V PowerShell cmdlets (and some other tools) might not work. Of course, I realize that any given cmdlet might error for a wide variety of reasons. I am currently only addressing issues that block all Hyper-V cmdlets from running.

For troubleshooting a failure of a specific cmdlet, make sure to pay careful attention to the error message. They’re not always perfect, but they do usually point you toward a solution. Sometimes they display explicit text messages. Sometimes they include the hexadecimal error code. If they’re not clear enough to understand immediately, you can use these things in Internet searches to guide you toward an answer. You must read the error, though. Far too many times, I see “administrators” go to a forum and explain what they tried to do, but then end with, “I got an error” or “it didn’t work”. If the error message had no value the authors wouldn’t have bothered to write it. Use it.

[the_ad_group id=”229″]

6 Common Hyper-V Configuration Mistakes to Avoid [eBook]

6 Common Hyper-V Configuration Mistakes to Avoid [eBook]

Hello once again everyone!

Seems like I’ve been running into a lot of questions lately for issues that could have been easily avoided, but alas, if you don’t know about certain pitfalls you just don’t know.

With technology as complex as virtualization, there are always going to be some potential pitfalls, and it certainly is helpful to sidestep these pitfalls up front if you can. While the below is by no means a fully comprehensive list, I’ve listed 6 of the most common mistakes that Hyper-V administrators tend to run into during day-to-day operations.


6 Common Hyper-V Configuration Mistakes

  1. Unnecessary Software is Installed Inside of the Management OS: While you certainly are able to install software inside of the management operating system of the Hyper-V Host, there are only certain cases where you should do so. Software installed at the host level is mainly used only when you have need of something that is for support of the virtual environment (such as backup software). All other cases add complexity, increase patching needs, increase the attack surface, and may have licensing implications.
  2. Hyper-V Hosts are Left in the General Workstations OU in Active Directory: By default, when machines are added to a domain, they often get dropped inside of a “default OU”. This is likely fine for workstations, but it poses several major problems for Hyper-V hosts. The main problem being the prospect of GPOs for another set of end-points being applied to core virtualization infrastructure, which can create unpredictable issues for services running on those hosts.
  3. Security Best Practices Are Not Followed: It’s the new mindset in IT, that security is EVERYONE’s responsibility, and this applies to infrastructure engineers as well. Some of the default settings in Hyper-V are not the most secure, and many new Hyper-V administrators will adjust certain settings with ease-of-use in mind, without really comprehending the effects this change could have on the overall security of the solution. Needless to say, leaving core systems unsecured could pose MAJOR issues for you and your organization later on down the line, should a breach occur.
  4. NUMA is Configured Improperly: NUMA is often a misunderstood feature in Hyper-V. When used correctly it can provide numerous performance benefits, but if misconfigured, it will do the exact opposite. NUMA can cripple your performance if you don’t understand what you’re doing with it. So, it goes without saying that you should have a full understanding of NUMA, how it affects your systems, and how best to go about using it.
  5. Anti-Virus Exclusions and Best Practices are Not Followed: If you search for best practices around using AV with Hyper-V, you’ll find a plethora different responses and stances online. Some people feel that by running Windows Server in Core mode, and with careful control of access to the management OS, the host itself can be kept quite safe. While this works for some organizations, others may have industry or corporate regulations in place that equate to a hard requirement for having AV on all endpoints, virtualization hosts included. It is possible to run AV effectively on Hyper-V hosts, but there are a number of potential performance impacts in doing so. In certain extreme cases, AV has been known to bring down virtual machine services on Hyper-V hosts when configured improperly. So, it behooves you to makes sure that if you are indeed using anti-virus on your Hyper-V Hosts, that it be configured properly.
  6. Checkpoints are Used Improperly or are Being Used as Backups: Checkpoints in Hyper-V, like NUMA, is another often misunderstood (if not misused) feature in Hyper-V. If used improperly, there is a real risk of downtime or even data loss. Additionally, many organizations operate under the assumption that checkpoints are something of a backup, which couldn’t be further from the truth. Checkpoints do not replace backups. Not only is there no retention associated with checkpoints, you’re utilizing production storage as well in most cases. This is not to mention several other potential issues related to the misuse of checkpoints as well.

What you should be doing instead

Now that you know what not to do, you’re probably thinking about what you should be doing instead. You didn’t think I’d be telling you about several pitfalls when working with Hyper-V without providing you the needed information to avoid them, did you? Have no fear!

Enter the new Altaro eBook! It contains all the information you’ll need to avoid the pesky issues laid out above! Authors Dave and Cristal Kawula have done a fantastic job going into more detail about each of these common mistakes and explaining what you should be doing instead. Armed with the knowledge within, you’ll be putting yourself and your Hyper-V infrastructure on a much better footing and will pave the way for more successful day-to-day operations by avoiding some of the most common Hyper-V issues in the industry today.


6_mistakes ebook cover image


Download Altaro eBook - 6 Configuration Mistakes

As always if there are any follow-up questions or comments, be sure to use the comments section below this post and we’ll be sure to get back with you!

Note: a lot of time and effort goes into producing our eBooks free of charge to our audience. If you find the eBook useful please share with your social followings, colleagues or friends 🙂

I hope you enjoy the eBook!

Upgrading Hyper-V 2012 R2 to Hyper-V 2016

Upgrading Hyper-V 2012 R2 to Hyper-V 2016

Ready to make the jump from Hyper-V 2012 R2 to 2016? With each successive iteration of Hyper-V, the move gets easier. You have multiple ways to make the move. If you’re on the fence about upgrading, some of the techniques involve a bit less permanence.

What This Article Will Not Cover

I’m not going to show you how to install Hyper-V. The process has not changed since 2012. We probably owe the community a brief article on installing though…

I will not teach you how to use Hyper-V or its features. You need to know:

  • How to install Hyper-V
  • How to install and access Hyper-V’s native tools: Hyper-V Manager, PowerShell, and, where applicable, Failover Cluster Manager
  • How to use Hyper-V Replica, if you will be taking any of the HVR options
  • How to use Live Migration

I won’t make any special distinctions between Hyper-V Server and Windows Server with Hyper-V.

I will not show anything about workgroup configurations. Stop making excuses and join the domain.

I’m not going to talk about Windows 10, except in passing. I’m not going to talk about versions prior to 2012 R2. I don’t know if you can skip over 2012 R2.

What This Article Will Cover

What we will talk about:

  • Virtual Machine Configuration File Versions
  • Rolling cluster upgrades: I won’t spend much time on that because we already have an article
  • Cross-version Live Migration
  • Hyper-V Replica
  • Export/import
  • In-place host upgrades

Virtual Machine Configuration File Versions

Each new iteration of Hyper-V brings a new format for the virtual machine definition file. It also brings challenges when you’re running different versions of Hyper-V. Historically, Hyper-V really only wants to run virtual machines that use its preferred definition version. If it took in an older VM, it would want to upconvert it. 2016 changes that pattern a little bit. It will happily run version 5.0 VMs (2012 R2) without any conversion at all. That means that you can freely move a version 5.0 virtual machine between a system running 2012 R2 Hyper-V and a system running 2016. The Windows 10/Windows Server 2016 version of Hyper-V Manager includes a column so that you can see the version:


The version has been included in the Msvm_VirtualSystemSettingData WMI class for some time and exposed as a property in Get-VM. However, the Get-VM cmdlet in version 2 of the Hyper-V module (ships with W10/WS2016/HV2016) now includes the version in the default view:

upgrading hyper-v 2012 r2 to 2016 - version 5.0

The capability of 2016 to directly operate the older version enables all of the features that we’ll talk about in this article.

Rolling Cluster Upgrades

2016 gives an all-new upgrade option. “Rolling cluster upgrade” allows you to upgrade individual cluster nodes to 2016. At least, we describe it that way. More accurately, clusters of Hyper-V hosts can contain both 2012 R2 and 2016 simultaneously. So, “upgrading” may not be the correct term to use for individual nodes. You can upgrade them, of course, but you can also wipe them out and start over or replace them with all-new hardware. Whatever you’re doing, the process boils down to: take down a 2012 R2 node, insert a 2016 node.

A feature called “cluster functional level” enables this mixing of versions. When the first 2016 node joins the cluster, it becomes a “mixed mode” cluster running at a “functional level” of 2012 R2. Once the final 2012 R2 node has been removed, you just run Update-ClusterFunctionalLevel. Then, at your convenience, you can upgrade the configuration version of the virtual machines.

Adrian Costea wrote a fuller article on rolling cluster upgrades.

Cross-Version Live Migration

Due to the versioning feature that we opened the article with, Live Migration can freely move a version 5.0 virtual machine between a 2012 R2 system and a 2016 system. If both of the hosts belong to the system cluster (see the previous section), then you don’t need to do anything else. Contrary to some myths being passed around, you do not need to configure anything special for intra-cluster Live Migrations to work.

To Live Migrate between hosts that do not belong to the same cluster, you need to configure constrained delegation. That has not changed from 2012 R2. However, one thing has changed: you don’t want to restrict delegation to Kerberos on 2016 systems anymore. Instead, open it up to any protocol. I provided a PowerShell script to do the work for you. If you’d rather slog through the GUI, that same article shows a screenshot of where you’d do it.

Special note on constrained delegation configuration between 2012 R2 and 2016: Constrained Delegation’s behavior can be… odd. It gets stranger when combing 2012 R2 with 2016. On a 2016’s systems property sheet, always select “Use any authentication protocol”. On a 2012 R2 system, always select “Use Kerberos only”. I found that I was able to migrate from 2016 to 2012 R2 without setting any delegation at all, which I find… odd. When moving from 2012 R2, I found that I always had to start the migration from the 2016 side. Nothing I did ever allowed for a successful move when I initiated it from the 2012 R2 side. I expect that your mileage will vary. If you get errors, just try a different combination. I promise you, this migration path does work.

Cross-Version Hyper-V Replica

If you’re reading straight through, you’ll find that this section repeats much of what you’ve already seen.

Hyper-V Replica will happily move virtual machines using configuration version 5.0 between 2012 R2 and 2016 systems. The fundamental configuration steps do not change between the two versions.

Export and Import

The export feature has changed a great deal since its initial inception. Once upon a time, it would create an .exp file in place of the XML file. Without that .exp file, Hyper-V would not be able to import an exported virtual machine. That limitation disappeared with 2012. Since then, Hyper-V can import a virtual machine directly from its XML file. You don’t even need to export it anymore. If you wanted, you could just copy the folder structure over to a new host.

However, the export feature remains. It does two things that a regular file copy cannot:

  • Consolidation of virtual machine components. If you’ve ever looked at the settings for a virtual machine, you’d know that you can scatter its components just about anywhere. The export feature places all of a virtual machine’s files and attached VHD/Xs into a unified folder structure.
  • Active state preservation. You can export a running virtual machine. It will be imported where it left off.

When you export a virtual machine, it retains its configuration version. The import process on 2016 does not upgrade version 5.0 virtual machines. They will remain at version 5.0 until you deliberately upgrade them. Therefore, just as with Live Migration and Replica, you can use export/import to move version 5.0 virtual machines between 2012 R2 and 2016.

In-Place Host Upgrades

Windows has earned a reputation for coping poorly with operating system upgrades. Therefore, a lot of people won’t even try it anymore. I can’t say that I blame them. However, a lot of people haven’t noticed that the upgrade process has changed dramatically. Once upon a time, there was a great deal of backing up and in-place overwrites. The Windows upgrade process no longer does any of that. It renames the Windows folder to Windows.old and creates an all-new Windows folder from the install image. But, the matter of merging in the old settings remains. Most problems source from that.

I have not personally attempted an upgrade of Windows Server for many years now. I do not exactly know what would happen if you simply upgraded a 2012 R2 system directly to 2016. On paper, it should work just fine. In principle…

If you choose the direct upgrade route, I would:

  • Get a good backup and manually verify it.
  • Schedule enough time to allow for the entire thing to finish, go horribly wrong, and rebuild from scratch
  • Make a regular file copy of all of the VMs to some alternative location

Wipe and Reinstall

If you want to split the difference a bit, you could opt to wipe out Windows/Hyper-V Server without hurting your virtual machines. Doing so allows you to make a clean install on the same hardware. Just make certain that they’re not in the same location that you’re wiping out. You can do that with a regular file copy or just by holding them on a separate partition from the management operating system. Once the upgrade has completed, import the virtual machines. If you’re going to run them from the same location, use the Register option.

Leveraging Cross-Version Virtual Machine Migration Options

All of these options grant you a sort of “try before you commit” capability. In-place upgrades fit that category the least; going back will require some sacrifice. However, the other options allow you to move freely between the two versions.

Some people have reported encountering performance issues on 2016 that they did not have with 2012 R2. To date, I have not seen any reason to believe that 2016 possesses any inherent flaws. I haven’t personally involved myself with any of these systems, so I can only speculate. So far, these reports seem isolated, which would indicate situational rather than endemic problems. Hardware or drivers that aren’t truly ready for 2016 might cause problems like these. If you have any concerns at all, wouldn’t you like the ability to quickly revert to a 2012 R2 environment? Wouldn’t you also like to be able to migrate to 2016 at your leisure?

Cross-Version Virtual Machine Limitations

Unfortunately, this flexibility does not come without cost. Or, to put a more positive spin on it, upgrading the configuration version brings benefits. Different version levels bring their own features. I didn’t track down a map of versions to features. If you upgrade from 5.0 to the current version (8.0 as of this writing), then you will enable all of the following:

  • Hot-Add and Hot-Remove of memory and network adapters
  • Production Checkpoints/Disable Checkpoints
  • Key Storage Drive (Gen 1)
  • Shielded VM (Gen 2)
  • Virtual Trust Platform Module (vTPM) (Gen 2)
  • Linux Secure Boot
  • PowerShell Direct

When you’re ready to permanently make the leap to 2016, you can upgrade a virtual machine with Update-VMVersion. You’ll also find on the VM’s right-click menu:


For either method to be successful, the virtual machine must be turned off.

Page 2 of 2712345...1020...Last »