Live Backup of a Hyper-V Guest VM with Hyper-V VSS Writer

Live Backup of a Hyper-V Guest VM with Hyper-V VSS Writer

An earlier posting, “Backing Up Hyper-V Guests – Host-Based vs Guest-Based Methods”, discussed the differences between backing up virtual machines as though they were physical computers against utilizing software installed within Hyper-V’s parent partition to back up virtual machines as complete resources. When using the latter method, some backup software has the ability to coordinate with Hyper-v’s VSS writer to back up virtual machines without taking them offline. The considerations for enabling this to occur are examined in this article. (more…)

VSS Crash-Consistent vs. Application-Consistent VSS Backups (post 2 of 2)

VSS Crash-Consistent vs. Application-Consistent VSS Backups (post 2 of 2)

When is Application-Consistent Backup Vital? Not all situations require an application-consistent backup. Things such as file and print servers will be fine with crash-consistent and possibly inconsistent backups. If your application doesn’t provide a VSS writer, there might not even be a way to get an application-consistent backup of it while its containing machine is live. The most common need for application-consistent backups is the usage of database-backed applications.

NOTE: This is the second blog post in a 2-series post.  You can read the first post in this series here.

When is Application-Consistent Backup Vital?

Not all situations require an application-consistent backup. Things such as file and print servers will be fine with crash-consistent and possibly inconsistent backups. If your application doesn’t provide a VSS writer, there might not even be a way to get an application-consistent backup of it while its containing machine is live. The most common need for application-consistent backups is the usage of database-backed applications. In particular, Microsoft Exchange and SQL Server are fairly dependent upon application-consistent backups. In their default configurations (non-circular logging and full recovery modes, respectively), they do not normally write directly to their database files but instead rely upon log files. When their VSS writer is triggered, they flush all pending operations from memory and commit the contents of their log files into the main database files. This behavior ensures that a crash during normal operations will pose a risk to the active log file, not the primary database. If application-consistent backups are not taken on a regular basis and these applications are left in their default configurations, the log files will eventually consume all available space. Even if transaction quantity is low enough and disk space ample enough to prevent consumption from being a major concern, restoring either system from a crash-consistent state is much more painful and risky than restoring from an application-consistent state.  Also, if these applications are left in their default configuration and backed up using a non-application-consistent method, even if it is a full image-level backup, they will not commit their logs.

Backup Operations in Hyper-V

Looking at the simplistic file-based layout of a virtual machine, it is tempting to assume you can use any available file-based backup or even copying software. Unless you are shutting down or saving the virtual machines, avoid falling into this trap. If you are saving the VMs, ensure you are including the VSV and BIN files in your copies. Even if you follow all these steps, Hyper-V takes a particular ownership of its virtual machines and won’t like it if you try to simply copy in the files of a virtual machine that were running on a different host. If you intend to back up the virtual machines without saving them, ensure that whatever method you utilize triggers VSS. There are a handful of Internet posts suggesting that you can simply run copying tools like Robocopy against your VHD files and everything will be fine. Without calling on VSS to pause I/O first, this technique can easily result in a VHD that is unusable. If you intend to use nothing but freely available tools and scripts, research things like Diskshadow and implement your scripts properly.

Hyper-V has its own VSS writer. This deceptively simple sentence means that by deploying on Hyper-V, you automatically have the ability to get application-consistent backups of virtual machines from the host level. There are some conditions to be met in order to ensure this works as expected, but the basic operation is that when Hyper-V’s VSS is triggered, it notifies VSS within the virtual machine, which in turn notifies all of its registered VSS writers that a backup is taking place.

No VSS Writer Available?

In some cases, you need an application-consistent backup but there is no VSS writer available. One example of this is MySQL. Hyper-V backups of virtual machines containing MySQL will always result in either a crash-consistent or an image-level backup. For MySQL, the latter is probably acceptable as MySQL doesn’t perpetually expand the log file. However, if you’re using MySQL within a VSS-aware VM, then a Hyper-V-based backup tool is going to take a crash-consistent backup. MySQL (like any other database system) isn’t always recoverable from a crash-consistent backup; even when recovery is possible, it may be painful. MySQL is just one example; any number of line-of-business applications could tell a similar tale. In the case of MySQL, one solution is to find a guest-level backup application that is MySQL-aware and can back it up properly. For applications for which no backup application has a plug-in, you may need to have pre- and post-backup scripts that stop services or close applications. If brief downtime is acceptable, you can disable the Backup item in Hyper-V Integration Services, thereby forcing Hyper-V to save the state of the VM during backup. This technique results in an image-level backup and can be used on any application that doesn’t have a VSS writer.


General Requirements for Application-Consistent Backups in Hyper-V

Note that some backup software contains its own methods to ensure application consistency and may be able to circumvent some or all of these rules. Please consult with your application vendor for further details.

  • The VM must have a current version of Hyper-V Integration Services installed and the Backup component must be enabled.
  • The VM must have a compliant implementation of VSS (Vista Enterprise or later for desktop operations and Windows Server 2003 SP2 or later for servers)
  • All of a VM’s VHDs must reside on the same volume (whether it is local or CSV)
  • All of the VHDs must be formatted with NTFS. This is a limitation even in a physical environment as VSS does not interact with FAT volumes.
  • None of the VHDs can contain Dynamic volumes; they must all be Simple (not to be confused with Dynamic vs. Fixed VHD, either of which can be used)
  • The volume holding the VHDs must have enough free space for the VSS snapshot; volumes contained by the VHDs must also have enough space to hold their VSS snapshots. The exact need will vary based on several conditions but the safe line is at 20%.

Special Considerations for Cluster Shared Volumes

With VSS operating at the volume level, it might seem that a Cluster Shared Volume is automatically covered, but this isn’t precisely true. VSS does operate on volumes, but it is constrained to the host it is running on. A CSV is simultaneously connected to multiple hosts. In normal mode, only the metadata for a CSV is handled by a single host; all other virtual machine data can be directly accessed by the host that owns it. Therefore, when VSS quiesces a CSV and takes a snapshot, no other host in the cluster is aware of it. In this instance, the volume-level nature of VSS can be a threat. Even if you’re specifically calling VSS on a host to back up only a VM owned by that host, VSS is snapping the entire CSV. No other host is aware of the VSS operation and will carry on with its I/O operations, so the entire status of the VSS snapshot is questionable. Furthermore, and much more dangerous, there is nothing preventing another host from initiating a VSS snapshot while the first is still being used. Partly to address this situation, Microsoft implemented “Redirected Access” mode. In this state, all I/O for a given CSV is handled by the coordinator node. If any other host in the cluster wishes to access that CSV, its I/O is encapsulated in SMB blocks and funneled through the coordinator node. This ensures that the VSS snapshot is not disturbed and that it is not trampled on by another VSS operation.

If you are using CSVs, your backup solution absolutely must be able to turn on redirected access mode. There is no guaranteed safe way to copy virtual machine data off of a CSV without turning on redirect access unless the VSS snapshot is not used and the VM is saved or turned off.


NOTE: This is the second blog post in a 2-series post.  You can read the first post in this series here.

Have any questions?

Leave a comment below!


Backing up Hyper-V

If you’d like to make backing up your Hyper-V VMs easy, fast and reliable, check out Altaro Hyper-V Backup v4. It’s free for up to 2 VMs and supports Hyper-V Server 2012 R2! Need more? Download a 30-day trial of our Unlimited Edition here:


(Don’t worry, we hate spam too!)

VSS Crash-Consistent vs Application-Consistent VSS Backups (post 1 of 2)

VSS Crash-Consistent vs Application-Consistent VSS Backups (post 1 of 2)

When designing any IT solution, many administrators often consider “Backup” to be little more than another box on a long list of items to check off. They verify that the software and hardware they’re using will handle the load, configure it to back up on a reasonable schedule, and forget about it. Some will take the extra step of restoring some data to an alternate location as a test. Hardly any go through the full exercise of simulating an actual catastrophe. Most of the time, this practice is completely harmless. Unfortunately, if disaster does strike, there are often more questions than answers. Planning ahead is critical, and that involves knowing what sort of backup you need and if your backup application can provide it.

NOTE: This is the first blog post in a 2-series post.  You can read the second post in this series here.

Consistency Definitions

To determine what sort of backups you need, you must first understand the terminology in use. The backup types applicable to this discussion are inconsistent, crash consistent, and application consistent.

Inconsistent Crash-Consistent Application-Consistent Image
Live backup clip_image001[16] clip_image001[17] clip_image001[18] clip_image002
Interdependent files guaranteed to be the same version clip_image002[1] clip_image001[19] clip_image001[20] clip_image001[21]
Protection against transactional data loss (application-aware) clip_image002[2] clip_image002[3] clip_image001[22] clip_image003


Inconsistent Backup

An inconsistent backup is the oldest type. Backup software starts at the beginning of the file structure and copies out data until it gets to the end. If any file changed after it was backed up but before the job completed, then the result is an inconsistent backup. It means, simply, that the files contained in the backup job are mismatched. In some instances, this isn’t a problem. Inconsistent backups are often acceptable for things like simple file servers and data locations that rarely change. These can be dangerous to the point of uselessness for database systems. Databases are typically composed of multiple interdependent files, so they must all be synchronized in order for a backup to be considered successful. Also, an inconsistent backup method only grabs what is on disk at the time; if a database has a pending operation in memory, it is completely lost.

A software program can take an inconsistent backup at any time without any real interaction with anything except the operating system. It may struggle if it isn’t running with sufficient permissions to grab a file or encounters a file that’s locked for reading, but otherwise it’s just another process running on the system.


Crash-Consistent Backup

The most important distinction between a crash-consistent backup and an inconsistent backup is that all data within a crash-consistent backup set is captured at exactly the same time. This is the most common method in use by commercial backup software today and is almost always sufficient for applications that do not rely on a database. One thing it shares with the inconsistent method is that it does not capture the contents of memory or any pending I/O operations. If a crash-consistent backup is restored in its entirety, then the data will be in the same state it would have been if the system had crashed at the exact moment that the backup was taken. This is reason it is considered “crash-consistent”. If a database system is restored to a crash consistent state, then it is necessary to follow any procedures that would be followed if the system had actually crashed at that point. Many applications, like Active Directory, have an automated recovery mechanism and will attempt to handle the problem without administrator intervention. If these automated systems aren’t successful, ensure that you know what the application vendor’s process is for crash recovery and be prepared to follow those steps. For Microsoft Exchange, you may need to know how to set up a recovery group and use its utilities to integrate log files. For Microsoft SQL, you may need to know how to replay logs into a database file.

A crash-consistent backup requires more effort on the part of backup software than inconsistent backups. Techniques vary, but in the Windows world, the Volume Shadow Copy Service (VSS) was designed as a component of the operating system specifically to assist backup software. A backup application coordinates its efforts with VSS. When triggered, VSS pauses I/O on the volume and takes a block-level snapshot of it, and then the backup software pulls its backup from that snapshot at its leisure.


Application-Consistent Backup

An application-consistent backup is the most involved but also the most desirable. Again, techniques vary, but the general process for Windows applications is for the application manufacturer to provide a VSS writer. When the VSS service is triggered, it will notify these writers that a backup is occurring. It’s then up to the VSS writer how to handle it. In general terms, a database application will respond to its VSS writer being triggered by flushing all of its memory and I/O operations so that the database is completely consistent. In doing so, there is nothing in memory and no pending I/O to be lost. A proper VSS writer should effectively place all the data for an application in the same state it would be if the application were properly closed. When the VSS snapshot is complete, it signals the VSS writers, which are then to resume normal operation of the attendant application while the backup software safely copies out of the snapshot.

If an application does not provide or properly register a VSS provider but its data resides on a volume with VSS enabled, the data is backed up in a crash-consistent state.

Image-Level Backup

An image-level backup is quite different from the other types. All previously mentioned methods operate in some fashion within a machine that is actively running. Since no backup software can properly capture the active state of memory and I/O, the most reliable way to completely replicate all aspects of a machine is to shut it down. Then, the storage medium can be copied out block-by-block and there is absolutely no concern of anything being lost. Virtualization does add another option; Hyper-V manifests this as “Saved State”. This is similar to hibernation on a physical machine, except that the virtual machine itself is unaware of the state change. I/O in the virtual machine is paused and the contents of memory are copied to a file on disk. As long as every single file related to the virtual machine is copied, a restore of that backup will result in a virtual machine that is in exactly the same state it was when it was saved.

NOTE: This is the first blog post in a 2-series post.  You can read the second post in this series here.


Hyper-V Page File Settings

Hyper-V Page File Settings

When you install Hyper-V or a copy of Windows Server for the express purpose of running the Hyper-V role, its default configuration for the page file (also called a swap file) is generally wasteful, although not harmful. Page files for individual virtual machines are tuned in the same fashion as normal physical machines, but there are a couple of things to think about that are unique to VMs. (more…)

Backing Up Hyper-V Guests – Host-Based vs Guest-Based Methods

There are two ways to back up a virtual machine. One is to treat it as a traditional machine and back up using a locally installed software package or agent. The second method is to back up the virtual machine as an object from the perspective of the host computer. The traditional method is the most direct and the easiest to understand and implement. The second method requires specialized software that runs on the host and is more complicated. Both have their strengths and weaknesses. (more…)

How to install Microsoft hotfixes / patches on server core & native Hyper-V

How to install Microsoft hotfixes / patches on server core & native Hyper-V

As with all infrastructure software, it is critical to stay abreast of updates applicable to Hyper-V. If you’ve got a particular Hyper-V issue and want to know if there’s a hotfix, or if you like to know the intricate details of each and every patch prior to deploying on your system, you’re in luck: Microsoft maintains a complete list of all updates to Hyper-V. The list can be automatically synchronized to your computer via the “Subscribe to Article (RSS)” link on the right side. If you’re on R1, there is a hyperlink in the first “Note” paragraph that will take you to that list. (more…)

Free Download – PowerShell Script for Hyper-V Export

Free Download – PowerShell Script for Hyper-V Export

As promised, Altaro Software , maker of Hyper-V virtual machine backup software, is releasing a Free PowerShell script to automate Hyper-V VM export.  Altaro realizes that even though all companies recognize the importance of backing up their virtual machines, some of them simply do not have the capability to acquire a proper backup solution. While not a true backup, Hyper-V’s export feature can be used as an interim measure to ensure some duplicate of your virtual machines exists somewhere.

We also have Altaro Hyper-V Backup Freeware Edition, so if you have a small setup you might want to check that out as well.

We plan to release more free software and scripts throughout the year so if you want to be notified, please subscribe to the Altaro Hyper-V blog.

Please note that this is the first version of the script, so bugs might exist. You can also open the PowerShell script with a text viewer to view the source. This script does not include any type of warranty and we take no responsibility of what it might or might not do.

Download the script from Free Download – PowerShell Script for Hyper-V Export. Instructions on how to use the script can be found on that same page.


Download the script from Free Download – PowerShell Script for Hyper-V Export. Instructions on how to use the script can be found on that same page.



Using Hyper-V Export for Backups

Using Hyper-V Export for Backups

One of the benefits of virtualized systems is the portability of the guest machines. At their core, they are nothing more than a collection of files. What the guest machines believe to be their “hard drives” are actually .VHD files. Their “hardware components” are determined by a list stored in an XML file. At this time, the files can’t simply be copied from one point to another without some fairly involved work, but the “Export” command can be used to prepare the VM and copy its data to another location. This also provides a rudimentary form of backup.  (more…)

Page 27 of 28« First...1020...2425262728