Save to My DOJO
Welcome to the first installment of this three-part series where I’ll show you how to set up a nested vSphere 6 environment comprising vCenter Server 6.0 for Windows and ESXi 6.0 hosts. If the term nested virtualization is alien to you, don’t worry. All it means is that we will be running a hypervisor as a VM running under another hypervisor. So, in this case, ESXi will be installed as a VM which is itself hosted on a physical ESXi host. One of the advantages of nesting, is that you can test out new VMware features or simulate upgrade paths to a new vSphere release. Nesting is also frequently used for QA/DEV and other testing environments as well as for home labs.
In Part 1, we will have a look at how to download the required VMware ISO images and how to create the VMs needed for the nested ESXi hosts.
In Part 2, we will install and configure ESXi on the VMs created in Part 1.
Part 3 covers the installation and configuration of vCenter Server, how to add nested ESXi hosts to it and also some basic tasks.
There are a few prerequisites you’ll need to take care of first.
1 – If you don’t have one, register for a VMware account here. Once you do, an activation link is sent to your email address. Activating it will allow you to download the ESXi hypervisor and vCenter Server ISO images.
2 – A physical ESXi host (v5.5 or later) managed by vCenter Server
3 – On the physical ESXi host, do the following.
- Create a virtual machine and install Windows Server 2008 or a later version on it.
- Enable promiscuous mode and forged transmits on the virtual switch’s portgroup as per Fig.1. This is where you’ll connect the ESXi VMs you’ll go on and create it later on.
Downloading the VMware ISO images
You first need to download the ESXi and vCenter Server for Windows ISO images. Go to this link and download the ISO shown in Figure 2. Save the ISOs to a local folder or upload them to a datastore mounted on the physical ESXi host. Either method will work just fine however for this post, I’ll be using the local folder method.
As an alternative to the vCenter Windows version, you can also try out vCenter Server Appliance (VSCA) which is a Linux based appliance. You can read about the two vCenter Server versions here. Both ESXi and vCenter Server can be fully and freely evaluated for up to 60 days.
Creating the ESXi VMs
We start off by creating the VMs on which ESXi will be installed. Log in the vCenter server managing the physical ESXi host or directly to the physical ESXi host using the thick vSphere client or the vSphere Web client. I’ll be using both clients in this series just to illustrate any differences but mostly because some functionality is only exposed by the web client.
UPDATE: Since this series was written, vSphere 6.5 has been released. The bulk of what’s covered here still applies however you will not be able to connect to vCenter Server using the thick vSphere (C#) client since it is no longer supported.
Step 1 – Log in vCenter Server with the thick vSphere client.
Step 2 – Switch to the VMs and Templates view. Right-click on the <datacenter> name, select New Folder and create a folder called NestedESXi.
Step 3 – Right-click on the NestedESXi folder and select New Virtual Machine.
Step 4 – Choose Custom configuration and click Next.
Step 5 – Specify a name for the VM. Let’s just call it ESX6_1. Click Next.
Step 6 – Select the physical ESXi host on which to create the VM and click Next.
Step 7 – Where applicable, specify a resource pool, otherwise select an ESXi host instead. Click Next.
Step 8 – Pick a datastore where the VM is created and click Next.
Step 9 – Select Virtual Machine Version: 8. At a later stage we will be able to upgrade the VM to version 10. Click Next.
Step 10 – Select Other from the Guest Operation System list and Other (64 bit) from the Version drop-down list. Click Next.
Step 11 – Set the number of virtual sockets and cores per virtual sockets to 2 each. Click Next.
Step 12 – Assign 8GB of RAM and click Next.
Step 13 – Specify the Network the VM will connect to. This should correspond to a portgroup on the virtual switch listed under the requirements section. Also, set the adapter type to VMXNET 3 for optimal performance and tick on the Connect at Power On option. Click Next.
Note: Starting with vSphere 6, VMware Tools are embedded in the ESXi ISO image so you won’t need to install them manually. This also means that the VMXNET 3 adapter drivers are automatically installed.
Step 14 – Make sure the LSI Logic Parallel SCSI controller is selected and click Next.
Step 15 – Select Create a new virtual disk and set its size to 4GB. Select Thin Provision from the Disk Provisioning option and leave the remaining settings as set. Click Next at the next 3 screens shown.
Step 16 – Click Finish to create the VM.
Step 17 – It’s important that the VM’s virtual hardware is upgraded. This will expose the assisted virtualization features of the physical ESXi host’s processors to the underlying guest hypervisor. Right-click on the VM just created and select Upgrade Virtual Hardware. In the Summary tab, you will notice that VM Version changes from vmx-8 to vmx-10.
Step 18 – Using the vSphere Web client, log in vCenter Server. We need to apply a few final settings to the VM before we can install ESXi. This is the reason why vCenter Server is listed as a requirement since the vSphere Web Client is available on available when it is installed.
Step 19 – Right-click on the VM just created and select Edit Settings. Expand the CPU properties page and enable the Expose hardware-assisted virtualization to the guest OS option.
Step 20 – With the Edit Settings dialog box still open, switch to the VM Options tab and select VMware ESXi 5.x from the Guest OS Version list. If your physical ESXi host is running ESXi 6.0, a VMWare ESXi 6.x option will also be available. You can safely ignore the Unsupported warning.
Step 21 – We also need to make sure that we can host VMs with a 64-bit guest OS on the nested ESXi hosts after we finish installing them. From the VM Options tab, expand Advanced and click on the Edit configuration. Click Add Row and add the vhv.enable = “TRUE” line.
Note: This line must also be added to the /etc/vmware/config file on the physical ESXi host(s). To do so, enable SSH on the physical ESXi hosts and SSH to it using something like putty. Run the following command;
echo 'vhv.allow = "TRUE"' >> /etc/vmware/config
This last step completes the creation and customization of the nested ESXi shell VM. We can now clone the VM as needed.
Step 22 – Locate the VM just created and select Clone to Virtual Machine. Follow the steps shown in the next sequence of screenshots to create an additional 2 VMs for a total of 3 under the NestedESXi folder.
This concludes the first part of this series. So far, we’ve seen how to create VMs which are then used to host our nested ESXi hosts. In part two, we’ll go ahead and install ESXi 6.0 on the three VMs just created. Make sure to stay tuned for more!
[the_ad id=”4738″][the_ad id=”4796″]
Not a DOJO Member yet?
Join thousands of other IT pros and receive a weekly roundup email with the latest content & updates!