Save to My DOJO
When was the last time you tried to remote console to a VM only to be greeted with the annoying Unable to connect to the MKS or similar message? Some of the causes leading to this issue range from firewalls blocking the required ports, badly configured DNS, and ESXi services in need of a restart. Today’s post explores a number of ways you can use to remote console to a VMware virtual machine and what you can do when remote connections fail.
Speaking of firewalls, adding a persistent firewall rule to ESXi can be daunting. I discuss how to do this in How to create persistent firewall rules on ESXi.
What is MKS?
MKS stands for Mouse, Keyboard and Screen. In vSphere, you can remote console to VMs using the standard vSphere clients and products such as VMware Workstation.
For pre-6.5 vSphere releases, you can use the legacy vSphere client. As shown next, just highlight the VM you want to connect to and click on the Console tab. Alternatively, right-click on the VM and select Open Console. The latter method opens a stand-alone console window for you.
If you have vCenter installed, the vSphere Web Client provides you with two remote console options these being VMRC (the standalone VMware Remote Console) and the Web Console. Clicking on the cog wheel on the VM’s console screen under Summary, gives you both options along with the ability to set the default console and install VMRC if it is not already. VMRC is installed by following the link given in vSphere client or by downloading it from here.
NOTE: Users who have VMware Workstation installed will have noticed that selecting Launch Remote Console brings up Workstation’s VMRC instead of the standalone one. To override this behavior, uninstall the standalone VMRC and reinstall it back.
Back to vSphere Web Client, clicking on the VM’s console screen or selecting Open Console from the VM’s context menu will launch the default remote console.
The behavior is almost identical when using the new vSphere Client (HTML5) or ESXi’s host client (embedded).
Interestingly, you can configure a VM such that you connect to it using a remote client such as tightVNC. To do this, add the following lines to the VM’s configuration (Edit Settings -> VM Options tab -> Advanced Settings -> Edit Configuration). You must disable the ESXi firewall for this to work or add a rule to allow 5900 through, this being the default VNC port. If memory serves me right, this worked with ESXi 5.x versions but to be honest, I had little time to try it out on 6.x. With ESXi 6.5, I manage to establish a connection but it keeps asking for a password regardless of the one specified in the VM’s configuration.
What can go wrong and how to fix it
Some of the MKS errors you might see are as follows:
- Error connecting: Host address lookup for server <SERVER> failed: The requested name is valid and was found in the database, but it does not have the correct associated data being resolved for.
- Error connecting: cannot connect to host <host>: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
- Error connecting: You need execute access in order to connect with the VMware console. Access denied for config file.
- Unable to connect to MKS: failed to connect to server IP:903.
Depending on the version of ESXi / vCenter Server installed, TCP ports 902, 903, 443 and 9443 need to be reachable when using the web based and standalone remote consoles. With vSphere 6.5, I managed to replicate a common problem where a personal firewall, or one sitting between the client computer and ESXi networks, is blocking one or more ports especially 902. To do this, I enabled the Windows firewall and created an outbound rule blocking 902 as shown.
When the firewall rule is enabled, the following is what you get when trying to remote console to a VM. The error message might differ slightly depending at which point the connection is dropped.
Changing the firewall rule to Allow from Block fixes the issue. Often, the lost ability to remote console occurs right after someone installs or rolls out security software incorporating a personal firewall. From my experience, other primary suspects include updates to group policies governing the Windows firewall and unannounced changes to ACLs on switches / routers. If you’re not the one administering the latter, talk to your colleagues to try and narrow down the source of the problem.
You can use netstat to verify that connections on port 902 are being established when you remote console to a VM. If you don’t see any entries, the issue is definitely firewall / network related. Another trick you can use is to telnet <ESXi IP> 902 from a command prompt or terminal. If you get a reply, it means that ESXi is listening on the port and ready for connections.
If you’re not running a personal firewall, SSH to ESXi and temporarily disable the ESXi firewall by running esxcli network firewall set –enabled false. Verify that you can connect. If not, it’s a firewall sitting in between the management station and ESXi that’s dropping the connection.
If the problem persists after having ruled out firewalling as a problem source, move on to the next check.
Connection to remote console can also be impeded if ESXi and vCenter Server hostnames do not resolve correctly. To determine correct DNS resolution, use ping <ESXi hostname> or nslookup to verify that your ESXi and vCenter Server hostnames are resolving to the correct IP addresses and, likewise, to the correct hostnames. If the issues persists, try adding your ESXi hosts and IP addresses to \windows\system32\drivers\etc\hosts, run ipconfig /flushdns and retry.
192.168.10.10 ESX65-a.lab.local 192.168.10.11 ESX65-b.lab.local 192.168.10.12 ESX65-c.lab.local
3. Free Disk Space
If firewalling and DNS are not the culprit, verify that the partition where /var lives has enough free disk space. To do this, run df -h from an SSH session on the ESXi hosting the VM you’re trying to remote console to. Make sure that the partition where /var is located is not full.
Another command you could use is vdh -h.
Verify that the permissions set on the VM’s VMX file are set correctly (755). To check the permissions, run the command:
ls -l *.vmx -rwxr-xr-x 1 root root 3995 Aug 18 10:00 Windows 7.vmx
We can see that the permissions are correctly set for the VMX file of a VM called Window 7.
- rwx = 111 = 7
- r-x = 101 = 5
- r-x = 101 = 5
If the VMX file permissions are not set to 755, run the following command:
chmod 755 'Windows 7.vmx'
Note: On ESXi 6.5, setting the permissions to read only had no effect so the permissions issues might only apply to older versions of ESXi.
5. If nothing else works!
If none of the above worked, try doing the following:
- Check that the time on ESXi is set correctly and in sync with the rest of the infrastructure.
- vMotion the VM you’re unable to remote console to, to another ESXi host and retry.
- Restart the management agents on ESXi as per the following KB.
- Reboot the VM.
- Reboot ESXi.
Note: Sometimes the fix is as trivial as pressing a key while the focus is given to the console window. This was common with older versions of the legacy client and vSphere releases where nothing is displayed in the VM’s console window. Hitting Enter or another key usually brings the console back to life. An alternative fix is to launch the console in its own window by right-clicking on it and selecting Open Console.
Being able to remote console to a VM is, of course, an integral part of any admin’s job. Seeing the dreaded “Unable to connect to the MKS” message and being able to console can be highly frustrating more so when you need to fix something fast. In most cases, the problem lies with a firewall blocking one or more required ports and missing or bad DNS entries. For the rest, it’s usually enough to vMotion a VM to another host or restart the management agent on ESXi.
If you enjoyed reading this post, why not have a look at The Complete List of VMware Articles.
[the_ad id=”4738″][the_ad id=”4796″]
Not a DOJO Member yet?
Join thousands of other IT pros and receive a weekly roundup email with the latest content & updates!