VMware ESX 4 can even virtualize itself

May 18, 2009 in Virtualizationism | 154 comments

spacer

NEW: VMware vSphere 5 makes this even easier and supports nested 64-bit guests.

Running VMware ESX inside a virtual machine is a great way to experiment with different configurations and features without building out a whole lab full of hardware and storage. It is pretty common to do this on VMware Workstation nowadays — the first public documentation of this process that I know of was published by Xtravirt a couple of years ago.

But what if you prefer to run ESX on ESX instead of Workstation?

You may be pleased to know that the GA build of ESX 4 allows installing ESX 4 as a virtual machine as well as powering on nested virtual machines — VMs running on the virtual ESX host.  You can even VMotion a running virtual machine from the physical ESX to a virtual ESX — on the same physical server!

VMware vSphere 4.1 UPDATE: VMware ESXi 4.1 has a keyboard issue when virtualized on an ESX 4.0 host.  In order to virtualize ESXi 4.1, the underlying host must be 4.1.  However, ESX 4.1 classic will work on ESX 4.0.

The extra tweaks to make it all work are minimal, and I will show you how without even opening up a text editor.

After installing ESX 4 onto your real hardware, configure as desired and enable promiscuous mode on a vSwitch:

spacer

Create a new VM with the following guidance (choose “Custom”):

  • Virtual Machine Version 7
  • Guest OS: Linux / Red Hat Enterprise Linux 5 (64-bit)
  • 2 VCPUs, 2GB RAM
  • 2 NICs – e1000
  • LSI Logic Parallel
  • New disk – reasonable size greater than 10GB for ESX classic

After you have the VM ready, simply attach a VMware ESX 4 ISO image, power on, and install ESX as a guest OS.

spacer

After installation, add the new virtual ESX to vCenter 4 and create a new VM.

spacer

If you do not need to run VMs on your virtual ESX, you can stop there.  However, if you try to power on that nested VM, you will see the following error:

spacer

To prevent this, just one tweak is needed:

  • Shut down the virtual ESX VM
  • Click Edit Settings
  • Click the Options tab
  • Click Advanced / General / Configuration Parameters…
  • Click Add Row
  • For the Name/Value enter: monitor_control.restrict_backdoor / TRUE

spacer

The above procedure is just an alternative to hand-editing the .vmx file — if you prefer to do it that way, feel free.

Now you are ready to power your virtual ESX VM back on, as well as the nested VMs.  This capability should come in handy as you start investigating the new features of vSphere 4.

UPDATE:  In order to take snapshots of a VM running ESX, a quick configuration change is necessary on the physical host .

You may be interested in this related post where a VM is migrated between the physical and virtual ESX hosts.

Related posts:

  1. VMware vSphere can virtualize itself + 64-bit nested guests
  2. VMotion from physical ESX 4 to virtual ESX 4
  3. Taking snapshots of VMware ESX 4 running in a VM
  4. If VMware ESXi 4 is so small, why is it so big?
  5. Pentacore CPUs in VMware ESX 4

Tags: ESX, installation, Technical, vSphere

154 comments

Comments feed for this article

Trackback link: www.vcritical.com/2009/05/vmware-esx-4-can-even-virtualize-itself/trackback/

  1. spacer

    Rick Vanover on May 18, 2009 at 7:32 pm

    Eric: Good stuff. I think this is a great way for administrators to get started with the base functionality of the product without allocating a tremendous amount of hardware. This won’t help too much with upgrades, but will definitely help with the base product!

    Cheers.

    Reply

    1. spacer

      PAramesh on July 8, 2010 at 8:26 pm

      Thx you very much . it was very use full for me .. i don’t find any Youtube video on this ..

      Reply

      1. spacer

        Louis Ndjouou on November 1, 2010 at 11:46 am

        This was very helpful stuff especially for testing environment. I actually had two ESXi 4.0 running as Guess within ESXi 4.0 on the physical machine.

        Reply

    2. spacer

      Mike on July 29, 2010 at 9:05 am

      Thanks, this works well. I do have a problem loading the tools on the vm’s I create on the virtual esx. Does anyone have any ideas.

      Thanks

      Reply

      1. spacer

        Mike on July 29, 2010 at 9:10 am

        i’ve also noticed that you can’t clone a vm from one virtual esx to another virtual esx.

        Reply

        1. spacer

          Eric Gray on August 3, 2010 at 8:35 pm

          You should be able to come up with a way to do this, but there are some issues — such as duplicate MAC addresses — that need to be dealt with.

          Feel free to chime in if anyone out there has some tips for Mike.

          Reply

  2. Pingback from VMware ESX 4 can even virtualize itself « DeinosCloud on May 18, 2009 at 11:29 pm

  3. spacer

    Hany Michael on May 19, 2009 at 12:36 am

    Eric,

    Great post. I’ve been struggling to achieve this since long time with no luck. Unfortunately though its not working on the ESX 4.0 RC, so I guess I’ll have to wait for the GA. Besides, I believe you need to add this inside the nested VMs config: mx.allowNested = “TRUE” ? or it’s not required any more?

    thanks

    Reply

  4. spacer

    Anton Zhbankov on May 19, 2009 at 4:03 am

    Eric, great post!

    Reply

  5. spacer

    Eric Gray on May 19, 2009 at 4:55 am

    Rick, you are probably right about not helping with upgrades from ESX 3.x, but going forward this environment could be used to test the ESX 4 patches and upgrades. Actually, I’m told that ESX 3 will also run in a VM like this but it is hardware dependent, and I have not tried it myself.

    Hany, that additional configuration element is no longer needed — it was in the past for certain configurations. By the way, internally VMware has been running ESX on ESX for a long time. A special build was required, though.

    Anton, spasebo vam! It’s very cool to see my articles translated into Russian. Especially if you take a look at my background on the About page…

    Reply

    1. spacer

      Mike on July 30, 2010 at 5:54 am

      Hi Eric,

      I am using esx on esx. I have a number of w2k3 vm’s running under it. I’m having problems loading the tools. It says Vmware tools should be installed inside a virtual machine.

      Any ideas? Is there another switch I need to set.

      Thanks
      Mike

      Reply

  6. Pingback from New vSphere feature allows ESX as a VM - in ESX - Virtualization Pro on May 19, 2009 at 7:45 am

  7. spacer

    Justin on May 19, 2009 at 10:29 am

    Eric,
    Would you mind testing out installing ESX 3.5 as a VM on ESX4? This would be huge for us here!

    Reply

  8. Pingback from Finally! Running ESX as a VM inside a physical ESX host!| Virtualization .. and every thing around it| HyperViZor on May 19, 2009 at 10:29 am

  9. spacer

    Mike Laverick on May 19, 2009 at 1:15 pm

    Great post. I would have posted something similiar myself but I was told I would be in breach of the NDA agreement I’d signed…

    Reply

  10. spacer

    Eric Gray on May 19, 2009 at 4:04 pm

    Justin, you can run ESX 3.5 in a VM on certain hardware — AMD Barcelona and Shanghai, probably Intel Nehalem.

    Reply

  11. Pingback from Welcome to vSphere-land! » Running ESX/ESXi in a VM Links on May 20, 2009 at 10:38 am

  12. Pingback from ESX as a VM - nothing new @ A. Mikkelsen on May 20, 2009 at 12:11 pm

  13. Pingback from Cheap vSphere Server « Virtualization, Windows, Infrastructure and all that “stuff” in-between on May 20, 2009 at 11:41 pm

  14. spacer

    Hany Michael on May 21, 2009 at 2:54 am

    Eric,

    I’m a bit confused here. I know that the ESX VM is 4.0, but what about the underlying physical ESX host, does it have to be ESX 4.0 also, or it can be ESX 3.5?

    I just tired the ESX 4.0 GA as a VM on ESX 3.5U3 but it didn’t work for me, the VM starts then a I get a blank screen and nothing happens, few seconds later the VM shutdown by itself and no errors are given.

    Any ideas?

    Thanks

    Reply

    1. spacer

      Eric Gray on May 21, 2009 at 4:57 am

      In my example, the physical host is also running ESX 4.

      Reply

  15. spacer

    Yue Yao on May 21, 2009 at 10:24 pm

    Hi Michael,
    In my testinstallation, vESX4.0(GA,RC1) is running on physical ESX3.5U4(but I have trouble with vswif0 after installation), and on W2K3+VMWorkstation 6.5.2.

    Reply

  16. spacer

    Astor on May 23, 2009 at 5:13 pm

    Eric,
    I try this with a ESXi4 final, but I can install and boot only Esxi3.5(very unstable).
    I can not install ESXi4, ESX3.5 and ESX4(Final)(PSODs and more PSODs).
    Where I download this version (GA)?
    My hardware is a whitebox(Athlon64 X2 7750 extreme)
    Sorry for my english, thanks.

    Reply

  17. spacer

    Hany Michael on May 23, 2009 at 10:53 pm

    thanks

gipoco.com is neither affiliated with the authors of this page nor responsible for its contents. This is a safe-cache copy of the original web site.