terrible vagrant/virtualbox performance on mac os x
I almost made it a year without a blog post.
I recently started using Vagrant to test our auto-provisioning of servers with Puppet. Having a simple-yet-configurable system for starting up and accessing headless virtual machines really makes this a much simpler solution than VMware Fusion. (Although I wish Vagrant had a way to take and rollback VM snapshots.)
Unfortunately, as soon as I tried to really do anything in the VM my
Mac would completely bog down. Eventually the entire UI would stop
updating. In Activity Monitor, the dreaded kernel_task was taking
100% of one CPU, and VBoxHeadless taking most of another. Things
would eventually free up whenever the task in the VM (usually apt-get
install
or puppet apply
) would crash with a segmentation fault.
Digging into this, I found an ominous message in the VirtualBox logs:
AIOMgr: Host limits number of active IO requests to 16. Expect a performance impact.
Yeah, no kidding. I tracked this message down to the "Use host I/O cache" setting being off on the SATA Controller in the box. (This is a per-VM setting, and I am using the stock Vagrant "lucid64" box, so the exact setting may be somewhere else for you. It's probably a good idea to turn this setting on for all storage controllers.)
When it comes to Vagrant VMs, this setting in the VirtualBox UI is not very helpful, though, because Vagrant brings up new VMs automatically and without any UI. To get this to work with the Vagrant workflow, you have to do the following hacky steps:
- Turn off any IO-heavy provisioning in your Vagrantfile
vagrant up
a new VMvagrant halt
the VM- Open the VM in the VirtualBox UI and change the setting
- Re-enable the provisioning in your Vagrantfile
vagrant up
again
This is not going to work if you have to bring up new VMs often.
Fortunately this setting is easy to tweak in the base box. Open up
~/.vagrant.d/boxes/base/box.ovf
and find the StorageController
node.
You'll see an attribute HostIOCache="false"
. Change that value to
true
.
Lastly, you'll have to update the SHA1 hash of the .ovf
file in
~/.vagrant.d/boxes/base/box.mf
. Get the new hash by running
openssl dgst -sha1 ~/.vagrant.d/boxes/base/box.ovf
and replace the
old value in box.mf
with it.
That's it. All subsequent VMs you create with vagrant up
will now
have the right setting.