I’ve been lusting after my very own ESX install at home for a while now, especially after following the news that you get ESXi 5 running on a Mac Mini. A key factor for me is that I want to run multiple OS X instances so the Mini is a requirement without going down the hackintosh route. The other is purely practical as I don’t really have any available space so whatever I build has to fit on my desk with the rest of the stuff already there. Noise and power consumption are key factors.
Starting with a Mac Mini Server (5.3) as the base component. I debated going with one of the regular models, but the fact that it has two 500Gb internal drives plus a quad core i7 gave it the edge. RAM is currently pretty cheap so I pushed it to 16 Gb with aftermarket memory.
I intend to run a mix of production machines and pure test bed stuff. So the idea is to have a VM running Solaris have a mirrored zpool created from two VMDK files, presented locally via NFS for any production machines which will be replicated to the main NAS (an HP N40L running Solaris) via my auto replicate scripts.
The left over space on the two drives will be used for all of the miscellaneous test machines that I don’t care about so losing a drive is not going to bother me. I debated hosting the production machine directly on the NAS, but it’s getting a little full and I’d prefer to do that with a proper isolated storage network and I need to buy an additional Thunderbolt GbE adaptor for the Mini and a network card for the HP so that can wait.
Step by step
I started with the ready to go ESXi 5.0 image available from Angry Cisco Guy. Simply burn it to a CD, and using the external Apple DVD player boot it up (hold down C on startup). Then it’s just a standard ESXi install process.
Note: there are some excellent instructions for building your own custom installer over at Paraguin and Virtually Ghetto that you can use to go straight to 5.1 with the latest driver, but I ran across the Angry Cisco Guy link first and had already downloaded the image so I went the lazy route.
Now that I have the baseline running, I opened up the SSH shell and copied over the 5.1 offline update bundle and installed it using:
esxcli software install vib -d ....
After a little touring around, I found the latest tg3 Broadcom driver (which would be required for the Thunderbolt adaptor) and installed it, again using esxcli.
Note that the bundle you download from VMware is not directly installable. You need to unzip it and use the offline-bundle file inside.
VMs
Now that the machine is running nicely, I installed the Solaris VM, assigned it two VMDK files, one from each internal drive.
This allows me to create a mirrored zpool for the additional read performance and reliability.
On the ESX side, I mount the new NFS share and I’m ready to deploy my production machines.
The first project is a replacement server for my aging second generation Mini that is an OS X Server instance that has been running since 10.1 and getting upgraded to every version up until 10.7 so it’s just full of cruft.
I had started building the replacement server in Fusion on my MacBook, so I figured that I should just convert it over. This however is not something sufficiently mainstream to be supported by the regular P2V or V2V tools out there.
I tried a number of different approaches including VMware Converter Standalone, copying the Fusion VMDK file to the NFS share and mapping it to the VM without any luck.
So the best approach to virtualization OS X computers seems to be to create a new VM on ESXi and then use the Apple Migration Assistant to bring over the old machine’s files and settings.
At the moment, ESXi has support for OS X machines up to 10.7 so I started there, mapping the installer dmg file to the new VM and installing from there. VMware does have VMware Tools for OS X so these got installed too. Then I copied the 10.8 installer image into the VM and ran it with no problems.
A quick tour with Software Update and I now have a fully up to date VM.
But then I ran into a limitation that the Migration Assistant will not do a machine to machine migration for a machine using OS X Server. I tried restoring from a Time Machine backup, but for some reason the Migration Assistant didn’t like the backup in question (running on Netatalk on the Solaris servers). So I had to fall back to the tried and true SuperDuper clone of the disk to a network share and play musical disks mapping the disk to a temporary VM to do the restore, disconnecting and booting the destination machine.
OS X is sufficiently hardware agnostic that cloning the drive from a physical Mac should work equally well.
Gotchas
A few things that may come back to bite you with the default settings applied by OS X:
- The screen saver is on by default which chews up CPU, and makes it almost impossible to VNC in some of the time.
- The power saving features are on by default and the machine will go to sleep so you need to disable these options
- Be very careful with international keyboard mappings when assigning the initial password. I was going from a Mac via RDP to a Windows machine, to the VI-Client console of the new OS X VM and my first attempt was impossible to recreate. Keep it simple until you get to the native Screen Sharing connection.
Side notes
The Mac Mini’s only real fault that makes it ineligible to be a supported ESXi platform is the fact that it doesn’t support ECC RAM. So if you have a Radar account, filling out a request might be help Apple decide to add that into future versions.
Something else to keep in mind about the current generation of Minis is that Thunderbolt is nothing more than a transparent extension of the PCI bus. So if you have the budget, you can imagine some silly/amazing/freaky configurations using the Sonnet External PCIe boxes with a 10GbE or FC card.
They also have a really interesting rack mount kit that includes PCIe slots. But it’s horribly expensive compared to a standard off the shelf 1U Intel based Server.