VMware Mac OS X Guest Package

1.0 Introduction
1.1 Background
VMware products allow guest operating systems to be run on a variety of hardware and operating
systems. Currently only VMware Fusion can run virtualized Mac OS X out of the box, and then only
the Server version due to EULA restrictions imposed by Apple. This new package allows Mac OS X
versions to be run as virtual guests on VMware’s Windows and Linux products, and on the ESX and
ESXi hypervisors.

The package supplies a VMware tools image, darwin.iso, which is suitable for running an Apple
Darwin based operating system within VMware. This image is used as a bootstrap for the guest and
is derived from the work done by David Elliott on boot-132, and the others that have also worked in
this area such as the Voodoo team. The background can be found at David Elliott’s web site
http://www.tgwbd.org/darwin/.

All the VMware products transparently mount the boot CD as in VMware Fusion. This behaviour is
hard coded into VMware products and cannot be changed. Darwin.iso contains the boot-132 code
and Mac OS X extensions to boot on version 7 of the VMware virtual hardware chassis. As VMware
uses signed SHA-1 digests to ensure the integrity of the VMware tools ISO images, we need to re-
sign all of the tools so that we can replace or insert the darwin.iso into the relevant program
directory. We do this using our public and private keys, as naturally VMware do not publish their
private key for obvious reasons. This was originally explained in this post on the web.

1.2 Supported VMware Products

The following products are supported when running on a machine with a suitable CPU:

 VMware Workstation 6.5 (Windows & Linux)

 VMware Player 2.5 (Windows & Linux)
 VMware Server 2.0 (Windows & Linux)
 VMware Fusion 2.0 (Mac OS X)
 VMware ESX 4.0
 VMware ESXi 4.0

1.3 Supported systems

CPUs need to be compatible with the retail version of Mac OS X. Please check that you have a
compatible CPU, mainly Intel Core and Core 2 processor types, before using this package. However
this is not a guarantee, as the CPU MUST also support VT-x extensions, and have a system BIOS that
enables VT-x. For example, Sony Vaio’s ship with compatible CPUs but does not enable the VT-x
extensions and cannot be used. There are 32-bit only processors that enable VT-x, and these can
also be used to run the retail version of Mac OS X in 32-bit mode. Intel provides a good list of VT-x
enabled processors here. Note that some of these processors still will not be suitable due to checks
in the vanilla Apple kernel; for example, Pentium D processors do not work. Finally there are

Version 1.0.0 © Donk 2009 Page 1

 VMware Mac OS X Guest Package

currently issues with Nehalem based processors, which VMware are working on. If it is not
compatible then this package will not work with the vanilla installers. It can be used for other
distributions but has not been tested and is not supported.

To summarize:

 VT-x present
 VT-x enabled in BIOS
 64-bit – can run 64 & 32-bit versions of Leopard
 32-bit – can run 32-bit version of Leopard

1.4 Supported Mac OS X Versions

Workstation Player Server Fusion ESX(i)

Tiger 10.4 ? ? ? ? ?
Leopard 10.5 Y Y Y Y Y
Snow Leopard 10.6 N N N Y N

1.5 Package Contents

File Description
AppleAC97Audio.kext.zip Audio driver
darwin.iso VMware Darwin tools ISO image
libeay32.dll Windows OpenSSL Shared Library
manual.pdf This document
openssl.exe Windows OpenSSL
readme.txt Text file with important information
setup.cmd Windows setup command
setup.sh Linux/Mac OS X/ESX(i) setup script
ssleay32.dll Windows OpenSSL Shared Library
tools-key.pem Replacement private key
tools-key.pub Replacement public key

2.0 Installation

2.1 Windows
You will need to be comfortable with using the command prompt with Administrator level privileges.
To setup the system follow these steps.

1. Unzip the archive to a folder

2. Open a command prompt
3. To install run the script as Administrator:
setup.cmd install

Version 1.0.0 © Donk 2009 Page 2

 VMware Mac OS X Guest Package

4. When it is running it will print details of the progress on the screen:

VMware ISO Signature Creator 2.0.0
==================================
Core=VMware Workstation
InstallPath=C:\Program Files\VMware\VMware Workstation\
C:\Program Files\VMware\VMware Workstation\tools-key.pub
1 File(s) copied
C:\Program Files\VMware\VMware Workstation\freebsd.iso.sig
C:\Program Files\VMware\VMware Workstation\linux.iso.sig
C:\Program Files\VMware\VMware Workstation\netware.iso.sig
C:\Program Files\VMware\VMware Workstation\solaris.iso.sig
C:\Program Files\VMware\VMware Workstation\winPre2k.iso.sig
C:\Program Files\VMware\VMware Workstation\windows.iso.sig
6 File(s) copied
1 file(s) copied.
1 file(s) copied.

To remove the changes follow these steps.

1. Open a command prompt

2. To uninstall run the script as Administrator:
setup.cmd uninstall

3. When it is running it will print details of the progress on the screen:

VMware ISO Signature Creator 2.0.0
==================================
Core=VMware Workstation
InstallPath=C:\Program Files\VMware\VMware Workstation\
.\backup\tools-key.pub
1 File(s) copied
.\backup\freebsd.iso.sig
.\backup\linux.iso.sig
.\backup\netware.iso.sig
.\backup\solaris.iso.sig
.\backup\winPre2k.iso.sig
.\backup\windows.iso.sig
6 File(s) copied

2.2 Linux and Mac OS X

You will need to be comfortable with using the terminal in these operating systems, and running
programs via sudo or as root. To setup the system follow these steps.

1. Unzip the archive to a folder

2. Open a terminal prompt
3. Make the setup script executable:
chmod +x setup.sh

4. To install run the script via sudo or as root:

sudo ./setup.sh --install

5. When it is running it will print details of the progress on the screen:

VMware ISO Signature Creator 2.0.0
==================================
VMware Workstation 6.5.2 build-156735
Running on Linux and using /usr/lib/vmware/isoimages
Backing up files
Creating new SHA-1 digests
SHA-1 digest for darwin.iso
SHA-1 digest for freebsd.iso
SHA-1 digest for linux.iso
SHA-1 digest for netware.iso
SHA-1 digest for solaris.iso
SHA-1 digest for winPre2k.iso

Version 1.0.0 © Donk 2009 Page 3

 VMware Mac OS X Guest Package

SHA-1 digest for windows.iso

/root/vmware-darwin-200
Finished!

To remove the changes follow these steps.

1. Open a terminal prompt

2. To uninstall run the script via sudo or as root:
sudo ./setup.sh --uninstall

3. When it is running it will print details of the progress on the screen:

VMware ISO Signature Creator 2.0.0
==================================
VMware Workstation 6.5.2 build-156735
Running on Linux and using /usr/lib/vmware/isoimages
Restoring backup files
Finished!

2.3 ESX and ESXi

The installation on ESX(i) is a little more complicated as we will need to copy files to the host
machine via vSphere client and then run locally on the host. This is so it can support the free ESXi
version as well as licensed versions, which have various APIs enabled that remain unusable in the
free version. You will need to be comfortable using the vSphere client and running scripts on the
host as root. For ESX the Console Operating System is a version of Redhat Linux, for ESXi it uses the
BusyBox program to give console support. Please check that you know how to access the relevant
consoles.

1. Unzip the archive to a folder

2. Start the vSphere client and connect to the host machine
3. Copy files to host system via the Datastore Browser:

Version 1.0.0 © Donk 2009 Page 4

 VMware Mac OS X Guest Package

4. Login to the console on the host

Version 1.0.0 © Donk 2009 Page 5

 VMware Mac OS X Guest Package

5. Change to the datastore folder used to store the files

cd /volume/vmfs/datastore/volumes/datastore1/vmware-darwin-200

6. Make the setup script executable:

chmod +x setup.sh

7. To install run the script:

./setup.sh --install

8. When it is running it will print details of the progress on the screen:

VMware ISO Signature Creator 2.0.0
==================================
VMware ESXi 4.0.0 build-164009
Running on VMkernel and using /usr/lib/vmware/isoimages
Backing up files
Creating new SHA-1 digests
SHA-1 digest for darwin.iso
SHA-1 digest for freebsd.iso
SHA-1 digest for linux.iso
SHA-1 digest for netware.iso
SHA-1 digest for solaris.iso
SHA-1 digest for winPre2k.iso
SHA-1 digest for windows.iso
/volume/vmfs/datastore/volumes/datastore1/vmware-darwin-200
Finished!

To remove the changes follow these steps.

1. Open a terminal prompt

2. Change to the datastore folder used to store the files
cd /volume/vmfs/datastore/volumes/datastore1/vmware-darwin-200

3. To uninstall run the script:

./setup.sh --uninstall

4. When it is running it will print details of the progress on the screen:

VMware ISO Signature Creator 2.0.0
==================================
VMware ESXi 4.0.0 build-164009
Running on VMkernel and using /usr/lib/vmware/isoimages
Restoring backup files
Finished!

3.0 Creating Guests

3.1 Create a virtual machine
Apart from Fusion none of these products support the direct creation of the relevant settings for a
Mac OS X guest directly in the user interface. To build a new guest you will need to create a guest as
close to what you require using the main programs, and then manually modify the generated VMX
file to add the relevant settings to allow the installation of Mac OS X.

The main settings to ensure are configured during the setup of a new guest are:

Version 1.0.0 © Donk 2009 Page 6

 VMware Mac OS X Guest Package

1. Create a FreeBSD virtual machine in 32 or 64-bit format depending on whether you want to
run the 32 or 64-bit version of Mac OS X. Fusion users can directly select the Mac OS X
version from the options menu.
2. Memory should be 1024MB as a minimum.
3. Disks should be created with about 16GB as a minimum size and using either:
a. IDE
b. SCSI with LSI Logic SCSI Adapter
4. Network adapter must be Intel E1000 which should be configured if FreeBSD selected as a
starting point.
5. Ensure that USB 2 is enabled in the guest. This can now be included on ESX 4.0 guests.
6. CD/DVD is also mandatory for the guest, and also I suggest you configure it to point to the
DVD disk or image you will be using during the installation process.

Power the guest on and off to allow some settings created at runtime to be added to the VMX file.

The settings in the screenshot are what I tend to use.

3.2 Modify virtual machine settings

Making sure that the virtual machine is off and has no snapshots or suspended sessions. If you are
using Server or ESX(i) please remove the guest from the Inventory before proceeding. Close VMware
and then open the VMX file in a text editor. The following lines will need to be either modified or
added to the file:

1. Guest operating system needs to be changed to allow internal virtual hardware needed to
support the Darwin kernel. The setting will be dependent on what you want to run.

Mac OS X Version Setting VMware Product

Leopard 10.5 32-bit darwin All
Leopard 10.5 64-bit darwin-64 All
Snow Leopard 10.6 32-bit darwin10 Fusion 2.0 only
Snow Leopard 10.6 64-bit darwin10-64 Fusion 2.0 only

Version 1.0.0 © Donk 2009 Page 7

 VMware Mac OS X Guest Package

An example is:
guestOS = "darwin-64"

2. Guest operating system name needs to be commented out for ESX(i), by adding a comment
character “#” at the front of the line:
#guestOSAltName = "FreeBSD (64-bit)"

3. Enable the ICH7M capabilities in the virtual hardware:

ich7m.present = "TRUE"

4. Tell the virtual hardware whether to pass through the AppleSMC controller to the guest OS.
If running on a real Mac this should be TRUE otherwise set it to FALSE.
smc.present = "FALSE"

5. Enable the virtual hardware USB emulation of keyboard and mouse. (Note this is not
dependent on the host keyboard and mouse configuration.)
keyboard.vusb.enable = "TRUE"
mouse.vusb.enable = "TRUE"

Finally, add the guest back into the Inventory on Server and ESX(i) systems. If you later decide to
modify settings of the guest there are 2 things to be aware of:

1. ESX(i) will display this harmless warning message which can be safely ignored .

2. Workstation on Linux will crash as it does not deal with the new “guestos” parameter. You
will need to manually change it back to “freebsd” or “freebsd-64” before making changes.
Sorry it is a bug in the VMware code and cannot be patched.

3.2 Install Mac OS X

Attach the physical DVD or ISO of the install DVD to the VMware session and then start it up. You
should boot to the installation DVD. Note that the DVD may become disconnected on ESX(i), if it
does then a popup message is displayed. Just reconnect the DVD and press the “c” key to continue.

Version 1.0.0 © Donk 2009 Page 8

 VMware Mac OS X Guest Package

You should then be at the main installer screen for language selection. Work through the setup as
for any other system. If you need to install as 32-bit version of Mac OS X, pass the “-legacy” flag from
the boot options screen. Remember to use Disk Utility from the menu to partition the virtual disk
before progressing into the installer.

4.0 Hints and Tips

4.1 VMware Tools

The darwin.iso supplied has the Fusion 2.0.5 tools and can be used to install to the guest after Mac
OS X has been installed. The current tools supplied by VMware are a subset of those found for the
other supported operating systems but do include:

 Copy & paste between host and guest

 Transparent mouse entry and exit to guest
 Shared folders
 VMware SVGA II video driver

4.1.1 Shared Folders

Shared Folders do not work out of the box and requires a change to the start-up shell script to work
properly. Again you will need to edit a file as root or via sudo to correct the problem.

1. Open a terminal session and get the uid from the id command
2. sudo nano -w /Library/Application Support/VMware Tools/services.sh
3. Edit the line
mount -t vmhgfs .host:/ '/Volumes/Shared Folders'

to
mount -t vmhgfs -o uid=<XXX> .host:/ '/Volumes/Shared Folders'

Version 1.0.0 © Donk 2009 Page 9

 VMware Mac OS X Guest Package

where XXX is the uid to be used for the guest user.

4. sudo '/Library/Application Support/VMware Tools/services.sh' –restart

4.2.2 Disk shrink

There is no tool available to reclaim unused virtual disk space in Mac OS X. However the effect can
be achieved simply using root or sudo from terminal prompt:

1. dd if=/dev/zero of=/zerofile bs=4096k count=1000000; sync; sleep 1; sync; rm -f /zerofile

2. /Library/Application\ Support/VMware\ Tools/vmware-tools-guestd --cmd "disk.shrink"

4.2 Video
VMware do not provide accelerated video drivers for running Mac OS X, and so you won’t get Quartz
Extreme support and Core Image works in software mode. Also at this time Workstation will not
automatically resize the screen when entering or leaving full screen mode. Fusion does this
seamlessly, and it is due to an extension to the VMware SVGA II specification which Fusion supports
and Workstation does not. To change screen resolutions you will need to pass the resolution at boot
or setup a default resolution in com.apple.Boot.plist. There are plenty of tutorials on how to do this
on the web. You can setup multiple resolutions in separate configuration plist files and select them
at boot using the “config” boot switch.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Kernel</key>
<string>mach_kernel</string>
<key>Kernel Flags</key>
<string></string>
<key>Graphics Mode</key>
<string>1280x1024x32</string>
</dict>
</plist>

4.3 Sound
Sound is a problem in that VMware emulates a Creative AudioPCI ES1371 card and the only know
driver for this was produced by Maxxuss. I have included that version in the package but
performance is variable, especially when running on a Linux platform. To use this, copy the extension
to the guest and install as normal for extensions.

An alternative solution I have used is the Rogue Amoeba Airfoil driver and network speakers. I have
acceptable performance running on a Windows XP host with this, and can recommend it. I do
suggest that you try it first as sound performance for VMware is highly variable. Please note that I
have no affiliation with Rogue Amoeba Software.

4.4 Keyboard & Mouse

There are reported problems using the virtual USB keyboard when not running on Fusion. This can
be corrected by reverting back to the PS2 emulated keyboard. I have included the Voodoo PS2
Controller extension in darwin.iso so you will not need to install any additional extensions. To switch
off USB keyboard emulation, you will need to edit the VMX file, as explained earlier, and set the
following value to FALSE.

Version 1.0.0 © Donk 2009 Page 10

 VMware Mac OS X Guest Package

keyboard.vusb.enable = "FALSE"

If you have mouse issues the same can be achieved be editing this setting:
mouse.vusb.enable = "FALSE"

If you want to make the keyboard more useable due to a PC layout then I suggest using Ukulele to
find a more suitable keyboard layout. Also the Apple key can be mapped to a Windows/Linux key in
the Settings Keyboard dialog.

5.0 Troubleshooting
Here are some hints to help with troubleshooting.

 Boot with -v flag and see if there are any error messages
 After a software update to Mac OS X itself boot once with -v -f -x flags immediately after
installation
 If you have a Nehalem based CPU it may not boot the guest. Try adding these 2 lines to the
VMX file:
monitor.virtual_exec="hardware"
monitor.virtual_mmu="software"

Finally enjoy yourselves!

Acknowledgements
 Apple – great OS
 VMware – great products
 InsanelyMac forums – help and feedback
 Voodoo Team – boot, kernel and extensions

History
26/06/09 - Version 1.0

Version 1.0.0 © Donk 2009 Page 11