Coding Horror: Creating Smaller Virtual = Machines

=AB=20 Why Can't Database Tables Index Themselves? | Main | I Pity = The Fool Who=20 Doesn't Write Unit Tests =BB

July 19, 2006

Creating Smaller Virtual Machines

Now that Virtual = PC is=20 finally free, I've become obsessed with producing the smallest = possible=20 Windows XP Virtual PC image. It's quite a challenge, because a = default XP=20 install can eat up well over a gigabyte. Once you factor in the swapfile = and=20 other overhead, you're generally talking about around 2-4 gigabytes for=20 relatively simple configurations.=20

My best result so far, however, is a 641 megabyte virtual = machine=20 image of a clean, fully patched Windows XP install. Not bad. And = here's=20 how I did it.=20

First, start with the obvious stuff:=20

Install Windows XP SP2. Take all default options.=20
Connect to Windows update; install all critical updates.=20
Install VM additions.=20
Turn off system restore.=20
- Right click My Computer; select properties=20
- Click the System Restore tab=20
- Click the "Turn off System Restore" checkbox=20
- OK all the way back out
Set Visual Effects to minimum.=20
- Right click My Computer; select Properties=20
- Click the Advanced tab=20
- Click the Performance Settings button=20
- Click the "Adjust for best performance" checkbox=20
- OK all the way back out.
Shut down.

Don't install anything else yet! Remember, we're trying to get = to a=20 minimal baseline install of Windows XP first. A nice, flat platform to = build on.=20

It's critical to turn off system restore, because that eats up = hundreds of megabytes of disk space. In a virtual machine environment, = having a=20 rollback path doesn't make sense anyway. And if the Windows software = environment=20 wasn't so pathological, we wouldn't need complex rollback support = embedded in=20 the OS, either, but I digress.=20

Now let's put together our toolkit of virtual machine optimization:=20

XPlite ($)=20
Crap Cleaner=20
TweakUI=20
Whitney=20 Defragger=20
Invirtus VM Optimizer ($, = optional)=20

Thes utilities are mostly free. And, except for Crap Cleaner, they = don't even=20 require installers. Just plop all the files for each one into a folder; = I call=20 mine VM-utils. Copy this folder to the target VM.=20

Use TweakUI to turn on = automatic=20 login. Otherwise you have to distribute login credentials = with your=20 VM, and who wants to do that?=20
Now, use XPlite = to tear=20 out all the annoying, unnecessary bits of Windows XP:=20
=20
XPlite is easily the best utility of its type; it removes scads of = useless=20 things built into XP that have no explicit uninstall mechanism. = Unfortunately,=20 XPlite is payware. There is a free version, but it's crippled; it can = only=20 remove a fraction of the items the full version can. See the full list = of=20 items it can remove along the right-hand side of the product page.=20
By default, XPlite generally shows things that are safe to remove. = Note=20 that the "Advanced Components" item is shown in that screenshot, which = is=20 definitely stuff that's not safe to remove unless you really = know what=20 you're doing. Anyway, here's what I consider totally safe to remove in = XPlite's standard list:=20
- Accessibility Options=20
- Communication and Messaging=20
- Server Components=20
- Games=20
- System Services
The others require a bit of judicious selection.=20
- Accessories - you probably want Notepad, Calc, and the other = essential=20 applets. A world without Notepad is a world I don't want to live in. =
- Internet Utilities - if you want to keep the default IE6 inside = XP, I'd=20 leave this alone. With the notable exception of MSN Explorer, which = is=20 always safe to drop.=20
- Multimedia - if you have sound enabled, selectively keep some of = this,=20 otherwise dump it all. It's highly unlikely you would ever want to = watch=20 videos or listen to music inside your VM, right? Right?=20
- Operating System Options - you may want to keep the core fonts = if you're=20 planning to browse the web within the VM. Also, beware of removing = the=20 service pack update files. Most of this is safe to dump, though. = However,=20 you will need the VB6 runtimes for Crap Cleaner to run!=20
- System Tools & Utilities - I'd leave Dr. Watson, and = possibly=20 PerfMon, WSH and Zip folder support.
Once you've made your selections, let XPlite do its thing. It's = worth the=20 effort, because you'll have an unbelievably squeaky clean Start menu = when it's=20 done. Who knew Windows XP could be this.. simple?=20
Install and run Crap = Cleaner.=20 Perform the default analysis, then do a cleanup. This step is = really=20 optional; it only cleans up a couple megabytes of log files and = miscellaneous=20 junk. Be sure to uninstall Crap Cleaner when you're done, too.=20
Now that we've cleaned everything up, we need to defragment the = disk.=20
=20
You can use any defragmenter you like, of course, but this one is = free and=20 works quite well.=20
1. Navigate to the folder where you put your VM utilities, = including the Whitney=20 Defragger.=20
2. Open a command prompt=20
3. Copy the defragmenting program to our windows system folder:=20
```
copy bootdfrg.exe c:\windows\system32\
```
4. Install the defragmenting service:=20
```
defrag -i
```
5. Schedule a defragmentation of the c:\ drive for the next boot:=20
```
defrag -d c: -B
```
6. Restart the virtual machine.=20
7. The defragmenter will run before Windows loads. Let it run to=20 completion. It may take a little while, but it provides lots of = textual=20 feedback on what it's doing.=20
Now we have to zero the free space on the drive. You have = your=20 choice of the free Microsoft Virtual PC Pre-Compactor, or the = inexpensive Invirtus VM Optimizer. Both do = the same=20 thing, but the Invirtus tool results in an image that's about 15 = percent=20 smaller (641 megabytes vs. 758 megabytes, in my test) than the = Microsoft tool.=20
Either way, you're mounting an ISO. The Microsoft Pre-Compactor is = in a=20 folder named "Virtual Machine Additions" under your Virtual PC install = folder.=20 Once mounted, the precompactor will autorun. Let it prep the drive; = this=20 doesn't take long.=20
Cleanly shut down the virtual machine.=20
Finally, shrink the virtual machine hard drive using the = disk=20 wizard available from the Virtual PC UI:=20
1. Click the File | Virtual Disk Wizard drop-down menu=20
2. Edit an existing virtual disk=20
3. Select the correct disk image=20
4. Select "Compact it"=20
5. Select "replacing the original file"
.. and prepare to marvel at the tiny size* of the resulting hard = drive=20 image!

It's really quite amazing how snappy and compact Windows XP can be, = once=20 you remove all the useless cruft from it.=20

* that's what she=20 said.=20

Posted by Jeff Atwood at July = 19, 2006=20 11:59 PM | TrackBack=20

Comments

The resulting 758 megabyte .vhd (Virtual Hard Drive) image is also = highly=20 compressible.

I created a self-extracting RAR image of the .vhd and the .vmc file = -- it's=20 only *243* megabytes!

Really cool!

Jeff=20 Atwood on July 20, 2006 11:19 AM

Have you tried running VMOptimizer on the final image, to see if you = can=20 squeeze it down a bit more?

http://www.invirtus.com/

Kevin=20 Dente on July 20, 2006 11:37 AM

This is del.icio.us material! What a cool concept that you can = distribute a=20 full VM on a single CD-R!

Since my laptop doesn't have a DVD Burner, I struggled with this.

However, as soon as you install Visual Studio / Sharepoint / SQL = Server, and=20 such it will push it over the 800mb limit, but at least you can have a = CD of=20 something to start with.

Good work.

Ben=20 Scheirman on July 20, 2006 11:39 AM

You may want to look at the free nLite tool to create a stripped-down = Windows=20 XP CD. It appears to include some of the same functionality as XPlite, = but=20 applies it before the OS is installed.

http://www.nliteos.com/

Alex=20 Tsang on July 20, 2006 11:50 AM

Nlite's great for creating very stripped-down disc images to install = from. It=20 provides cleaning options very similar to XpLite and is free.

Pat=20 cavit on July 20, 2006 12:05 PM

it would be nice to have a heavily trimmed down windows xp install = entirely=20 geared towards gaming

for us linux and mac users to install on another partition

oldman=20 on July 20, 2006 12:09 PM

Compress it with 7-Zip instead of RAR, and you'll save even more = space.

http://www.7-zip.org/

It's free (LGPL).

C-J=20 Berg on July 20, 2006 12:43 PM

> Have you tried running VMOptimizer on the final image

I was skeptical, but VMOptimizer worked. You use their special ISO to = compact=20 instead of the buil in Pre-Compaction ISO. Here's what I got...

Original size: 758 megabytes
VMOptimizer size: 641 = megabytes

Jeff=20 Atwood on July 20, 2006 12:44 PM

Did you turn off hibernation support? hiberfil.sys is a big as the = VM's=20 ram.

Leonel = on July 20,=20 2006 01:58 PM

I have an XP boot CD made using...something (I forget what), so = that's=20 obviously a very small build.

[ICR] on July 20, 2006 02:04 PM =

"Unfortunately, XPlite is payware"

Why is paying for a good product an issue? Don't you
like to get = paid for=20 the develpment work you do?

Like many out there I like open source and free whenever it makes = since but=20 clearly you liked the product and shouldn't the developer or team that = created=20 be rewarded.

VHF on July=20 20, 2006 02:12 PM

> hiberfil.sys is a big as the VM's ram.

Hibernation is generally off by default in XP.. I always have to = manually=20 enable it.

> Why is paying for a good product an issue?

It's not an issue. In fact I bought a copy of XPlite during the = composition=20 of this post. But one of these things is not like the other..

Jeff=20 Atwood on July 20, 2006 02:22 PM

I'll second the VMOptimizer recommendation. It saves tons of time, = does a=20 great job, and it's a very inexpensive. One of the few pieces of = software I was=20 happy to buy.

And 7-Zip's worth it when you get to files that size. I wrote up some = info on=20 it before here:
= http://weblogs.asp.net/jgalloway/archive/2005/05/07/406074.aspx
I = think one of the switch names may have changed in newer 7-Zip = releases.

True, a lot of people don't have 7-Zip installed, but it's really = easy to=20 make an SFX (self-extracting zip), so you distribute an EXE file which = unzips=20 itself.

Jon=20 Galloway on July 20, 2006 03:01 PM

Great post, but seriously, I'm amazed you'd waste this kind of time = and not=20 just use Virtual Optimizer.

http://www.hanselman.com/blog/InvirtusVirtualMachineOptimizer.aspx=

On my post you said this functionality wasn't worth $40. You MUST = have spent=20 at least an hour messing with this and writing this post. How much are = you paid=20 an hour? ;)

Scott=20 Hanselman on July 20, 2006 05:16 PM

And another thing...no way I'm installing a defrager made by the = "Whitney=20 Family." Who do you trust?

pagedfrg by SysInternals:
http://www= .sysinternals.com/Utilities/PageDefrag.html

Scott=20 Hanselman on July 20, 2006 05:18 PM

> I'm amazed you'd waste this kind of time and not just use = Virtual=20 Optimizer.

The Virtual Optimizer only zeroes out the drive. it doesn't..

1) Reduce the size of Windows XP
2) Defragment the drive

It does provide a 15% size reduction, which is nice, but it's hardly = the=20 entire point of the article. It does work as advertised, but personally, = I'd=20 rather spend that $40 on XPlite.

> no way I'm installing a defrager made by the "Whitney Family." = Who do=20 you trust?

Yes, it might destroy my.. virtual.. machine.. oh no. ;)

Jeff=20 Atwood on July 20, 2006 09:51 PM

There was a big thread at neowin.net's forum where some guys tried to = have=20 the smallest windows folder. One of those guys wrote a complete guide = which you=20 can read at htt= p://www.bold-fortune.com/forums/index.php?showtopic=3D229=20

Everything is done manually

FlorentG on July 21, 2006 12:57 AM =

I wonder if Parallels' compressor (trialware) would compress any = better than=20 Invirtus VM Optimizer or Virtual PC?
htt= p://www.parallels.com/en/products/compressor/workstation/

George=20 on July 21, 2006 02:15 AM

Why free matters: I'm allowed to use free utilities at work, but = anything I=20 have to pay for requires a lot of red tape. It is actually easier to get = a $600=20 tool than say a $40 tool.

rsbohn=20 on July 21, 2006 05:18 AM

I assume you did this, but not specifically mentioned:

Go into the Windows directory and delete all the patch backup = directories,=20 these can easily add up to > 50MB space on disk.

Richard on July 21, 2006 06:55 AM =

Jeff:

Have you also gotten rid of all of the driver cab files. Your = hardware is a=20 known issue, so you shouldn't need any of the autodetect hardware stuff. = I=20 believe it is "%windows%/driver cache." In the past when Connectix owned = it, I=20 would even minimize the size of the swapfile. With 2GB RAM, my VM's are = hardly=20 noticable during demos.

I must say though, if anyone reading this is going to do = presentations with=20 VM's--don't say..."It's the VMs fault"--it's your fault if you haven't = prepared=20 a correctly working VM (including plenty of RAM to go with it)! It's = like=20 Hanselman's comment on "I know you can't read this, but..." See=20 hanselman.com/tools for ways to fix that too.

Brian=20 Sherwin on July 21, 2006 07:18 AM

Hi,

I thought you might be interested in this thread concerning the so = called=20 "TinyXP" build that is claimed to be sub 70 MB in size.

htt= p://www.xtremesystems.org/forums/showthread.php?t=3D84885

I haven't personally verified claims, so i will be very interested in = the=20 results if you choose to test it.

Here's another idea - how come we don't see VM image freely = distributed over=20 Bit-Torrent and such? I mean as far as i understand - if the system is=20 Sysprep'ed prior to distribution - it will require license key and in = many ways=20 will be indentical to a regular install, so that doesn't necessarily = equate to=20 piracy. At least that's not my intention bringing it up.

Fine, maybe Microsoft can be opposed to 3rd party distribution for = one reason=20 or another - but they themselves could defenitely distribute beta = versions of=20 their products in such form, no?

Vlad=20 Ivanov on July 21, 2006 07:35 AM

I am sorry, i had the link in my bookmarks for a while, so my memory = of the=20 contents was a little bit hazy - i just visited the link again prompted = by your=20 post and i'd like to make a correction:

It's a 100MB install that results in a 400MB installed image, that is = claimed=20 to only use 40MB of RAM to run.

Seems like still an improvement over your results.

Vlad=20 Ivanov on July 21, 2006 07:42 AM

You should change your path from:

C:\windows\system32\

to:

%WINDIR%\system32\

:-)

Omar=20 Shahine on July 21, 2006 08:40 AM

> Have you also gotten rid of all of the driver cab files. Your = hardware=20 is a known issue, so you shouldn't need any of the autodetect hardware = stuff. I=20 believe it is "%windows%/driver cache."

Good idea. I ticked those two boxes in XPlite (remove cached server = pack=20 files, remove cached driver files). I tried to run Invirtus again, but = it=20 refused -- "can only run one time per VM for trial edition". Instead, I = ran the=20 invirtus drive zero-ing command line tool ( InvirtusFreeSpace.exe ) = that's on=20 the Invirtus trial ISO image. That produced the same compaction results = as doing=20 an autorun (!)

The resulting image is now 587 megabytes.

Jeff=20 Atwood on July 21, 2006 10:18 AM

Does all this work with virtual server too, or this a vpc thing=20 only?

Malcolm=20 Anderson on July 21, 2006 03:20 PM

Here's an easy one:

Still not using that good old modem and are not planning to in the=20 foreseeable future? Got a xDSL line, Wifi connection or LAN you always = hook up=20 to?

I thought so...

Then why not kill all the modem driver description files from=20 "%systemroot%\inf"? They're the "mdm*.inf" ones and they take up quite a = few=20 megabytes.

Btw.: Does anyone know a simple way to locate other deletable driver = files...=20 like say: old ISA-card drivers, drivers from obsolete graphic cards and = TV-tuner=20 devices?

Henrik=20 Spuur Hansen on July 21, 2006 05:56 PM

this post is a keeper!

(lb tucks the entry away in his del.icio.us account)

lb=20 on July 23, 2006 03:13 AM

You can also delete %WINDIR%\$NtUninstall--------$ folders: I am = assuming=20 that you would keep a backup of 'original' virtual image somewhere safe = (like=20 DVD or CD), and these folders can run up to several hundred MB..

I would also reduce/disable trash bin size and space your browser = allocates=20 for disk cache.

Maksim=20 Pavlishin on July 24, 2006 11:55 AM

Has anyone had any luck with Win2k3? I can only get to about 1.5 GB = with SQL=20 Server and IIS.

dave on=20 July 31, 2006 01:37 PM

Another interesting idea: once the "base" drive image is created you = can=20 switch the virtualPC machine to use a differential drive based on the = "base"=20 drive. This differential drive can be kept in sync using FolderShare so = you can=20 use the same Virtual machine from different computers without many=20 synchronization troubles.

Paperino=20 on July 31, 2006 03:47 PM

Sounds great - no good for test machines where you need it to be a = pure=20 'out-of-the-box' self-resetting VM, but just the ticket for specialist = needs,=20 such as setting up a Subversion server, or creating a sandboxed Internet = browser=20 VM.

Hmm - your filter is now blocking (dot)info web addresses as = 'questionable=20 content'.

Paul=20 Coddington on August 1, 2006 05:58 AM

Just shrunk my Win2k3 base drive to < 1GB using the methods = outlined here=20 and the Invirtus optimizer.

Thanks!

Bryant=20 Likes on August 22, 2006 05:16 PM

We've got a 14Mb XP iso booting cmd.exe as shell and a standard = project that=20 will create a 50Mb XP boot CD with explorer as shell (not to mention a = few other=20 features like the hability to boot and run from RAM)

This is done using WinBuilder and the projects that come included = with it.=20 All our works and developments are free to everyone.

Nuno=20 Brito on August 25, 2006 04:27 AM

hey, just randomly came across this when searching for something = else... just=20 goign to say, i run TinyXP and it does what it says, i'm using it as a = main OS=20 though as my windows xp was broken and i didnt realise when i formatted = my pc...=20 but tinyxp does use 40mb of ram and around about 400mb installed... it's = a great=20 OS.

lonewolfxix on=20 September 13, 2006 03:42 PM

Hmmm, seems possible to get it small enough to fit on a flash drive.=20

lee on = November 4, 2006=20 10:55 AM