A couple of posts by Jeroen Frijters caught my attention this week. The first was on the Win Tech Off Topic list. Jeroen pointed to a paper(pdf) by VMWare employee Carl Waldspurger which describes a technique for sharing pages of memory which have identical content, either similar pages within a single guest OS or similar pages across multiple guests OS's running under a single host. VMWare server scans pages in use and generates a 64-bit hash based on their contents. The hash value is used as a lookup key into a hash table for other pages that have already been scanned. If a match is detected a full comparison of the pages is then made in the extremely unlikely chance that pages with different content have the same hash value. A copy-on-write policy is then followed with only a single copy of the page being held in memory, until one of the guest OS's writes to the page in which case a fault is generated and a separate copy of the page is created for the guest OS writing to the page.
There are more details about this and other techniques in the Waldspurger's paper. Page sharing is non-invasive as far as the guest OS is concerned, whereas "ballooning" requires the installation of a driver in the guest OS. The driver communicate with VMWare server via a private channel and is used when VMWare wants to reclaim memory from the guest OS: the balloon driver allocates pinned pages with the guest OS causing the latter to page memory out to its own virtual disk. VMWare can then reclaim the corresponding machine page. The balloon driver polls VMWare once a second to query what action it should take. If the guest OS requires more memory, the balloon is "deflated" by freeing the pages that the driver had allocated.