01-The Test Bench and Test Types
TweetFollow Us on Twitter

01-The Test Bench and Test Types

The Test Bench

When we were choosing computer models, we chose the current model of Macs that give a good representation of what most people may have. Certainly, the faster models of these computers will perform even better.

We chose five Mac models to compare alongside each other: MacBook Air, MacBook Pro, a tricked out MacBook Pro Retina, iMac (late 2011), and Mac Pro. Given the RAM requirements of Windows 7, the minimum configuration tested (including on the MacBook) was 4 GB.

8 GB MacBook Air 13-inch, 1.8 GHz dual-core Intel Core i5 processor
Specifically: 8 GB/256 GB, Intel HD Graphics 4000

4 GB MacBook Pro 15.4-inch, 2.3GHz quad-core Intel Core i7 processor
Specifically: 4GB 1600MHz memory/500 GB 5400-rpm
Intel HD Graphics 4000 and NVIDIA GeForce GT 650M with 512MB of GDDR5 memory

16 GB MacBook Pro with Retina 15-inch, 2.7GHz quad-core Intel Core i7 processor
Specifically: 16GB 1600MHz DDR3L SDRAM/768GB Flash Storage
Intel HD Graphics 4000 and NVIDIA GeForce GT 650M with 1GB of GDDR5 memory

4 GB iMac 27", 2.7GHz Quad-Core Intel Core i5 processor
Specifically: 4 GB/1 TB 7200-rpm
AMD Radeon HD 6770M with 512MB

6 GB Mac Pro, One 3.2GHz Quad-Core Intel Xeon processor processors
Specifically: 6 GB/1 TB 7200-rpm / ATI Radeon HD 5770 with 1GB GDDR5

Memory for virtual machines can be configured with a wide array of settings. As a rule, both VMware Fusion's and Parallels Desktop's default memory for each configuration (of physical RAM and "guest" OS) were the same, and we made sure that was the case. Windows 7 and 8 virtual machines ran with 1 GB of virtual machine RAM (except for gaming). Lion and Mountain Lion (OS X) ran with 2 GB of virtual machine RAM. For gaming, we used 1.5 GB for 4 GB hardware, and 2 GB on hardware with 6 GB or more.

Similarly, for disk allocation we set up all of the virtual machines similarly. We used default sized (64 GB on Parallels, and 60 GB on VMware) expanding disks in a single file (e.g., not 2 GB chunks), but the disks were pre-expanded so that expanding actions wouldn't affect results. The disk location of the virtual hard drive was in a similar physical location between computers, as that can make a significant difference in disk performance.

The tests compared VMware Fusion 5.0.2 with Parallels Desktop for Mac 8.0.18314.813278, running on Mac OS X 10.8.2 with all updates. All "important," but not "optional," Windows updates also installed for Windows 7 and 8. By the time we got to testing games, we saw new versions released that could have impact, and therefore (games only) were tested on VMware Fusion 5.0.2 with Parallels Desktop for Mac 8.0.18354.823166, including additional updates to OS X 10.8.2.

Test Types

There are a variety of often referred to, and utilized benchmarking suites in the computer industry including SPEC tests, PCMark, WorldBench, Performance Test, Unixbench and others. Each of these tests uses a series of tests that measure in a consistent way to assess specific types of performances. The key to each of these tests is appropriateness, repeatability, and accuracy.

We are sometimes asked how we select the tests we have in benchmarks. The goal is to represent the types of actions that virtualization users are doing on a regular basis. In particular, we focus more on user tasks rather than installation or configuration (because they are typically only done once or infrequently).

In the PC market, PC World (magazine) uses WorldBench as their benchmarking tool of choice. If you don't know about WorldBench, you can see more at http://www.worldbench.com. WorldBench 7 uses a combination of applications bundled together to determine benchmarks. This may be a very good way for PC World to approach things, but obviously it's not reflecting the virtualization part of the experience or weighing towards the typical types of things people do in virtualization on the Mac.

There are a variety of other benchmarks available, and we've looked at many of them as possible tools to include in our mix. Often, we find their measurements simply don't reflect a true user experience. Other times, they don't reflect how a virtualization user may be using Windows, or even they simply report erroneous results. For example, in previous attempts to use PassMark, we found their graphics scores were not at all representative of real life graphics performance in games or other graphic uses. Those tests showed items as faster when, in fact, they were much slower. Benchmarks are only useful if they reflect the real world.

Rather than use WorldBench or others, we focus on the types of measurements we believe best represent the experience we see (in terms of speed). And while it takes practice and some skill, we test virtualization operations with a stopwatch—as the closest representation of how a real user would see it. CPU crunching ability, for example, is measured through tests like zip'ing files directly in Windows. For Office users, we use the most up to date version of Microsoft Office for Windows (2010), again with updates.

There are two exceptions to this: graphics/gaming and CPU utilization. In these two cases, we found that testing utilities not only work well, but also are necessary to give the most repeatable and concrete results. In the case of graphics benchmarking, we used 3DMark06 (the industry standard for tuning gaming systems). For game playing performance, we used FRAPS to measure frames per second. Finally, for CPU utilization we used "top" combined with Apple Remote Desktop.

Remember, benchmarks are not a review: they are benchmarks. They are simply meant to just tell you which runs faster. If you are interested in a specific feature, support, the user interface, or any of the other criteria for deciding on a product, that's a different article.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

calibre 6.18 - Complete e-book library m...
Calibre is a complete e-book library manager. Organize your collection, convert your books to multiple formats, and sync with all of your devices. Let Calibre be your multi-tasking digital librarian... Read more
War Thunder 2.25.1.119 - Multiplayer war...
In War Thunder, aircraft, attack helicopters, ground forces and naval ships collaborate in realistic competitive battles. You can choose from over 1,500 vehicles and an extensive variety of combat... Read more
Garmin Express 7.17.0.0 - Manage your Ga...
Garmin Express is your essential tool for managing your Garmin devices. Update maps, golf courses and device software. You can even register your device. Update maps Update software Register your... Read more
Wireshark 4.0.6 - Network protocol analy...
Wireshark is one of the world's foremost network protocol analyzers, and is the standard in many parts of the industry. It is the continuation of a project that started in 1998. Hundreds of... Read more
BBEdit 14.6.6 - Powerful text and HTML e...
BBEdit is the leading professional HTML and text editor for the Mac. Specifically crafted in response to the needs of Web authors and software developers, this award-winning product provides a... Read more
Microsoft OneNote 16.73 - Free digital n...
OneNote is your very own digital notebook. With OneNote, you can capture that flash of genius, that moment of inspiration, or that list of errands that's too important to forget. Whether you're at... Read more
iMovie 10.3.6 - Edit personal videos and...
With a streamlined design and intuitive editing features, iMovie lets you create Hollywood-style trailers and beautiful movies like never before. Browse your video library, share favorite moments,... Read more
Microsoft Remote Desktop 10.8.3 - Connec...
Microsoft Remote Desktop for Mac is an application that allows connecting to virtual apps or another PC remotely. Discover the power of Windows with Remote Desktop designed to help you manage your... Read more
MegaSeg 6.3 - Professional DJ and radio...
MegaSeg is a complete solution for pro audio/video DJ mixing, radio automation, and music scheduling with rock-solid performance and an easy-to-use design. Mix with visual waveforms and Magic... Read more
Carbon Copy Cloner 6.1.6 - Advanced back...
Carbon Copy Cloner is an advanced backup and file copying application for macOS. Looking for something better than Time Machine? With just a few clicks you can set up CCC to make hourly or daily... Read more

Latest Forum Discussions

See All

TouchArcade Game of the Week: ‘Super Cat...
If you have been a reader of this website in any way over the years, chances are pretty good that you’ve heard one of us singing the praises of the Super Cat Tales series at some point in time. The original set the benchmark for unique and intuitive... | Read more »
SwitchArcade Round-Up: ‘Skye Tales’, ‘Fi...
Hello gentle readers, and welcome to the SwitchArcade Round-Up for May 26th, 2023. Gosh, yesterday was quite the day. Counting the few games that popped after I finished my writing, there were forty-five new games that hit the eShop. Today is a... | Read more »
‘Cat Quest: Pirates of the Purribean’ An...
Developer The Gentlebros. released the original Cat Quest way back in August of 2017, and it was a game we had been eyeing for more than a year at that point and were incredibly excited for. In short, it did not disappoint. | Read more »
Minecraft 1.20 Trails and Tales Update R...
Mojang and Microsoft previously revealed the official name and more details for the Minecraft ($6.99) 1.20 update. The Minecraft 1.20 Trails and Tales update was confirmed for release this year, and it has been confirmed to release on June 7th... | Read more »
Apple Arcade Weekly Round-Up: Updates fo...
Just like last week, a few notable games on Apple Arcade have gotten updates this week with no new releases. If you’re wondering why there is no new game this week, Apple’s release of a few Apple Arcade Originals and App Store Greats earlier this... | Read more »
The Latest ‘Marvel Snap’ Update Introduc...
It’s been a busy week over in the world of Marvel Snap (Free). Following on from last week’s big update, this week saw a major new card introduced, a handful of significant balance tweaks over the air, and a new update schedule from the team. I... | Read more »
Recruit yourself a shockingly powerful a...
Tower of Fantasy welcomes their latest simulacrum to the field, and she is one powerful character. The former head of the Listener Project, Dr. Rubilia is swapping research for adventuring and comes with some truly powerful skills. [Read more] | Read more »
‘Yolk Heroes: A Long Tamago’ is a Hybrid...
Virtual pets are pretty cool. Raise, take care of, and ensure the safety of a little virtual creature that lives inside your pocket or on a keychain. Fun stuff. But wouldn’t it be even cooler if that little pet actually, you know, did something... | Read more »
SwitchArcade Round-Up: Reviews Featuring...
Hello gentle readers, and welcome to the SwitchArcade Round-Up for May 25th, 2023. We start things off today with a review of Convergence from our pal Mikhail. After that, it’s new release time. There are somewhere near forty new games hitting the... | Read more »
‘Real Bout Fatal Fury 2’ Review – A Furi...
When it comes to fighting games, the NEOGEO had more than its fair share. Its top three franchises in terms of name recognition were almost certainly The King of Fighters, Samurai Shodown, and Fatal Fury. | Read more »

Price Scanner via MacPrices.net

Apple set to show off new features of iOS 17...
Apple is set to unveil iOS 17 during the WWDC 2023 Keynote speech on June 5th. Here are, briefly, some of the expected features of the new release: When the iPhone is locked and in landscape mode,... Read more
Memorial Day Weekend Sale: 13-inch Apple M2 M...
B&H Photo has 13″ MacBook Pros with Apple M2 processors in stock and on sale today for $200 off MSRP as part of their Memorial Day Weekend sale. Their prices are among the lowest currently... Read more
Memorial Day Weekend Sale: $30 off Apple AirP...
Amazon has Apple AirPods on sale for up to 38% off MSRP as part of their Memorial Day Weekend sale. Shipping is free: – AirPods 2nd-generation: $99 $30 (38%) off – AirPods Pro 2nd-generation: $199.99... Read more
Memorial Day Weekend Sale: Apple Watch Ultra...
Amazon has Apple Watch Ultra models (Alpine Loop, Trail Loop, and Ocean Bands) on sale for up to $97 off MSRP, each including free shipping, as part of their Memorial Day Weekend sale. Their prices... Read more
Memorial Day Weekend Sale: $100 off Apple iPa...
Amazon has 10.9″ M1 WiFi iPad Airs on sale for $100 off Apple’s MSRP, with prices starting at $499, as part of their Memorial Day Weekend sale. Their prices are the lowest available among the Apple... Read more
Memorial Day Weekend Sale: $100 off Apple iPa...
Amazon is offering Apple’s 8.3″ iPad minis for $100 off MSRP, including free shipping, as part of their Memorial Day Weekend sale. Prices start at $399. Amazon’s prices are the lowest currently... Read more
The cheapest M2 Pro-powered Mac is available...
Apple is now offering M2 Pro-powered Mac minis in their Certified Refurbished section starting at $1099 — $200 off MSRP. Each mini comes with Apple’s one-year warranty, and shipping is free. The... Read more
Memorial Day Weekend Sale: Apple AirPods Max...
Amazon has Apple AirPods Max headphones in stock and on sale for $100 off MSRP as part of their Memorial Day Weekend sale. Price is valid for all colors at the time of this post. Shipping is free: –... Read more
Record low price: Open-box 16-inch Apple M1 M...
QuickShip Electronics has open-box return 16″ M1 Max MacBook Pros in stock and on sale for $1200 off original MSRP on their eBay store today. According to QuickShip, “The item in this listing is an... Read more
Open-box 16-inch M2 Pro MacBook Pro available...
QuickShip Electronics has open-box return 16″ M2 Pro MacBook Pros in stock and on sale for $600 off MSRP on their eBay store today, only $1899. According to QuickShip, “The item in this listing is an... Read more

Jobs Board

*Apple* iOS CNO Developer (Onsite) - Raytheo...
…Pacific Boulevard Building CC4, Sterling, VA, 20166-6916 USA Position Role Type: Onsite Apple iOS CNO Developer Cyber Offense and Defense Experts (CODEX) is in need Read more
*Apple* Technician - CompuCom (United States...
…assistance. Join **e** **X** **cell** **!** Our client is currently seeking an ** Apple Technician** to join their team onsite in Bloomfield, CT. This is a Read more
*APPLE* Intermediate Administrative Assistan...
…administrative services related to planning, hosting, and supporting the national APPLE Training Institutes - the leading national substance misuse prevention and Read more
Cashier - *Apple* Blossom Mall - JCPenney (...
Cashier - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Blossom Mall Read more
Operations Associate - *Apple* Blossom Mall...
Operations Associate - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.