TweetFollow Us on Twitter

Office 2004 Benchmarks on Intel-based Macs

Volume Number: 22 (2006)
Issue Number: 5
Column Tag: MacTech Labs

Office 2004 Benchmarks on Intel-based Macs

How well does Office 2004 run on Rosetta technology?

by the MacTech Editorial Staff

The Big Question

If you are a Microsoft Office user on the Mac, there's likely a question on your mind if you are considering purchasing a new Mac based on the Intel processor. You see, currently, Microsoft Office is not "Universal." In other words, it runs on top of Rosetta (Apple's technology to dynamically translate PowerPC-based applications to work on Intel-based Macs). Now, Microsoft has announced that the next version of Office will be Universal, but no one expects this imminently. (It's a big job, Apple released their Intel-based machines earlier than expected, and Office isn't borne from Xcode.)

The big question is this: Does Microsoft Office 2004 run well enough on the new Intel-based Macs? Or should you delay your purchase of these machines?

The Test Bench

We chose three machines to compare. Our baseline machine is a PowerBook G4 15-inch, running a 1.5 GHz PowerPC G4, with 2 GB RAM, and an 80 GB/5400 rpm hard disk.

We compared a MacBook Pro 15-inch, running a 1.83 GHz Intel Core Duo, with 2 GB RAM, and a 120GB/5400 rpm hard disk.

And finally, we compared an iMac 20-inch, running a 2.0 GHz Intel Core Duo, 1.5 GB RAM, and a 250GB 7200-rpm Serial ATA hard disk.

For those interested in the benchmarking methodologies, see the more detailed testing information in Appendix A. For the detailed results of the tests used for the analysis, measured in seconds, see Appendix B. Both appendices are available on the MacTech web site.

Overview

We won't keep you in suspense. In general, Office 2004 under Rosetta works "well enough" to "very well," and in some cases, it's even faster than on the PowerPC machine.

To determine this, MacTech ran over a thousand tests across three models of Macs, and the four major Office applications: Word, Excel, PowerPoint and Entourage. And, since graphics code is shared between Office applications, we ran a suite of graphics tests as well. These are each covered in more detail below.

In one of the most critical set of tests, we specifically looked at whether the user could type or interact faster than Office could keep up, and even in the slowest of scenarios, we never found the user waiting for typing, or other interactions like selecting menus. Even when typing at over 100 wpm, Word was able to stay ahead of the user.

Of the four applications, PowerPoint, is the one that struggled the most. It appears this is due to Office graphics engine shared by all of the Office applications.

At the other end of the spectrum, Entourage was not only on par, it was faster in many cases than our PowerPC baseline. In fact, with the exception of launching the application, Entourage was faster across the board on the Intel iMac, while the MacBook Pro was about on par with the PowerBook G4 (slightly faster in some cases, slightly slower in others).

What was interesting is that even on those tests that were slow, they were still acceptable. For example, the test with the worst performance was inserting a large JPG (over 10mb) into a document was considerably slower. In most cases, a 10 megabyte JPEG would be larger than an 8.5x11 page. How often do users insert large JPEGs in a single sitting? And, when they do, does it matter when that something takes 10 seconds more? Not in our opinion ... and we use Word a lot at MacTech.

It appeared to us that the more modern the application, the better it did under Rosetta. Furthermore, the more that something used the underlying Mac OS, the better it did as well. For example, Entourage did very well with networking related items. Raw imports and opening did significantly better especially when a faster disk was involved.

And, the more times you did a function, the better it performed especially on the second iteration of a command. While it's difficult to confirm, this is due to a combination of code working smarter with caches, both in Rosetta as well as within the Office code base. An obvious benefit for those tasks that are most sensitive to time: the repetitive ones.

The Test Suite and Results

The tests used were selected specifically to give a real-world view of what Microsoft Office 2004 is like to run. We eliminated those tests that we ran that were so short a time frame (e.g., fast) that we could not create statistically significant results, or that had imperceivable differences.

We did one test suite for each of the four major applications in Office, as well as a series of tests focused on the graphics libraries. As you may or may not know, Office 2004 uses a common set of graphics libraries across the applications in Office. With that in mind, we considered a set of tests that would test these technologies, and they are a good representation across all the applications in Office. For example, when you import a graphic, it should be the same across the Office suite.

To give users a good idea of what it's like to use each of these apps, we came up with a list of tests that represented what we felt were the most relevant to regular use. For example, we ran tests on application launches, but did not focus on them since it's something that people tend to do only a couple of times a day. But, we did include the most repetitive of tasks within an application, as those affect productivity most.

The one thing that we did see across the board is that the Intel iMac is consistently faster than the MacBook Pro. Since the iMac had slightly less RAM, and the processor and front side bus speeds are the same, we looked further inside. The iMac has some pretty serious sub-systems design to make it a screaming machine (see <http://developer.apple.com/documentation/HardwareDrivers/Conceptual/iMac_06Jan/>. When you take into account that an iMac has SATA (instead of ATA), a faster hard drive (a 7200 vs. the MacBook Pro's 5400 rpm), and a variety of optimized sub-systems that would be more difficult to implement on a laptop, it's easy to see why the iMac is faster.

It's important to realize that many of the actions that users do when using these applications are so fast already, that even a degradation of 50% may not even be noticeable for most tasks. And in the non-repetitive tasks, they are nearly irrelevant. For example, if when launching an application, you have to wait several seconds, many users will notice, but it won't matter to their overall productivity.

Other areas, like repetitive tasks or editing actions, are far more important, and speed makes a great deal more difference in not only perception, but in productivity.

An overview of tests that best represents productivity is displayed in Figure 1. These metrics include a variety of tests normalized to give you an overview. Smaller bars are faster (better). Information on what is included in each of these overview metrics, as well as additional testing information and graphics are shown later for each application.




Figure 1: Office 2004, Average User Tests


Word

Of all the applications, Word is probably the most widely used by people. The most relevant tests to most Word users are those that are repeated throughout the use of Word. For productivity tests, we selected a variety of scrolling tests, saving, word count, find & replace, opening files, pasting and printing.

The end result is that, for productivity tasks, Word on MacBook Pro performed at 82% of the speed of a PowerBook G4, and on an Intel iMac at 92%. In a real life scenario, these numbers put Word at working pretty well. There's not much a perceptual difference for most tasks.

Our top level grouping included a application launching, variety of scrolling tests, saving, word counts, find & replace, opening documents, pasting and printing. The results are in Figure 2.




Figure 2: Word Benchmark Results


But, of course, the most important of the tests - typing - were so fast that we couldn't measure them. No matter what the computer model, Word was ahead of the typist (even at 100 wpm).

The areas that Word performed "well enough" in are the initial launch of the application launching and printing. Fortunately, these are things that most users do relatively infrequently and don't materially affect productivity.

The balance of the tests ran on par, or in some cases faster on the Intel iMac. As is consistent throughout, the Intel iMac was faster than the MacBook Pro, and in some cases (most notably saving and pasting), the iMac was faster than the PowerBook baseline.

Our one and only crash during the entire Office testing set was during a find and replace in Word. A quick re-launch, and everything proceeded as expected.

The end result is that while there are a couple of areas that a user may notice it's slower, Word works well on a MacBook Pro; and, on the Intel iMac, it works very well, under Rosetta.


Excel

For Excel, we wanted to focus again on the most repetitive tasks. For productivity tests, we selected fill range, scrolling vertically, zooming out, arranging windows, subtotals, auto-formatting, and editing in-cell. On average for the productivity tests, Excel performed at 83% of baseline for the MacBook Pro, and 91% for the Intel iMac.

Most users use relatively small spreadsheets, but it's important to take a look at what a more experienced spreadsheeter may experience. As a general rule, editing and calculations were very fast and in some cases, faster than on the baseline PowerPC.

For our top level testing, we looked at the common commands including fill range, vertical scroll, zoom out, arranging windows, subtotals, auto-format, and editing in-cell. The results are displayed in Figure 3.




Figure 3: Excel Benchmark Results


Scrolling did leave a bit to be desired, but it definitely worked well enough, even on the slower MacBook Pro. In our tests, we used a very large Excel document, and even so, the test ranges were 6.18 to 10.15 seconds. Certainly enough to perceive, but given the very large size of the test document, under normal circumstances, it's not enough to affect productivity.

The end result is that for Excel on a MacBook Pro, it works very well, and on the Intel iMac, even better. Users may notice slower scrolling, but that's about it as far as the most common tasks.


PowerPoint

For PowerPoint, we chose a variety of tests to represent productivity including opening, saving, scrolling slides in the sorter view, adding slides, viewing slide shows, slide transitions, text animations, printing, inserting images from the ClipArt Gallery, changing color schemes and applying templates. As we already mentioned, PowerPoint was the slowest of the Office suite, but in our most indicative tests for productivity, it did perform at 77% of the speed for the MacBook Pro, and nearly 90% for the Intel iMac.

In our top level testing, we took a look at launching, opening files, saving, scrolling slides in sorter view, adding slides, viewing slide transitions, viewing text animations, printing, changing color schemes and applying templates. The results are shown in Figure 4.




Figure 4: PowerPoint Benchmark Results


The areas PowerPoint was weakest were with complex files, printing and applying templates. That said, PowerPoint did pretty well with view slides and transitions, animations, and scrolling in the sorter view.

The bottom line is that even if you use PowerPoint regularly, it definitely works well enough for most people, and in the most important areas ... the presentation itself ... it works very well. If you are a heavy PowerPoint user, you will notice sluggishness, and you should take a closer look.


Entourage

For Entourage, the productivity tests included IMAP account sync, empty deleted items, opening messages, pop message download, opening folders, sending messages via SMTP, sorting and grouping 2000 messages, and searching the address book, mail and tasks. For the productivity tests, Entourage shined with the MacBook Pro performing at 92% of the baseline speed, and the Intel iMac performing 14% faster than the baseline PowerPC.

In our top level testing, we took a look at launch time, empty deleted items, POP message download, sending SMTP messages, sorting and grouping thousands of messages, and searching the address book, mail and tasks. Results are shown in Figure 5.




Figure 5: Entourage Benchmark Results


Like Word, the most important of the tests, typing, were so fast that we couldn't measure them regardless of which test machine.

Boot time was clearly slower - about double in fact. But in a game of seconds, and for something that you generally only do a couple of times a day, it's not particularly relevant.

Entourage is clearly the best performer in the suite, presumably because Entourage is the most modern code base of the Office 2004 suite, and because it relies the most on Mac OS X technologies that have already been made Universal.

The end result is that, across the board, Entourage under Rosetta performed terrifically. The MacBook Pro usually performed as good or better than the PowerBook. The Intel iMac was faster in almost every test.


Microsoft Office Graphics

The weakest part of Office under Rosetta is the graphics code shared across Microsoft Office. PowerPoint shows this most given its nature.

In reality, however, this affects the overall Office experience relatively little. Furthermore, for the graphics libraries, our tests were designed to point out weaknesses and give us measurable results. But let's be real: how often do you import a 10-megabyte JPEG? Most JPEG's that size would be much larger than a full page.

That said, we focused on the tests on inserting large graphics, opening, saving, printing, working with the WordArt features, and the ClipArt library.

The end result is that inserting JPEG's are slow, and we saw the impact across all the Office applications. Fortunately, most users don't tend to do this repetitively. Other formats like inserting EPS and PSD files did much better. In other words, you'll have no issues with your logos and template graphics.

Opening graphics files was ok, but fortunately, saving files was significantly faster (a good thing since users typically save much more often than open files). The ClipArt interfaces work very well including the importing and searching.




Figure 6: Office Graphics Benchmark Results


Conclusion

As we stated at the beginning of this article, in general, Office 2004 under Rosetta works "well enough" to "very well," and in some cases, it's even faster than on the PowerPC baseline machine.

Given the amount we were pushing these apps, we were thoroughly pleased with the stability of the entire Office suite of applications. Considering how complex Rosetta is, and how big a code base Office is, it's entirely remarkable how stable we found it. Furthermore, we expect to see Apple further optimize Rosetta, particularly in launch times.

While Microsoft has already announced that it will be making the next version of Office a "Universal" application, Mac users wishing to dive into Intel-based Macs now can rest assured in knowing they can move forward and be patient for the Universal version.





Appendix A: Benchmarking Methodology

Introduction

The purpose of this appendix is to outline the basic parameters for how MacTech Magazine performed benchmarking tests on Microsoft Office 2004 for Intel iMacs and the MacBook Pro compared to a PowerBook G4.

Benchmarking Methodology


Consistency

Since the tests involve both multiple machines and multiple pieces of software, the focus was on creating as much consistency across the tests as possible. MacTech accomplished this in several ways.

First, all timing of tests were performed by a single MacTech staff member so as to eliminate any of the natural inconsistencies that often occur across individuals.

All of the tests were performed on the same version of the Mac operating system across the different hardware. At the time of the tests, this was Mac OS X 10.4.5 and included the most up-to-date versions of Apple patches as prescribed through "Software Update" in Mac OS X.

All of the tests were done on "virgin" systems, i.e., freshly wiped hard disks, with fresh Mac OS X and Office installations, with no third party software installed beyond the standard Apple and Microsoft Office installations.

All of the tests were performed with the most up to date set of patches for Microsoft Office as prescribed by Microsoft's Auto-updater. At the time of the test, this included the Microsoft Office 2004 for Mac 11.2.3 Update.


Avoiding Interactions

While the tests covered the bulk of the Microsoft Office suite, all tests (where appropriate) were performed with only that single application within Microsoft Office open. In other words, to the extent possible, no other applications will be running. (Obviously excluding background and OS tasks that are part of a standard install of Mac OS X or Office.)

To avoid issues with a noisy network, the test machines were installed on what was considered a "quiet" network with minimal traffic. MacTech monitored the use of the network to make sure that the machine does have network access, but is not impacted by the network.


Measurements, Testing and Outliers

For timed tests with results under 15 seconds, tests were measured to within 1/10th of a second. For those over 15 seconds, tests were measured to within a second.

Most tests were performed at least three times per test and per machine, and up to 5 times depending on the test. On average, tests were conducted three to four times per machine per test case. Outliers indicating a testing anomaly were retested as appropriate.

In most cases, tester used successive tests, not "Adam" or "first" tests to better emulate the daily use of Office products.

Those tests that could be impacted by the size of the window, were tested with the same window size under all scenarios.

Some tests were eliminated because the machines simply performed too fast to get an accurate measurement. For example, typing always displayed faster on the machine than the tester could type (even at 100 wpm).





Appendix B: Testing Results

Introduction

The below chart contains the raw data for the tests selected to judge Office 2004 applications. All results are in seconds, and represent the best time for each test on each machine.

                                   PowerBook G4   MacBook Pro   Intel iMac
Word:
Scroll Real World Doc                      8.12          9.47         8.34
Scroll Doc with Tables                     3.47          4.13         3.91
Scroll Doc with Graphics                   5.50          7.22         6.72
Save As Word Doc                           1.19          1.22         1.12
Word Count                                 1.69          2.22         1.84
Find & Replace                             0.85          0.93         0.88
Open Word Doc                              1.28          1.59         1.34
Paste                                      1.84          1.94         1.75
Print a Word Doc                           7.24         11.00        10.10

Excel:
Fill Range                                 1.41          1.66         1.68
Scroll Vertical                            6.18         10.15         8.97
Zoom Out                                   1.25          1.56         1.47
Window Arrange                             2.25          2.56         2.15
Subtotals                                  1.44          1.47         1.16
AutoFormat                                 0.94          1.06         1.00
Edit in-cell                               1.44          1.57         1.47

PowerPoint:
Open typical simple file                   1.47          1.78         1.56
Save complex file                          1.40          2.10         1.78
Scroll slide in sorter view                2.72          3.25         3.00
Add slides in sorter view                  2.16          2.56         2.47
View slide show                            1.19          1.12         1.16
View slide transitions                     0.82          0.85         0.88
View text animations                       0.81          0.85         0.66
Print simple file                          1.53          2.25         1.94
Insert from Clip Art Gallery               1.34          1.94         1.50
Change color scheme                        0.88          1.13         0.72
Apply Template                             2.59          4.88         4.38

Entourage:
IMAP Account Sync                          1.84          2.31         1.57
Empty Deleted Items                        5.25          6.50         4.06
Open Messages                              3.03          4.46         3.65
POP Message Download                      63.00         73.00        57.00
Open Folder                                0.69          0.69         0.56
Sending Messages SMTP                      1.35          1.19         1.16
Sorting 2000 messages                      0.81          0.81         0.78
Grouping 2000 messages                     1.78          1.65         1.50
Address Book Search                        1.06          1.06         0.85
Mail Search                                0.56          0.59         0.53
Task Search                                0.68          0.62         0.47

Graphics:
Insert Large JPG                           2.40         12.22        11.12
Insert Large EPS                          25.56         37.81        21.87
Open Document                              1.19          1.94         1.68
Save Document                              2.81          3.59         2.07
Print Document                            55.00        100.00        91.00
WordArt: Insert                            0.84          1.16         1.06
ClipArt: Import                           98.00         89.00        49.00
ClipArt: Search                            0.50          0.53         0.44




 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Avatar Legends: Realms Collide pre-regis...
Despite the staying power that the series has, Avatar: The Last Airbender surprisingly fumbled in the mobile gaming sphere, with Generations shuttering about a mere year after launch. Now, Tilting Point is giving it a go, with the upcoming Avatar... | Read more »
Return to the glory days of tactical RPG...
Back on the old Sega Mega Drive, or Genesis depending on where you are, there was a little series of tactical role-playing games called Shining Force, and I adored them. It started a love for this grid-based genre that persisted through the years... | Read more »
Take on the grandest beasts of all as th...
It has been hyped for a while but now it is finally here, the new Dancing in the Tempest season has arrived in Monster Hunter Now. Kicking off a banner summer for the game, it is time to strap on your boots and face the greatest threat yet, as the... | Read more »
The indie hit Vault of the Void will lan...
In a market where a lot of AAA games are starting to feel like reboots, remakes, or the same thing we’ve seen ten times with a different name slapped on it, indie developers are a boon. Especially ones who build a successful game singlehandedly,... | Read more »
Stumble Guys tries to catch up to the hy...
Do you remember when the Fallout TV series launched on Amazon and everyone nearly lost their minds? Well, it appears that Scopely missed that particular craze, and only now are we going to get a Fallout crossover, and by now, I mean in version 0.... | Read more »
Bid farewell to Penacony as Honkai: Star...
Penacony has been a story of twists, exciting new characters, and strong allies, and soon Honkai: Star Rail will be finishing it with a bang. Version 2.3, fittingly titled Farewell Penacony, will be launching June 19th and will feature updates to... | Read more »
HoYoverse roll out their plans for Anime...
For those who are looking to book a getaway in July, you might give some thought to Los Angeles between the 4th and 7th, which just so happens to coincide with the Anime Expo 2024. Amongst all the storied attendees is HoYoverse, who will be... | Read more »
The first rule of Brok the InvestiGator...
Mobile gamers were recently able to get their hands on BROK the InvestiGator, a point-and-click following the adventures of the titular reptile, a detective who can solve crimes through wit or brawn. If you were one that chose the latter then... | Read more »
Diablo Immortal celebrates second annive...
It has been two years since Diablo Immortal launched and despite some very valid criticism of its business model, it has done pretty well for itself. The Tempest class also gives it a lot of grace. To celebrate this anniversary, the March of the... | Read more »
Pokemon GO pulls on its jersey for a foo...
There have been a lot of jokes about this, some by me, but Pokemon Go has genuinely done a lot of good by getting people out and about.Pokemon GO Fest 2024: Madrid is fast approaching, and Niantic has set up a new area in a bit to get people to... | Read more »

Price Scanner via MacPrices.net

Apple Watch Ultra Watch 2 on sale for $719, s...
Amazon is offering an $80 discount on every Apple Watch Ultra 2 model this week. Their price is now $719. Shipping is free. For the latest prices & deals, keep an eye on our Apple Watch Price... Read more
New sale at Amazon: 16-inch M3 Pro and M3 Max...
Amazon is offering instant discounts on 16″ M3 Pro and 16″ M3 Max MacBook Pros ranging up to $350 off MSRP. Shipping is free. These are the lowest prices currently available for new 16″ Apple MacBook... Read more
Get a 13-inch M2 MacBook Air today at Apple f...
Apple has 13″ M2 MacBook Airs available for only $849 today in their Certified Refurbished store. These are the cheapest M2-powered MacBooks for sale at Apple. Apple’s one-year warranty is included,... Read more
Clearance Mac Studio with M1 Max CPU availabl...
Apple has clearance M1 Max Mac Studios available in their Certified Refurbished store for $270 off original MSRP. Each Mac Studio comes with Apple’s one-year warranty, and shipping is free: – Mac... Read more
Apple has 24-inch M3 iMacs on sale for $200-$...
Apple has a full line of 24-inch M3 iMacs available in their Certified Refurbished store starting at $1099 and ranging up to $260 off original MSRP. Each iMac is in like-new condition and comes with... Read more
24-inch M1 iMacs are available at Apple start...
Apple has clearance M1 iMacs available in their Certified Refurbished store starting at $1049 and ranging up to $300 off original MSRP. Each iMac is in like-new condition and comes with Apple’s... Read more
Back to School savings: Take $50-$100 off new...
Apple will take $50-$100 off new 11″ and 13″ M2 iPad Airs for all teachers, students, and staff of any educational institution with a .edu email address as part of their Apple Education discount,... Read more
Could A Smarter Siri Infused With AI (‘Apple...
FEATURE – The iPhone is already smart, but it’s about to become more intelligent. AI — short for artificial intelligence — is widely expected to be the main topic of discussion at this year’s WWDC (... Read more
Update: For WWDC, Amazon has lowered prices o...
Amazon has every configuration and color of Apple’s M3 MacBook Airs now on sale for $170-$210 off MSRP, starting at only $899 shipped, as Apple holds their annual WWDC conference this week. Their... Read more
Deal Alert! 2nd-generation Apple AirPods on s...
Amazon has 2nd generation Apple AirPods on sale right now for only $79.99 shipped. That’s $50 (38%) off Apple’s MSRP. Their price is the lowest currently available for a new set of AirPods from any... Read more

Jobs Board

Beauty Consultant - *Apple* Blossom Mall -...
Beauty Consultant - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
Senior Software Engineer - *Apple* Fundamen...
…center of Microsoft's efforts to empower our users to do more. The Apple Fundamentals team focused on defining and improving the end-to-end developer experience in Read more
Sublease Associate Optometrist- *Apple* Val...
Sublease Associate Optometrist- Apple Valley, CA- Target Optical Date: Jun 17, 2024 Brand: Target Optical Location: Apple Valley, CA, US, 92307 **Requisition Read more
Rehabilitation Medicine Technician - *Apple*...
Rehabilitation Medicine Technician - Apple Hill (Outpatient Clinic) - Day/Evening Location: York Hospital, York, PA Schedule: Part Time Sign-On Bonus Eligible Read more
Operations Associate - *Apple* Blossom Mall...
Operations Associate - Apple Blossom Mall Location:Winchester, VA, United States (https://jobs.jcp.com/jobs/location/191170/winchester-va-united-states) - Apple Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.