Volume Number: 24 (2008)
Issue Number: 03
Column Tag: MacTech Labs

Office 2008 Benchmarks

How well does Office 2008 run compared to Office 2004?

by Neil Ticktin, Editor-in-chief

If you are a Microsoft Office user on the Mac, there's likely a question on your mind about the new version of Microsoft Office 2008 for Mac. As you may remember, MacTech did extensive benchmarking on Microsoft Office 2004 for Mac, running under Rosetta, when the Intel Macs first came out. See http://www.mactech.com/articles/mactech/Vol.22/22.05/Office2004Benchmark/ to read the full article.

But Office 2008 is "Universal," meaning that it's designed to take advantage of the Intel processor, while still being compatible with PowerPC based machines. The big question is therefore, "How fast is the new Office 2008?"

To answer that question, we put Office 2008 through its paces on both Tiger and Leopard (Mac OS X 10.4.11 and Mac OS X 10.5.1 to be specific). With over 2,500 tests, we looked at Word, Excel, PowerPoint and Entourage as well as the graphics library resources that are shared across all of Microsoft Office for Mac.

We won't keep you in suspense. In general, Office 2008 is faster than Office 2004 when run on Intel Macs. On the PowerPC (represented by our PowerBook G4), it certainly runs "well enough" albeit marginally slower.

In the over 2,500 real world tests comparing Office 2004 with 2008, the vast majority were faster, with many features being 2-3x faster in the new version. On average, Office 2008 running on Intel was 28% faster than 2004 for those items we tested.

For PowerPC users, the issue of whether to upgrade should come down to whether you want the new features, or new file format supported integrated into the application.

Clearly, however, for Intel users, there are not only the benefits of new features and a more straightforward user interface, but many speed improvements as well.

Figure 1: Office 2008 vs. Office 2004? (shorter is faster)

In Figure 1, you can see how Office 2008 ran compared to Office 2004. The PowerPC machines (blue and red) and Intel machines (green and purple) represent show the two different setups for Office 2004 and 2008, respectively.

To determine this, MacTech ran tests across five models of Macs: PowerBook G4, MacBook, MacBook Pro, Intel iMac, and Mac Pro. Real world tests were run to make these determinations, such as:

• successive launch

• scrolling

• find and replace

• opening files

• pasting

• printing

• zooming

• subtotals

• auto formatting

• applying templates

• IMAP account sync

• Searching

• Inserting and importing graphics

Furthermore, for most users when running Office 2008 on Intel, it's usually slightly faster on Leopard than it is on Tiger. The only exceptions were PowerPoint and Entourage which ran just a bit slower. See Figure 2.

We ran tests in each of the four main Office 2008 apps, plus the graphics engine. What we found for each of them was as follows.

For Intel Mac users, on average:

• Word 2008 ran 37% faster than Word 2004

• Excel 2008 ran 22% faster than Excel 2004

• PowerPoint 2008 ran 40% faster than PowerPoint 2004

• Entourage 2008 ran 28% faster than Entourage 2004

• Graphics (common to all the applications) in Office 2008 ran 60% faster than Office 2004

To make this even more clear, when something says "30% faster," that means that if it took 10 seconds on 2004, it took 7 seconds on 2008. This tends to understate "faster" when compared to perception.

While we already had thousands of tests to perform, we thought it important to include a PowerPC machine in the mix. We chose the PowerBook G4 as a reasonable representation of PowerPC. Obviously, our tests needed to focus on machines shipping in the last couple of years, and that's Intel based Macs. But, if you are looking to deploy 2008 on PowerPC based Macs, either because you want consistency, the new file formats, or the new features, 2008's speed on PowerPC is very usable, and the slower stopwatch tests shouldn't stop you.

For PowerPC Mac users, represented by our PowerBook G4 testing, on average:

• Word 2008 ran 35% slower than Word 2004

• Excel 2008 ran 65% slower than Excel 2004

• PowerPoint 2008 ran 9% slower than PowerPoint 2004

• Entourage 2008 ran 22% slower than Entourage 2004

• Graphics in Office 2008 ran 30% faster than Office 2004

Figure 2: Intel Tiger vs. Intel Leopard? (shorter is faster)

When we were choosing computer models, we set out to choose not the fastest, latest models, but ones that would be a good representation of what most people may have. Certainly, the faster models of these computers will perform even better.

Specifically, these are the machines that we used:

• PowerBook G4: 2GB RAM, 1.5 GHz G4 processor

• MacBook: 2GB RAM, 1.83 GHz Core Duo processor

• MacBook Pro: 4GB RAM, 2.16GHz Core 2 Duo processor

• iMac: 2GB RAM, 1.83 GHz Core 2 Duo processor

• Mac Pro: 4GB RAM, Quad Core (two 2.66 GHz Dual-Core Intel Xeon processors)

The test bench included configurations of both Mac OS X 10.4.11 and Mac OS X 10.5.1, with Office 2004 and 2008 installed. All installations were completely clean installations of both Mac OS X and Microsoft Office.

The number of tests that we did across the number of Mac models across two Mac OS X versions would make looking at charts and results too complex for any normal human. With that in mind, let's focus on Office 2008 vs. Office 2004 on Leopard on Intel machines as this is the most common scenario. (Don't worry, the complete results for individual machines and tests are available in the results tables if you want to see them.)

Here is what we found for each application (Word, Excel, Entourage and PowerPoint). Note: Both Office 2004 and Office 2008 use a graphics engine that is shared by all the Office applications. With that in mind, the final set of tests focuses on that graphics engine.

Clearly, Word is the most used application in the Office Suite. As a result, it makes sense for us to run the largest test suite in Word. The test suite included:

• Successive launch (i.e., a launch after the application had been launched already once)

• Scrolling from top to bottom in a document

• Save as (not "save") of a real world document that included text, pictures, and more.

• Global find and replace

• Opening a larger document

• Pasting

• Printing a document with complexity (e.g., text, pictures, etc...)

As you can see from Figure 3, Word 2008 is a whole lot faster than Word 2004 with the exception of launching (slightly slower) and printing (which relies more on the OS).

The documents used for the test were real world sized, but on the larger side so that we could get quality test results.

Figure 3: Word 2004 vs. Word 2008? (shorter is faster)

For Excel, we focused on things that the application spent time "thinking about." As a result, the test suite included:

• Successive launch

• Zoom out

• Subtotals (and the formatting that goes along with that)

• Auto format

We also looked at scrolling, but it turns out that scrolling was so fast that Microsoft's user testing showed it needed to be slowed down. As a result, the test wasn't appropriate.

In all cases, Excel 2008 was faster than Excel 2004.

Figure 4: Excel 2004 vs. Excel 2008? (shorter is faster)

For Entourage, we focused on the items that were measurable. As a result, the test suite included:

• Successive launch

• IMAP account sync

• Purge deleted items

• Address book search

We had also looked at POP message download and sending messages via SMTP, but found that these were not good tests of the application ... that they had more to do with the network and the server. Additionally, we looked at mail and task search, but found the tests ran too fast to be measured in Entourage 2008.

Figure 5: Entourage 2004 vs. Entourage 2008? (shorter is faster)

So much of one's time in PowerPoint is spent reviewing slides and transitions, as well as typing. These are not good tests, however, as they test either the user, or are fixed length by design (e.g., a transition speed doesn't typically vary much). As a result, the test suite included:

• Successive launch

• Opening a complex PowerPoint file

• Saving a normal size file

• Scrolling through a large presentation in "sorter view"

• Applying a template throughout a large presentation

In all cases except one, PowerPoint was faster. Sometimes much faster. Launching was slightly slower.

Figure 6: PowerPoint 2004 vs. PowerPoint 2008? (shorter is faster)

As we mentioned earlier, Office 2008 and 2004 both rely on a common graphics library that is used across the Office suite. With that in mind, we tested some of the graphics functions to see how they fared, including:

• Inserting a large JPG

• Inserting WordArt

• Importing a number of pictures into ClipArt

In all of these cases, Office 2008 graphics were substantially faster.

Figure 7: Office Graphics 2004 vs. Office Graphics 2008? (shorter is faster)

MacTech's tests are primarily done using a stopwatch, and as such, they are timing things that are done solely by the computer. This is a good judge of the difference between Office 2004 running under Rosetta vs. Office 2008 running as a Universal application, native on Intel.

But, what Office 2008 does more than just be Universal is to bring some concrete advancements in creating work, as well as the level of quality that a "normal" person can accomplish. This is done through Document Elements, SmartArt, Ledger Sheets, Charts, and other devices. Clearly these are strongest in Word and Excel.

What these features do for the human being is make it easier to get there -- we'll call this "user speed." For example, charts can be created in one click, without having to go through dialog box after dialog box in a wizard. Or, building a formula in Excel. It's far easier than it has been in a long time.

From a quality of work product, Word and Excel in particular have new features that really raise the bar. For example, the templates provided vastly enhance the quality of a "newsletter" or a "flyer". And, SmartArt: wow. I can't even imagine any student or business user even thinking of doing this in any Office product before.

All of these items give users a great deal more "user speed" ... and those are not the type of things that can be measured with a stopwatch. But, in many cases, you will see significant "user speed" improvements, regardless.

As we stated at the beginning of this article, on Intel machines, Office 2008 runs faster for most things than 2004 does. For PowerPC based machines, 2008 works "well enough"; in fact, for many things, we don't think you'll notice much of a speed difference from 2004 on PowerPC.

For those that work in areas affected by the "user speed" enhancements, and we have to believe that's the majority of users, you'll see substantive improvements. Add to it the new features, and you have an application suite that allows you produce higher quality work, in far less time, either because it's Universal and faster, or because of what the new user interface and templates bring you.

In the end, if you want the new features, or the new file formats, then Office 2008 is the way to go.

Neil is the Editor-in-Chief and Publisher of MacTech Magazine. Neil has been in the Mac industry since 1985, has developed software, written documentation, been heading up the magazine since 1992. When Neil writes a review, he likes to put solutions into a real-life scenario and then write about that experience from the user point of view. That said, Neil has a reputation around the office for pushing software to its limits and crashing software/finding bugs. Drop him a line at publisher@mactech.com