Lab Notes: The Inside Scoop on Browser Speed Testing

Mozilla Firefox web browser logo

If you're a regular PCWorld reader, you may have noticed the Browser Blowout story we posted last week. In it, I looked at various aspects of the major Web browsers, including features, interface, security, and performance.

Browser performance -- more specifically, how long it takes to load a Web page or Web app -- can be a controversial topic. And there's no perfect method for testing browser performance. But I'd like to take a few moments to describe in more detail our approach to browser speed testing, and explain why we did what we did.

Option 1: Testing on "Live" Sites

The simplest, most straightforward way to test browsers is to load up a bunch of Web pages and stopwatch-time how long it takes for each browser to load the pages in question (we use a camcorder to record the page-loading tests, which we review later to determine how long it takes browsers to load the pages). This is also the most "real-world" approach to testing browsers--it lets us simulate what you'll experience when you're browsing the Web with your own PC.

But there are drawbacks to this method. If you test using live Web pages on the Internet, network congestion and server issues can impact the testing results. Also, site contents may change from day to day--and from testing pass to testing pass--so it can be hard to get a consistent read on page load times.

Option 2: Testing on a Closed Network

The second option is to save pages to another PC that has Web server software installed. This method lets us rule out a number of variables such as network congestion, server problems, and changes made to the page. But the results will also be skewed to some extent for normal users, as they won't necessarily be representative of real-world Web browsing.

Option 3: Benchmarking

The other option, of course, is benchmarking. Benchmarking is good for identifying how well browsers handle different Web technologies, and numerous benchmarks exist--most notably, the Sunspider and Peacekeeper benchmarks, to name a couple. But most browser speed benchmarks either aren't real-world tests or only test a specific element of browsing performance (for example, the Sunspider benchmark only tests JavaScript performance).

What We Did

Each of the three options I outline above have their pros and cons, and each test different aspects of a browser's performance, so we did all three. That way, we got a more complete picture of what the browser performance landscape is really like.

In any software speed testing, a lot is going to depend on your PC's hardware.We tested on a fairly recent notebook running Windows 7, but your results will most definitely vary depending on your OS, your computer's hardware, which browser version you're using, and so forth.

Hiccups and Glitches

No major testing project goes off without a hitch, and this one was no exception. The biggest issue I ran into was regarding automatic browser updates. Google Chrome, for instance, installs updates in the background without you realizing. And while I was working on the story, Mozilla pushed out a bug-fix update to Firefox 3.6.

Fortunately none of these updates affected our speed testing, but if a major new version rolled out during our testing, well, you would've heard a very loud scream coming from San Francisco's South of Market neighborhood.

Why We Say Speed Doesn't Matter

In our browser comparison story, I said the following with regard to speed testing:

"Now that speed has essentially become a baseline standard, you should put more weight on the other aspects of a browser: interface, stability, security, ease of use, and add-ons. The good news, of course, is that all of the major browsers are free to download and try. If you don't like one, you can always switch to another until you find the browser that works best for you."

I've tested browser page loading speed on a few occasions now, and while the rankings have changed some as browsers got updated, this bottom line remained true. So if you're running a browser that isn't "fastest," it's probably still "fast enough."

Like this? You might also enjoy...

• Lab Notes: Kingston vs. VelociRaptor Storage Smackdown
• Amazon Offers Cloud Supercomputing Service
• Mozilla Launches Firefox 4 Beta

Follow GeekTech on Twitter or Facebook.

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

• Bookmark this page
• Share this article
  •  
  • facebook
  • slashdot
  • digg
  • Reddit
  • stumbleupon
  • linkedin
  • twitter
• Got more on this story? Email CIO
• Follow CIO on twitter

• Share
  Share this article

  •  google+
  • facebookfacebook
  • slashdotslashdot
  • diggdigg
  • RedditReddit
  • stumbleuponstumbleupon
  • linkedinlinkedin
  • twittertwitter
• print
• email
• Bookmark