What's the fastest browser? Maybe you're measuring wrong

Why the usual browser benchmarks are useless

stopwatchflickr/wwarby

Most of today's browser speed tests focus on aspects such as JavaScript speed, HTML5 performance, or web page loading times. Supposedly, these tests are here to tell us something about a browser and help us either be proud of our choice or convince us to switch. In this article, I'll question typical browser benchmarks, such as SunSpider or V8, and show you why they're no more than meaningless values that are likely affected by an ocean full of non-controllable variables. Second, we'll talk about the memory footprint of web browsers and why this often-overlooked metric matters more than you think.

Why browser benchmarks have nothing to do with real life

Let's face reality: While browser speeds are still a theoretically valid criteria in testing browsers, the latest browser generations have improved on web performance so drastically that some of these benchmarks hardly translate to real-world scenarios anymore. In fact, I'd argue that even IT pros don't realize how scores on these speed tests translate to effects on their day-to-day browsing.

sunspider_benchmark_600x450.jpg
Do these fancy numbers really mean anything?

Among all those very scientific benchmarks we forget one factor: the user. Our human eye is incapable of distinguishing the 221 ms load time of a JavaScript applet in IE9 from the 220 ms load time in Firefox 17. Do we even notice if Chrome opens a website in 778 ms versus IE's 953 ms? Humans think in seconds, not milliseconds. Do we really notice that our Facebook timeline appears on screen a fraction of a second sooner in a particular browser? Would anyone ever care? Of course, we geeks love our milliseconds, but we may get lost in perfection and this obsession has infected the entire industry: today, literally all browser makers push out raw numbers in order keep out-marketing each other.

And all users fall for it: from tech journalists to IT pros to the beginner. Again, scripting and rendering speeds may still be a valid criteria to web developers or in certain scenarios (I'm thinking browser-based automation), but it's starting to reach a point where the differences can't be made out by the user. Let's take things a bit further: What really determines how fast (or slow) a website is displayed on your screen is much more than the browser. In fact, the browser is just one link in a chain of many technologies, applications, and devices that determine how fast ITworld or Facebook appears on your screen. Among those are:

  • Your GPU and graphics driver. Web browsers use GPU acceleration to display content on screen, and the addressed GPUs vary widely in speed -- you can't compare the rendering speed of a three-year-old onboard GPU with a GeForce GTX 590.
  • Current CPU load.
  • AV tools and firewalls.
  • Number of computers sharing the same bandwidth at the same time.
  • Network stack and filtering tools.
  • 3rd party tools accessing the web and transferring data, or background software updates/checks.
  • The connection from your PC to your router to your modem to your ISP switch.
  • Web Server speed, traffic, and nodes.
  • 3rd party content on websites and unnecessary/badly written scripts.
  • I'm sure I forgot half a dozen other things that lurk between the web server and your eyes!

A website's loading time is determined by all of these factors. If one or several links in the chain fail to deliver in time, there's no way to determine the speed of a browser. Therefore, judging a browser by its scripting performance or overall website display speed is -- in my opinion -- an extremely theoretical undertaking with little to no effect on any real-world scenario. And even if browser speed made a visible difference, you'd have to set up a local web server on your own PC and measure browsing speeds for local websites in order to avoid most (not all!) of the factors described above.

Then there's the factor of browser behavior: how do power users actually use a browser? Well, maybe we browse to a website and use a middle- or Ctrl-click to open one or many links in new tabs while we keep reading the current website. In the background, another website is loading, and will likely be ready and waiting for us once we're done reading the active page. That's just one of many different user routines. We're not all robots who type in the URL of a website and wait for it to fully load until we're able to read it. Of course, I don't want to make superficial statements about everyone's browsing habits, but you need to take a good look on how you actually surf the web before choosing your browser based on precise speeds.

Let's put the issue of classic speed tests aside. What (most) of the popular web browser comparisons neglect is that today's browser generation has become more resource intensive than ever. Operating systems like Windows 8 have become increasingly “slimmer” in order to fit on lower end or low-powered mobile devices (tablets, first-generation ultrabooks) or thin clients. One the other side, the browser, which is arguably the most important application on any PC, has become more bloated. And while a big memory footprint on high-end systems may only result in some annoying lag or crashes, it affects the performance of the aforementioned devices, including battery life and overall responsiveness, drastically. And that has a visible effect on your web browsing experience.

1 2 3 Page
From CIO: 8 Free Online Courses to Grow Your Tech Skills
Join the discussion
Be the first to comment on this article. Our Commenting Policies