November 29, 2004, 11:02 AM — For the past few weeks, I've been locked in various dark rooms (OK, it's not as bad as that) performing benchmark tests on a number of wireless LAN systems. Some of the products tested were residential in nature (router/access point combinations), and some were aimed more at the enterprise (standalone fat APs and systems based on wireless switches). I was (and, for that matter, remain more than ever) interested in various aspects of performance, including throughput, time-bounded (or isochronous) behavior critical to real-time services like voice and streaming video, load balancing and optimization, and ease of use. While I'm not going to cover the specific results here (they'll appear in various magazines and other articles), I would like to present you with a little background on some of the core issues, subtle and otherwise, surrounding the benchmarking of WLANs in the event you ever need (or just want) to spend hours or even days on this often-revealing task.
First of all, benchmarking has been around for a long time. The core idea is to present two different configurations of equipment that perform the same function with an identical set of conditions and then see how each does. To use a simple example, consider drag racing (the legal kind, mind you). Take two cars, have their respective drivers hit the gas at the same time and the first one to reach the finish wins. But note the variables - the driver, for example; some drivers may have slower reaction times than others, and thus the potentially superior performance of a given car is masked by a variable that, when and if addressed, could yield a better result on the next run. Thus, the core principle behind benchmarking is to eliminate all variables wherever and whenever possible.
And therein lays the biggest problem for anyone benchmarking WLANs - the rather large number of variables. Here are the important ones you need to be aware of: