By now you've seen all the ads pitching wireless companies' new 4G mobile broadband services and devices. But beyond all the buzzwords and hype, which companies can reliably provide next-generation speed?
We decided to find out by testing each of the four major national carriers--AT&T, Sprint, T-Mobile, and Verizon--in 260 locations spread among 13 U.S. cities. We found some clear winners and losers, and some good news about wireless service in the United States as a whole. Here are our conclusions.
Wireless data speeds have soared: Since this time last year, the major wireless carriers, as a group, have increased their average download speeds for laptop-modem users by more than threefold, an apparent result of their urgent transition from 3G to 4G network technology. (We measured the best service we could get--3G or 4G--in each testing location.) Over laptop modems, the Big Four carriers now have a collective average download speed of roughly 3.5 megabits per second in our 13 testing cities, versus a nearly 1-mbps average download speed in those cities at the beginning of 2010, a remarkable improvement.
In our previous wireless-network performance studies, we measured the "reliability" of the data service, expressed as the percentage of tests in which we could obtain a good connection. But our test results show that network service has improved to the point where it's rare to find an unusable signal or no signal at all. So we have retired our reliability measurement--another testament to the dramatic improvements of the past year.
Verizon's 4G LTE is for real: Verizon's 4G LTE service, which is now in 38 U.S. markets, was widely available in 12 of our 13 testing cities. (We didn't go out of our way to test in areas served by Verizon's LTE network; we haven't changed our list of testing cities in the three years we've done these tests.) Our laptop-modem tests on Verizon clocked speeds that were far faster than those on competing 4G networks in the same tests (twice as fast as the second-fastest service, in fact). Verizon's network had an average download speed of roughly 6.5 mbps and an average upload speed of 5.0 mbps.
One important caveat: A relatively small number of Verizon customers currently use this new network. During our testing period, Verizon offered only two laptop-modem models that worked on the network, and none of the company's smartphones could take advantage of the new 4G speeds. The performance of Verizon's network could degrade as more people--and devices--connect to it.
And there's a downside to Verizon's 4G success. While the new 4G LTE network is lightning-fast, our smartphone-based tests suggest that the 3G CDMA network that most Verizon smartphone customers use today may actually be getting slower. The connection speeds we measured on our Verizon (3G CDMA) testing smartphone (a Motorola Droid 2) stayed the same or decreased in 10 of our testing cities since last year. And at the moment, those CDMA phones are all that's available to Verizon Wireless customers.
T-Mobile smartphones are fastest: Verizon may have the fastest network for laptops, but in our tests T-Mobile had the speediest results for smartphones. The T-Mobile HTC G2 we used for testing produced a 13-city average download speed of almost 2.3 mbps; that's about 52% faster than the second-fastest phone, Sprint's HTC EVO 4G, which had an average download speed of 1.5 mbps.
T-Mobile also impressed in our laptop-modem tests. Although only half as fast as Verizon's, T-Mobile's download speeds averaged almost 3 mbps in our tests--more than a threefold increase from the carrier's nearly 0.9-mbps average download speed in our January 2010 survey. With these laptop- and smartphone-based results, T-Mobile is proving to be a worthy challenger to its much-larger competitors.
AT&T continues to grow, but perhaps not fast enough: AT&T, the big winner in our January 2010 survey, has continued to ramp up throughput speeds at about the same pace, judging from this year's survey results. Its average download speeds in our laptop-modem tests grew 76% to a roughly 2.5 mbps average this year. But each of its competitors showed bigger jumps in download speeds over the past year, resulting in a third-place finish for AT&T in this year's speed results.
And AT&T's speed gains didn't translate well to our smartphone-based tests: The average download speeds we measured on our Apple iPhone 4 (1.4 mbps) increased only 15% over the speeds we measured on the same device in early 2010. However, AT&T intends to launch its own 4G LTE network later this year, a move that might tip the balance of the 4G speed race in its favor once again.
Sprint needs more 4G: In the cities where Sprint offers its 4G WiMax service, customers saw large speed increases over the past year. Sprint's average download speeds grew 170% to 2.1 mbps in our tests this year; the result would have been even better had the WiMax service been more consistently available throughout our test locations. But in cities such as New Orleans, Phoenix, and San Diego, where Sprint still relies on its 3G CDMA network for data service, download speeds have fallen, and remain well below the 1 mbps mark.
4G Speed-Test Results: Reading the Charts
In our study we tested both with representative smartphones and with a laptop employing a USB modem recommended by the carrier. The laptop-based testing, which uses the Ixia industry-standard testing software, provides more precise metrics than smartphone testing does. The laptop results are a good measure of the maximum performance possible on a network and are a satisfactory predictor of the speeds that the network will likely deliver to smartphones in a year or so.
We use Ookla, an FCC-approved Web-based speed test, to measure data rates on smartphones. Those results aren't as precise for a number of reasons: we must use different smartphones on different networks, and the results necessarily reflect the limitations of the smartphone's radio chipset, processor, and battery, and the test itself comes with a somewhat higher margin of error.
The charts below (click to see enlarged versions) list the cities in the leftmost column; moving rightward across the chart, you can see the speed averages and network latency times for each of the four wireless networks. Speeds are expressed in megabits per second (mbps). Latency (or the time it takes a single small packet of data to travel to a network server and back) is represented in milliseconds. We recorded download and upload speeds and latency times during our laptop-modem tests, and download and upload speeds in our smartphone tests. (For more details, see "How We Test.")
Speed-Test Methodology in a Nutshell
Our testing method is designed to approximate the experience of a real laptop-modem or smartphone user on any given day in their city. PCWorld's testing partner, Novarum, tested in each of our 13 cities during the first six weeks of 2011. At each of our 20 testing locations in each city, we took a "snapshot" of the performance of each wireless service, testing for upload speed, download speed, and network latency.
We looked for the fastest signal available for each carrier, searching first for 4G service and then, failing that, defaulting to the carrier's 3G service. In all, we ran 177,000 timed performance measurements from 260 testing locations in both urban and suburban environments. (See "How We Test" for additional information.)
Because we couldn't test every city in the country, we chose 13 cities that are broadly representative of midsize and large wireless markets in terms of size and topography: Baltimore, Boston, Chicago, Denver, New Orleans, New York, Orlando, Phoenix, Portland, San Diego, San Francisco, San Jose, and Seattle. Because wireless signal quality depends to a large extent on variables such as network load, distance from the nearest cell tower, weather, and time of day, our results can't be used to predict exact performance in a specific area. Rather, they illustrate the relative performance of wireless service in a given city on a given day. Each speed number has a margin of error of plus or minus 5%.
Verizon LTE Blazes, 3G CDMA Slows
"Verizon's new LTE service smokes," says Novarum CTO Ken Biba, who helped test the network. The speeds tell the story: Verizon's 13-city average download speed for laptop modems is roughly 6.4 mbps, more than double the average download speed of our study's second-place finisher, T-Mobile.
And that average includes Verizon's result in Portland, the only city in our study that has no LTE service yet. Excluding Portland and looking at the performance of the LTE network only, Verizon's average download speed jumps to almost 7 mbps. Only in Orlando did the network average less than 5 mbps, coming in at roughly 4 mbps.
Upload speeds were just as impressive. Overall, Verizon's upload speeds averaged roughly 5 mbps in our 13 testing cities; average upload speeds reached nearly 9 mbps in San Diego and San Jose. LTE networks differ from older 3G networks in that they are designed to be symmetric--that is, the pipe going from the client device up to the network is as wide as the pipe going down to the client. In many of our 260 testing locations, the Verizon network delivered upload speeds that were faster than its download speeds. San Diego's average upload speed was faster than its average download speed.
Such fast upload speeds can make bidirectional apps like videoconferencing, online gaming, and, later, mobile Voice over IP (VoIP) work far more smoothly and look and sound better. In these apps, the data you send from your device is just as important as the data you receive.
Such apps also depend on near-instantaneous response from the network, with minimal delay. For instance, in real-time VoIP calls, network delay is usually the cause of "lag" and echo. To have a natural-sounding VoIP conversation, you need network latency of less than 150 milliseconds, and LTE proved better at assuring that than other networks in our tests. In our 12 testing cities where Verizon's LTE service is available, latency times averaged just 114 milliseconds, significantly shorter than latency times in the HSPA+ and WiMax networks we tested.
Verizon's LTE network gives us a nice look at the future of wireless service, but only a minority of the operator's customers are using the network at the moment. Verizon currently sells only two models of USB modems that can tap the network, and the company isn't saying how many modems it has sold. New LTE phones aren't likely to arrive until this summer. So Verizon's LTE network currently handles nowhere near the number of devices it will have to support in the future.
"Verizon's new 4G network is a screamer, but that's partly because there's hardly anyone using it yet," says Craig Moffett, a senior analyst for Sanford C. Bernstein & Co.
Verizon has been assuring skeptics that its network will remain just as fast when loaded up with devices. "We're very comfortable with the speeds we have said all along that our customers should expect: on average, 2 to 5 mbps on the uplink and 5 to 10 on the down," says Verizon Wireless spokesperson Thomas Pica. "That's on a fully loaded network.''
Moffett accepts that claim: "Even as [the network] begins to get loaded with the first smartphones this summer, it will probably keep the crown; as usual, theirs is the network to beat."
Still, at present, Verizon's smartphone subscribers rely on the company's 3G CDMA network. And that network, as demonstrated in our tests, actually became slower over the past year.
In our January 2010 survey of 3G service, we measured average download speeds of around 1 mbps in almost all of our testing cities (the 13-city average was 1.078 mbps) on our Motorola Droid smartphone. In those same cities this year, we saw very similar performance on our Droid 2 smartphone--again, most speed results were grouped around the 1-mbps mark, but the 13-city average download speed was 7% lower than last year's, at 1.008 mbps.
We found further evidence of a stagnant CDMA network in laptop-modem tests in Portland, where the Verizon LTE service is not available. We found an average download speed of 0.8 mbps in Portland last year, and clocked an average speed of only 0.55 mbps this year. This, of course, is lousy news for Verizon smartphone users, including those who recently bought the new Verizon iPhone.
Did Verizon build its impressive LTE network at the expense of further upgrades to its 3G CDMA network? Are the majority of Verizon subscribers paying the price for the blazing speeds enjoyed by just a few? Verizon chose not to comment on these questions.
T-Mobile Walks the Walk
T-Mobile began to brand its HSPA+ network service and phones as "4G" this year. Its ad campaign promoting the offering--you know, pretty girl, polka dots, poking fun at AT&T--has been hard to avoid. But our test results show that the carrier has been spending its money on far more than ad campaigns.
In short, T-Mobile's network is fast--far speedier and more reliable than it was just a year ago--and is indeed pumping out speeds that are competitive with the 4G services of the other providers. T-Mobile scored the fastest download speeds in our smartphone tests, and took a respectable second place behind Verizon Wireless in our laptop-modem tests.