T-Mobile more than tripled its download speeds in our smartphone tests since last year. In our smartphone tests using the T-Mobile HTC G2, we measured a 13-city average download speed of 2.3 mbps. T-Mobile's 13-city average a year ago (testing on an HTC G1) was 0.72 mbps. In Denver and Seattle, our T-Mobile phone averaged download speeds of more than 3 mbps. We were able to achieve a connection speed of more than 2 mbps in 52% of our tests.
Upload speeds also rose dramatically from last year, improving from a 0.134-mbps average last year to almost 1 mbps this year. The T-Mobile network produced average upload speeds above the 1-mbps mark in five of our testing cities: Baltimore, Boston, New York, Orlando, and Seattle.
T-Mobile also scored very well, and improved considerably, in our laptop-modem tests. The network averaged almost 3 mbps for downloads, with average results nearing the 4-mbps mark in New York, Orlando, and Seattle. Overall, T-Mobile's download speed in our 13 testing cities grew 226% from last year's (very 3G-like) 0.87-mbps average speed. Latency times averaged 173 milliseconds, not high enough to disrupt services like HD streaming video, but enough to degrade VoIP call quality slightly.
T-Mobile's competitors say that the HSPA+ technology it uses is not really 4G as T-Mobile claims. That may be technically true, but T-Mobile has proven that through systematic software enhancements it can deliver speeds that are competitive with the 4G networks of its rivals. Given the near-term upgrade path of HSPA+ technology, T-Mobile will likely be able to continue doing so for the next few years.
AT&T Growth Slows
Following T-Mobile's lead, AT&T began branding its wireless broadband service and phones as "4G" this year. And, like T-Mobile, AT&T's HSPA+ service is definitely delivering 4G-like speeds. In our laptop-modem tests, the service produced an average download speed of 2.5 mbps in our 13 testing cities.
AT&T tells customers to expect download speeds of "up to approximately 6 mbps" in "key markets such as Chicago, Houston, and Charlotte [North Carolina]." Although we didn't see many 6-mbps scores in our laptop-modem tests, the network did hit download speeds of more than 2 mbps most of the time (64% of the time, to be exact). In fact, AT&T showed average speeds of roughly 2 mbps or greater in all of the 13 cities in which we tested. The network produced its fastest average download speeds in Chicago (3.3 mbps) and San Francisco (3.0 mbps).
AT&T's upload speeds were also strong, and similar to T-Mobile's. Upload speeds in our laptop-modem tests grouped around the 1-mbps mark, with Baltimore hitting a high of almost 1.4 mbps. This is a substantial step up from AT&T's 13-city average upload speed of 0.77 mbps in last year's tests, if not as dramatic an improvement as we saw in AT&T's download speeds.
AT&T's HSPA+ network produced latency times that were very similar to T-Mobile's. We measured an average delay of 169 milliseconds across 13 cities (T-Mobile's average was 173 milliseconds); we saw the highest average latency scores in San Diego (273 milliseconds) and San Jose (226 milliseconds).
Yet the growth of AT&T's data speeds has slowed. Last year we found that AT&T's data speeds had increased 72% over the previous eight months. This round, AT&T's speeds continued to grow over the past year, but not as rapidly, and certainly not as swiftly as its competition.
Consequently AT&T finished third in both our laptop and smartphone performance tests. In our laptop-modem results, AT&T trailed T-Mobile only slightly, but showed well less than half the download speed of Verizon LTE.
AT&T's slowing growth was even more apparent in our smartphone tests. In our early-2010 study, we measured a 13-city average download speed of almost 1.3 mbps on our AT&T iPhone 4, an improvement of 54% over the previous year. In this year's tests using the same phone, that number moved up to 1.5 mbps, an improvement of only 15%.
Some cities were better than others for AT&T smartphones: Chicago saw an average speed of 2.5 mbps while San Diego averaged only 0.8 mbps. Upload speeds improved dramatically, however, as our AT&T smartphone averaged 0.2 mbps in our 2010 tests and improved to just about 1 mbps this year.
AT&T believes that its new 4G smartphones (which weren't available at the time of our testing) and other devices will better utilize the speed of its network. "AT&T has introduced two 4G phones--the Motorola Atrix and the HTC Inspire--and has announced plans for about 20 4G devices this year," the company says in an e-mail. "Regarding network speed, thorough and expansive testing has concluded time and time again that AT&T operates the nation's fastest mobile broadband network."
AT&T's speed increases over the past two years can be attributed to software upgrades and infrastructure improvements. The operator completed a networkwide upgrade to HSPA 7.2 technology in late 2009, then announced earlier this year that it had finished another upgrade to HSPA+ technology, which it says allows for maximum theoretical download speeds up to 14.4 mbps. AT&T also has been investing large amounts of capital in fiber-optic lines for the movement of cellular data to and from the core of its network.
AT&T plans to launch its own 4G LTE network, as well as some 4G LTE smartphones to match, later this year.
Sprint Needs More 4G, Less 3G
The good news for Sprint is that the overall speed of its data service has increased significantly during the past year, about 170%, in fact. The bad news is that while Sprint offers its WiMax service in most of our test cities, actually connecting with the WiMax signal using our Sprint 3G/4G modem proved a hit-or-miss proposition. For instance, in San Jose, California, we measured download speeds of below (sometimes well below) 0.5 mbps in 8 of our 20 testing locations, a sure sign that no WiMax service was available in those places.
When the 4G service is unavailable, Sprint devices downshift to the company's 3G CDMA service, which, our laptop-modem tests suggest, may have slowed somewhat over the past year. Average download speeds slowed considerably in New Orleans (-24%), Phoenix (-31%), and San Diego (-24%)--the three cities in our tests where no WiMax is available.
Sprint says no such slowdown has occurred. "The 3G speed results you saw do not match what we see, and what the independent third party testing our network has reported," says Sprint spokesperson Stephanie Vinge-Walsh. "We haven't seen any significant degradation in 3G from last year to this year; our 3G speeds remain in the same range and at the same high dependability levels."
Sprint's 13-city average download speed of roughly 2.1 mbps represents a mix of CDMA and WiMax--3G and 4G--connection speeds. Overall, we recorded throughput speeds of more than 2 mbps in about half of our tests. In the majority of our test cities where WiMax was available, we noted (anecdotally) a roughly 50-50 chance of connecting to the service. There were exceptions: In Baltimore, Boston, and Chicago, the laptop-modem speed results reflected that the 4G network was available throughout the cities, with a few exceptions.
Of its 4G WiMax service, Sprint says users should expect average download speeds of between 3 mbps and 6 mbps, with peaks of more than 10 mbps. Our tests left us skeptical of Sprint's claim. We never saw a speed higher than 7 mbps, and we reached speeds of 6 mbps or more in only 5 of our 260 testing locations. The WiMax network produced a fair number of speeds within the 3-to-6-mbps window, but not consistently.
Sprint's upload speeds also tell the tale of a 4G service with spotty coverage. In many of our testing cities, we saw mainly two kinds of upload speeds: those of 1 mbps and above, suggesting that we had managed to hook into the WiMax service, and those that were below (sometimes well below) 0.4 mbps, suggesting that we had connected to the 3G CDMA service. Overall, Sprint's average upload speed remains stalled in 3G-land, at just 0.6 mbps.
Sprint's CDMA and WiMax networks, combined, produced the worst average latency score in our tests, at 214 milliseconds. Such network delay can begin to degrade the smooth operation of real-time applications like video chatting and VoIP calling.
The same disparities in Sprint's 3G and 4G networks showed up in our smartphone tests. In locations where WiMax coverage was spotty or nonexistent, average download scores were well below the 1-mbps mark. In cities where we could regularly connect with the WiMax network (Boston, Chicago, and New York), we saw download-speed averages of 2 mbps or greater.
Despite its overall speed gains, Sprint's service ranked last in both download speeds and upload speed in this year's laptop-modem tests. Had Sprint's WiMax network been widely available in all of our testing cities, the results would have been much different. The 4G network isn't slow, it's just not in enough places.
"Coverage has always been their Achilles' heel in 4G, and financial problems at WiMax partner Clearwire have slowed down their 4G network expansion nearly to a stop," says Sanford C. Bernstein's Moffett. "A year ago, they were first to market; now they're at real risk of falling behind."
The 4G Cometh
An important transition from 3G to 4G is under way and will continue raising the bar for fast mobile broadband. If speeds continue increasing at the rate they have been over the past year, 3G data service (and speeds) will soon become just an unpleasant memory. Our tests show, conclusively, that the 4G wireless service the carriers now offer--if it's available in your neighborhood--is already significantly faster than 3G service.
What will that mean? The 4G service will very likely speed up your consumption of Web-based content, and smooth the operation of services such as streaming video. In fact, 4G speeds are likely to let you do things with your mobile device that you simply couldn't do with a 3G connection, applications such as video chatting, online gaming, and VoIP calling. 4G is the first incarnation of wireless broadband that might finally free people from the desktop, allowing us to manage our online lives whenever and wherever we want.
How We Test
In each city, we tested from 20 locations situated in a grid over the center of the city. These locations are roughly 2 miles apart, allowing us to measure service levels among and between numerous cell towers. At each testing location, we subjected the networks to industry-standard stress-testing using laptops, and we put the networks though Internet-based testing using smartphones.
Our laptop-modem tests use a direct TCP connection to the network to test the network's capacity--that is, the speed and performance that the network is capable of delivering to subscribers. To connect the laptop to the various networks, we used the fastest USB modem available, as suggested by the carriers themselves. We used the LG VL600 4G USB modem to test Verizon, the ZTE WebConnect Rocket 2.0 USB modem to test T-Mobile, the Sierra Wireless 250U AirCard to test Sprint, and the Sierra Wireless USBConnect Shockwave to test AT&T. Using the Ixia Chariot 4.2 testing tool on our laptop PC, we tested both the speed and the latency of the network.
To measure download speed, Chariot requests a number of large, uncompressible files from a server in the San Francisco Bay Area, then from another server in Northern Virginia. For each server, the software measures the speed of each transfer during a 1-minute period, and then creates an average of the results.
To measure upload speed, Chariot sends a number of files from the Chariot client on the laptop to the local and distant network servers, again timing each transfer during a 1-minute period. We report the average of all of these transfers, both from the local and distant server, at each location as the average for that location.
During the speed tests, the Ixia testing software also measures latency, or the time it takes for a packet to move from the client laptop to the network servers and back again. This metric, expressed in milliseconds, can reveal delays or bottlenecks in the flow of data through the network, and can foretell how well real-time applications such as voice calling and video chatting--which require nearly instantaneous packet transfer to work smoothly--will work on the service being tested.
Our smartphone tests, which we run from the same locations as our laptop-modem tests, approximate the real-world connection between specific smartphones and specific networks. For the tests, we used AT&T's Apple iPhone 4, Sprint's HTC EVO 4G, T-Mobile's HTC G2, and Verizon's Motorola Droid 2.
On each phone we run the FCC-approved mobile-broadband performance test from Ookla. The test sends a large file back and forth between the smartphone and a network server, and then measures the speeds at which the data transfers. We perform three upload tests and three download tests at each testing location.
We tested all 13 cities during January 2011 and February 2011, using the same locations, methodology, and personnel we used to test those cities in our January 2010 tests. Maintaining a consistent methodology allows us to compare the performance of the networks over time and to look for evolutionary changes.
Our research was not comprehensive. We did not exhaustively survey every city. We tested from stationary locations only, we didn't test indoor performance, and we did not measure voice service.
For quick tips on how you can try evaluating your phone's network, see "Test Your Smartphone Data Speed."
This story, "4G wireless speed tests: Which is really the fastest?" was originally published by PCWorld.