A new company has published results of their study of 3G wireless coverage and performance for the four major carriers (Verizon, AT&T, Sprint, and T-Mobile).

If you watch any TV at all, you can't miss the claims and counterclaims being traded between AT&T and Verizon over who has the best and/or most extensive 3G cellular data coverage in the US. While they have dropped their competing lawsuits (at least for the moment), a new start-up called Root Wireless has now shown up to throw some more fuel on the fire. They have published results of their study of 3G wireless coverage and performance for the four major carriers (Verizon, AT&T, Sprint, and T-Mobile) in and around seven major cities (Chicago, Dallas, LA, New York, Seattle, San Francisco, and Washington, DC).As you might reasonably expect, the results (Summarized in Table 1 at the end of this post) are a mixed bag, though a few key points stand out. In all of those markets, Verizon and AT&T had between 94% and 99.9% 3G coverage; Sprint came in almost as good but T-Mobile, who was late getting into the 3G game, had between 65% (Seattle) and 96% 3G coverage. In terms of data rates, AT&T had the highest downstream data rates in all 7 markets and Verizon came in 2nd in five of them; so it appears that the majors do offer the best downstream data performance. Interestingly, T-Mobile took second place in the other two, though that might be attributable to their rather poor sales, which results in having fewer people vying for access to their network. Be that as it may, you will notice that while Verizon's ads tout their coverage area, AT&T's ads are now talking about performance!

The upstream results were more mixed, with first and second spots divided roughly equally among AT&T, Verizon, and T-Mobile. Sprint went seven-for-seven in scoring the slowest upstream rates. The other major performance metric cited was the percentage of dropped connections. Sprint consistently ranked worst with 11% or 12% in all markets while AT&T and Verizon were typically in the 1 to 3% range; T-Mobile was 4% or 5%.

The real story, however, is in how the data were collected and the assumptions that were made--as always, the devil is in the details. Root Wireless has come up with an ingenious way of collecting their performance data. Rather then using the sophisticated mobile monitoring equipment the carriers use, Root installs their Root Mobile client on handsets that collect and forward the data automatically to Root's servers. They currently have clients for RIM, Android, and Windows Mobile devices with plans to add iPhone in the future. The data collectors, called "scouts," drive predefined grid routes through the coverage areas.

The biggest variable in the Root approach is how they determine data transfer rates. According to their Web site, the Root Mobile software analyzes the received signal strength by tapping into the software that drives the signal bars on the phone; recognizing the difference in receivers and antennas, they do lab tests to correlate bars with actual received signal strength. The big leap comes in translating that into data transfer rates. Unlike traditional PC bandwidth tests where test files are transmitted upstream and downstream, Root is apparently making assumptions regarding packet size and other factors and estimating the data transfer rate. As cellular data channels are shared among all of the users in that cell or sector, the biggest factor in data network performance is cell loading. In short, there are details that need to be sorted out.

Conclusion Overall, the Root Wireless performance data does provide some interesting insights but also raises some questions. We have seen the deficiencies in AT&T's 3G coverage (courtesy of Verizon), but the Root Wireless tests indicate that in the areas where they have 3G, AT&T seems to be outperforming Verizon. However, that information does not correlate with the boots-on-the-ground assessments of AT&T's iPhone users. In the meantime we have learned that AT&T has announced a "Mark the Spot" application for iPhone that allows customers to provide feedback on AT&T network performance from specific locations. First they wanted to charge us to install femtocells to improve their poor coverage while using our broadband Internet connections, and now we can help with their network diagnostics--what a deal!

The core issue with Root's approach is their ability to extrapolate meaningful network performance information regarding different devices (smartphones, laptops, etc.) doing different applications (email, Web surfing, file downloads, audio/video conferencing, etc.) based on the information their application collects. For the moment, they are only testing in major markets with significant 3G coverage, but the great thing about Root Wireless' approach is that they will eventually have a much more extensive network of scouts assembling data on a much wider scale.

If the information turns out to be meaningful, the question then becomes: who will get to see it? Will Root's testing results be provided solely to Root's testers, to the industry at large, or to the carriers who will use it to make even less and less enlightening claims regarding their performance? We're making arrangements to pose these questions to Root and we'll let you know what we learn.

A new company has published results of their study of 3G wireless coverage and performance for the four major carriers (Verizon, AT&T, Sprint, and T-Mobile).