So Who Really Won the Big Game on Sunday: Verizon, AT&T, T-Mobile or Sprint? Root Wireless Reports...

Root Wireless on the SuperBowl
By Kevin Michaluk on 9 Feb 2010 04:28 pm EST
-
loading...
-
loading...
-
loading...

Back at CES we told you about Root Wireless and their mission of reporting on carrier network performance using data collected from real users. In case you missed that post, be sure to check out the video here.

An event like the SuperBowl puts a whole lot of cell phone users in place, so the folks at Root Wireless decided this would be a good instance to see just how well each carrier performed. You can find the whole report below, but here are some of the highlights:

  • Verizon Wireless was the day's ‘loser,' recording far more connection failures than typical, while providing the slowest service at the game Sunday.
  • Of the ‘Big Four' carriers, AT&T provided the fastest service - despite the expected congestion created in part by the iPhone's popularity. But like Verizon, it reported much higher connection failure rates than usual; attempts to connect failed 16% of the time.
  • T-Mobile was a winner, far and away recording the fewest connection failures and providing transmission speeds on par with its everyday service. Sprint also performed close to par, relative to its everyday service. (Charts included with report.)

It's pretty interesting that prior to the SuperBowl three of the big four carriers issued press releases saying they would beef up their service for the game (read Verizon | AT&T | Sprint). Despite the high connection failure rate, it appears that AT&T's efforts paid off, given that they maintained speedy download speeds. As for Verizon... well... Root Wireless tells us they have never performed so poorly in one of their tests before. What's up with that?! Full Report below.

Root Wireless Release - Wireless Data Network Performance: Sun Life Stadium, Miami

More than 76,000 Fans at the Big Game put Wireless Data Services to the Test

The capacity crowd that packed Sun Life Stadium for The Big Game Sunday gave Root Wireless another opportunity to test the performance of wireless networks jammed by heavier than normal traffic demands. Monitoring network activity throughout the game, Root Wireless analyzed network speed tests designed not to determine absolute maximum data throughput speeds, but rather the actual performance experienced by smartphones users downloading and uploading data before, during and after the championship football game.

‘Big Game': AT&T Wins Speed Tests; Sprint, T-Mobile Perform Well; Verizon Slowest

Scouting the Miami marketplace prior to the week that lead up to the Big Game, Root Wireless learned that each of the carriers - AT&T, Sprint, T-Mobile and Verizon - provide competitive data services in the greater Miami community on any given day. On game day, however, data services provided by AT&T and Verizon Wireless reported significantly higher connection failure rates than usual, while services provided by Sprint and T-Mobile largely were on par with their everyday results.

  • AT&T provided the fastest data downloads, averaging 348 kbps; its average upload speed was 68 kbps. However, AT&T also reported the highest rate of data connection failures: 16% of connection attempts failed. AT&T's average signal was 81% of full bars.
  • Sprint's signal was the strongest, 96% of full bars, and its data service averaged 231 kbps when downloading, 68 kbps when uploading. Sprint's data connection failure rate was 9%, slightly higher than its typically reported failure rate.
  • T-Mobile reported the fewest number of data connection failures, 3%. Its data service averaged 237 kbps when downloading. Its service was fastest when uploading data, averaging 126 kbps. T-Mobile's signal strength averaged 72% of full bars.
  • Verizon reported significantly more data connection failures than usual: 11% of connection attempts failed. Furthermore, Verizon's transmission speeds were not in line with its typical performance, averaging 97 kbps when downloading data; its upload speeds averaged 58 kbps Verizon's average signal strength was 54% of full bars.

Methodology
Loading its proprietary Root Mobile crowdsourcing application onto smartphones purchased off-the-shelf from each of the ‘Big Four' carriers, Root Wireless conducted stationary tests at a fixed location in the Sun Life Stadium parking lot. The company ran Root Mobile continuously on each of the four phones from 1 p.m. to 11 p.m., February 7. The tests reported data transmissions speeds, cell tower IDs, network connection failures and other performance indicators delivered by Root Mobile. It is noteworthy that these tests differ from data transmission-speed tests conducted by other companies using PCs, precisely because Root Mobile was engineered to determine consumers' real-world experience using smartphones, not PCs. Also notable: results reported here are local to Sun Life Stadium. Root Wireless testing across several metropolitan areas has confirmed that all network performance is local, varying from neighborhood to neighborhood, and from service provider to service provider.

Summary Observations
Sprint and T-Mobile were both winners on game day, providing data service that nearly mirrored their respective baseline performances, as established on more typical days. Though AT&T provided the fastest service when downloading data, it reported far more connection failures than usual. Verizon also reported a much higher rate of connection failures than usual and, atypically, Verzion's service was significantly slower than its competitors' services.

Root Wireless SuperBowl

About Root Wireless
Root Wireless records objective data that measures carrier network performance using off-the-shelf smartphones, and reports right down to street level in specific neighborhoods. The Root Wireless charter is to provide a standardized, objective performance mapping service that helps consumers choose the service and smartphones that are best for them. Root Wireless gathers network performance data using commercially available smartphones equipped with Root Mobile, its proprietary crowdsourcing application that runs unobtrusively in the background as a service on RIM, Android and Windows Mobile smartphones.

Root Wireless does not work on behalf of any of the carriers. Its goal is to provide individual consumers with detailed network performance information personalized to the specific neighborhoods where they most often use their service. The company's initial findings have confirmed that there is no one ‘best carrier'. Which service works best depends upon where consumers actually use their phones. The Root Wireless performance mapping service is currently available in 15 major metropolitan markets; Root Coverage maps are accessible at http://reviews.cnet.com/coveragemap/.

28 comments

UFCfan

GO BIG RED!!!!!!!!!!!!!!!!!

stuaw11

Go Big Red? For being the slowest data speeds and 2nd most data errors?

Honestly, this shouldnt be shocking to anyone. The 2 carriers with 1/3 the customers of the 2 big boys had easier times connecting. Wow you dont say?!? 1/3rd the users equals more tower slots, shocking!

iLovemy_bb

I have T-Mobile and frankly, I and everyone else who uses T-Mobile am quite satisfied with their service.

Blechtman

I damn sure wasn't. Switched away from them asap

Fishkydid

I think this is a very good question. How many actual phone were on each network?
If Sprint/T-mobile only had a smaller larger then normal day. Might be why they faired much better. Were ATT and Verizon might have had 2 or 3 times the normal.

MrWireless31

stuaw11 hit the nail on the head! It makes sense that the two companies with the most subscribers in the US faired poorly because of just that - THEY HAVE THE MOST SUBSCRIBERS!!! T-Mobile and Sprint customers fared well because they had all of 1000 phones scattered across the city of Miami while VZW and ATT had probably about 20 times what the other two had combined. Lets compare apples with apples, people, when reporting ridiculous, skewed stats, shall we? It's called integrity, Root Wireless, and I dare say that's probably the reason why I've never heard of you before.

rizzzzoooo

Yeah but the other side to that argument is that they had the appropriate amount of towers/infrastructure to handle their traffic. So if ATT and VZ had "20 time" more than the hypothetical number you gave for say Sprint, then its their fault for not have the adequate infrastructure to handle their business. So yeah it still looks good for TMO and SPR that they maintained the performance they did because. Don't beat them down because of a numbers game.

chetpdes

There's an error in the graph on the left: They reversed the "Upload" and "Download" bars. How did Root not realize this?

nasmith7

don't know how accurate tests like these can be but hated to see that VZ came in last in everything in every city (didn't check San Fran). makes me wonder y i switched in the first place.
@chetpdes good catch, that is pretty sad...

gafly

Doesn't the founder of this company and a board member have former ties to AT&T?

thame555

The number of customers that a wireless company has got should be irrelevant. All I want to know is can I make my call or get my data quickly. If T-Mobile & Sprint make a better job at providing that to me than that is all that I need to know. If a company is successful in attracting customers then I expect it to grow its infrastructure with them so I can continue to receive the same level of service that I initially signed up for. Either VZW & AT&T's network has not kept up with their customers needs or T-Mo & Sprints has kept pace with their customers. You choose your reasons.

andysbarry

This "test" is irrelevant to practical network performance. Unless the networks can release specifics about their network load during these hours, there is no accurate measurement with the high number of variables in this situation. Show me usage data on customers versus available bandwidth in the area, then this whole test might be worth it, otherwise Root wasted their time on this. I'm ashamed to see that Cnet and Crackberry are willing to support Root Wireless content.

+1 MrWireless31
+1 stuaw11

meske

I disagree. Root is measuring user experience. Even if VZ was pushing 100x the bandwidth of T-Mobile, the fact remains that there were issues with the services ability to keep up with demand.

abolds4397

Who cares how many customers VZW and AT&T has! If they can't keep up then they are slacking! T-Mobile and Sprint did a better job at calls and data and that is a fact! Don't try to justify that the amount of people is the reason number 1 and 2 couldn't keep up! T-Mobile and Sprint did a good job at keeping pace and that is all that needs to be said. For those with VZW and AT&T, this info will hopefully help them in the future. I love T-Mobile and I have not had a problem with them.

xshawx

Obviously you are a iPhony user. Check Crackberry just a few days ago. The Blackberry Curve was the number ONE selling phone this year. At&t has 20 times more phones? They also have more towers, brag about their 3G coverage, and are vastly over-rated just like Apple products. I own some and formerly worked for them. The ipod/iTouch, but not the phone. Sorry

andysbarry

You've completely missed the mark on that. He's referring to the number of customers in the area of the Super Bowl on Sunday night. "VZW and ATT had probably about 20 times [the number of phones around Sun Life Stadium that] what the other two had combined."

I'm certain there weren't ten AT&T towers in the coverage area of the stadium more than the other carriers. The boosting of the network usually can occur with hardware on a single tower or it's all software based. At most there may have been 3 maybe 4 towers for EACH of the carriers within the cell. And if you were smart shaw, you would have checked MrWireless's public profile to see that he's on Verizon with a Storm2, if I'm not mistaken. Sorry.

And abolds4397, there's a reason you (and everyone) refer(s) to them as number 1 and 2. Customer base and typical network performance.

dj2big

I have had all these carriers
Verizon | AT&T | Sprint | US Cellular | T-Mobile

They all have there ups and downs but I believe that overall Sprint provides excellent service, good customer care and the best rates. US Cellular is my second choice, T-Mobile, Verizon and last AT&T. This is based on over 21 years of cellphone service. AT&T really changed there forgotten CS problems when they changed the name from Cingular. US Cellular is a small company but provide excellent service CS can be iffy but still ok. T-Mobile CS is terrible but there service is ok. Verizon to me they are a bunch of marketing hype. I really do think Sprint provides a good balance of service and customer care.

mrscott45

I suppose if you live and work under the umbrella of a 'Sprint strong' metropolitan area, or any other carrier's strong side, you could say the same.

But, try driving coast to coast, and see the roaming charges you accrue. It will be notable.

Specifically, Sprint states in the small print on their sales brochures, that the majority of your minutes, regardless of the number, must be used on a Sprint system.

Meaning that if you regularly drive in and out of extended network (roaming) areas, and half or more of your usage is extended network, you will be billed roaming charges.

I did, and they did.

I had a Sprint phone for a backup for a little over a month. ( until I received my first bill) In that time, they billed me for $80 in roaming charges above my plan cost of $125, because I spent more than half my usage on extended network towers. Drove over that phone with my big truck.

Sprint also limits your 'mobile to mobile' free calls to their network as well.

I've been a Verizon customer for 7 years. I'm a 48 states, long haul trucker. My average usage is nearly 5,000 minutes a month. I have the $79.99 900 min. Nationwide Talk and Text plan. (for the 5 friends and family). Naturally, the BB data plan, VZ Nav, and several other added features.
Unless I download apps, or use 411 several times, my bill is usually right at $135. No surprises, no roaming, even though I drive my thousands of cell towers every month. My mobile to mobile calls are always free calls, which represent nearly all of those 5,000 minutes. Data= always. I can use UberTwitter on Jimbo Billy-Bob's cell tower, in the middle of the high desert, 75 miles from the nearest cactus.

If I have difficulties in an area I frequent, a call to Customer Service, an American in America, as opposed to an Indian named Stevie in India, generally renders a conversation with a Data Technician, who dispatches a Service Technician to check up on 'ol Jimbo, and correct the problem.

So, to Sprint, AT&T, and their offspring, I say:

If you can't run with the 'Big Dogs', stay under the porch.

Go BIG RED!

clexman

This "Data" proves nothing when it comes to the services each of these carriers provide under normal circumstances. Whenever there are large events, mobile phone companies bring in temporary cell towers to add capacity. This just means that AT&T added the most capacity relative to their number of subscribers.

The only thing this tells us is that AT&T spent more money and effort on the Super Bowl than the other carriers.

If this was a NASCAR race, I can guarantee you that Sprint would have come out on top.

edit: I should have mentioned that I have worked in large high profile events for over 10 years and was at 5 Super Bowls in the last 10 years. They don't just turn up the power on existing towers, they add several more mobile units to the surrounding area and take them away when the show's over.

michaelm91

hey everyone,

just to drop a quick note about the stadium. living in Miami i go to a large amount of Dolphins and U of Miami football games there. on a normal game day trying to reach someone on AT&T is nearly impossible inside the stadium. once called my friend 40 times (iPhone) to pick up a pair of tickets and not one reached his cell.

i use Sprint and with my 8330 reception is decent and usable. The only difference i see while being at the stadium is my phone would sometimes drop from EVDO to 1xRT

edit: added the new name of the stadium... stupid they change it again..

clexman

That's for a normal game. During the Super Bowl, towers are added in the parking lots and media areas around the stadium. I have always had great data & voice coverage on any network starting 2 weeks before Super Bowls.

teecee81

Verizon with 91.2 million customers, AT&T with 81.6 customers, Sprint with 46 million customers and T-Mobile not worth mentioning on their site because it doesn't compare to VZW and AT&T. When you have more people using the network and some of these carriers use the same network (VZW & AT&T), you will see a difference in speeds, data connectivity and reliability. Something that should be considered.

manimsoblack

I'm pretty sure sprint and vzw use the same network as they are both cdma carriers. Further proof is the map in all the vzw commercials stating how much better their 3g coverage is than att. If it were the same network it would be identical or nearly identical maps no?

dperry360

I always knew T-mobile had it in them lol

caseydietz

It would sound to me like we have a lot of VZW apologists here... I thought that is why Verizon was so much more expensive than other carriers. "They take that extra $$$ and reinvest it into their network", I can only assume for situations like this? In my opinion they should be able to handle the extra traffic since their network is 'superior'.

jkpitcher

What percentage of each carriers customers had a drop call or drop in coverage. That is the main number. Verizon and AT&T may have had more issues overall but the percentage of users that may have had a concern could be less. That is what I want to see.

krs7272#AC

I worked the game and have Sprint and a girlfriend of mine that I was working right next to has an Iphone and she definitely had better results for data i.e. uploading pics then I did 10 fold.