March 2019

For the United States’ mobile networks, ensuring coverage across the world’s fourth-largest country is a considerable challenge, requiring tens of thousands of cell towers and a complex web of spectrum. But the nation’s ten largest cities contain over 25 million residents within a handful of square miles, and ensuring all those people can use their smartphones is a very different type of challenge.

To see how operators cope in the largest urban areas, Tutela has examined over three billion network measurements, taken from 1st September 2018 to 28th February 2019 in the US’s 10 largest cities. The records include over 14 million speed tests, 242 million latency tests, and 2.5 billion total records on cellular networks.

Key Findings

• Verizon dominates at the national level as well as when diving down city-by-city. It recorded the best consistent quality score in 7 out of the 10 cities examined, including New York, LA, and Chicago.

• Across all operators, New York had the best consistent quality of any city examined. This is despite NYC having the heaviest average data usage by far.

• In a surprising result, two MVNOs — Google Fi and Xfinity Mobile — beat their host networks in some cities. Google Fi had the best consistent quality in Houston and Philadelphia, while Xfinity Mobile won consistent quality in San Jose. Data deprioritization and throttling play an increasingly significant part in the kind of connections that users experience, which goes some way to explaining the difference in results between MVNOs and their host networks. In addition, Fi’s customers can switch between T-Mobile and Sprint networks depending on which has the best signal in any one particular area, so the result bodes well for the post-merger New T-Mobile network, once Sprint’s towers and spectrum have been integrated.

Jump to individual city results

Measuring network performance in cities

When looking at network performance in dense urban areas, network capacity is just as important as network coverage. With millions of users crammed into a few square miles, the network can become overloaded at peak times, even when you’ve got a perfect signal.

As a result, relying on an average download speed to evaluate network quality isn’t a good method. As operators have worked to upgrade 3G networks to LTE-Advanced technology, theoretical (and even real-world) peak throughput speeds have increased to a level that vastly outstrips the maximum needed for any common use-case. As a result, a handful of multi-hundred-megabit download tests taken at 2AM can greatly skew the results, and the fact that cellphones become unusable on a train during rush hour doesn’t show up in the results.

At its most basic, a good connection is one that doesn’t get in the way of users doing what they want to do. In the real world, smartphone users aren’t running speed tests all day -- they’re browsing the web, sending emails, using apps, voice calling their friends, streaming Netflix and YouTube, or making video calls.

To more objectively evaluate when networks are (and are not) enabling users to do those things, Tutela has developed a standard called consistent quality. The design of the standard is explained in further detail here. Simply put, it’s two sets of thresholds, called “excellent” and “basic”. If a connection hits the “excellent” standard, it’s sufficient for the most demanding mobile use-cases, like HD video calling or 1080p video streaming.  A “basic” connection is good enough for simple web browsing, emails, and VOIP calling, but users will experience delays or buffering when trying to use more demanding apps.

Tutela’s consistent quality score simply measures the percentage of time that users hit these thresholds. The higher the number, the more often users have a basic or excellent connection.

Excellent Quality

Basic Quality