SE Asia - Consistent Quality

Chapter 1 Key findings	Chapter 4 Data usage
Chapter 2 Consistent quality	Chapter 5 Coverage
Chapter 3 Download speeds

Download speed is most often used as a proxy for network quality, but while download throughput is important, it’s just one of several crucial requirements for a “good” connection. At its most basic, a good connection is one that doesn’t get in the way of users doing what they want to do. In the real world, smartphone users aren’t running speed tests all day -- they’re browsing the web, using apps, voice calling their friends, streaming Netflix and YouTube, or making video calls

To more objectively evaluate when networks are (and are not) enabling users to do those things, Tutela has developed a standard called consistent quality. The design of the standard is explained in further detail here. Simply put, it’s two sets of thresholds, called “excellent” and “basic”. If a connection hits the “excellent” standard, it’s sufficient for the most demanding mobile use-cases, like HD video calling or 1080p video streaming. A “basic” connection is good enough for simple web browsing, emails, and VOIP calling, but users will experience delays or buffering when trying to use more demanding apps.

Tutela’s consistent quality score simply measures the percentage of time that users -- whether for one operator or a whole country -- can hit the thresholds. The higher the number, the more often users have a basic or excellent connection.

Excellent quality:

4 Mbps down
2 Mbps up
0% packet loss
<30 ms jitter
<50 ms latency

Intended use cases: 4K video streaming, HD group video calling

Basic quality:

512 kbps down
128 kbps up
<5% packet loss
<50 ms jitter
<100 ms latency

Intended use cases: Web browsing, simple applications (Facebook, WhatsApp, email clients), VOIP calling

As the chart shows, all four countries in the report are relatively close when it comes to basic quality. Thailand takes first place, with users able to make a VOIP call or check emails at least 92.5% of the time when connected to one of the country’s networks. Malaysia was a virtual tie for first place, at 92.3%, while the Philippines and Indonesia were a few percentage points behind.

It’s when looking at excellent quality that we see the greatest differences between countries. Malaysia’s networks achieve the best excellent quality by far, with 56.9% of connections meeting the needs of the most demanding uses. Thailand is in second place, with Indonesia and the Philippines coming in third and fourth.

Malaysia

In Malaysia, three of the four operators score above 90% for basic consistent quality, with Maxis just taking first place ahead of DiGi and U Mobile. Celcom, which has the fastest average download speed of any individual operator, was in last place -- showing that there’s more to a good network than just fast average download speeds.

For excellent quality, the picture is reversed -- Maxis is still in the lead, with 68.9%, but Celcom is in second place at 61.6%. DiGi falls to third, with 57.5%, while U Mobile is in a distant last place. U Mobile’s last-place finish for excellent quality is unsurprising, given the average download speed is just 4.3 Mbps -- that is barely above the excellent quality threshold, and means that a large number of results will fall below 4.3 Mbps.

Indonesia

All five of Indonesia’s operators have a similar basic consistent quality score. Telkomsel is in the lead with 89.2%, but Smartfren is less than three percent back, at 86.8%. Consistent quality metrics only measure the quality of a network, not the coverage, which explains why Telkomsel -- despite having the biggest network by far -- only has a marginal edge in basic quality.

When it comes to excellent quality, however, Telkomsel’s advantage over the competition is much more marked. 57.8% of tests over Telkomsel’s network were sufficient for the most demanding uses, compared to just 32.6% on last-place Smartfren’s network.

Philippines

Smart has a small lead over Globe when it comes to basic quality, but excellent quality is where it really differentiates its network. Smart’s score of 48.8% is more than double that of Globe -- a particularly sizeable advantage when you consider that the two networks are only slightly different when looking at average download speeds.

Subscribers are unlikely to notice Smart’s marginally faster download speed -- the difference is just 2 Mbps -- but a consistent quality more than twice as good as Globe’s makes a marked difference in the user’s Quality of Experience.

Thailand

Thailand’s consistent quality scores are the tale of two network technologies. Thanks to its inherent design, 3G struggles to produce the kind of network performance necessary to hit Tutela’s bar for excellent quality. Although hitting a download throughput speed of 4 Mbps and latency less than 50 ms is theoretically possible over a 3G connection, it’s rare on congested, real-world networks. However, a good 3G network is perfectly adequate to provide a basic connection to users.

Looking at the difference between the basic and excellent quality charts, this difference stands out. TrueMove -- which has the best and most widely-available 4G network -- is in last place for basic quality, likely due to differences in signal strength and the quality of its 3G network.

When it comes to excellent quality, however, TrueMove is comfortably on top. Its score of 49.3% is the best of any operator, and comfortably above the national average. As other operators continue to acquire and deploy 4G spectrum, that lead is likely to dwindle, but for the time being, TrueMove’s investment in 4G is clearly paying off.

Southeast Asia

Malaysia

Indonesia

Philippines

Thailand