October 2021

While many industries across the world have experienced a down-turn in productivity and revenue in the past year, the performance of mobile networks has continued to improve year upon year. This is evident with Tutela recording a 20% increase in median download throughput, and a 16% increase in upload throughput, across a sample of 200 operators globally in the 12 months leading up to August 2021.

To the average mobile user this may sound like wonderful news, but how much of a difference will it make in terms of the overall experience when using common applications such as social media, video, web browsing, etc? Does a 20% improvement in download throughput equate to you, as a subscriber, being 20% happier? The answer is, it depends.

Mike Belshe, inventor of the SPDY protocol and one of the primary architects of the HTTP/2.0 specification, explains that whether you download a webpage at 10 Mbps or at 12 Mbps (an increased speed of 20%), the page load time will still be the same. However, the extra speed you may get for streaming a video with 20% faster download speeds may have some benefit in terms of playback quality, depending on other factors such as resolution and the codec used to encode the video.

If improved network performance doesn’t result in an improved application experience for subscribers, then this presents a risk of reducing the potential Return on Investment (ROI) for operators investing in network performance improvements. However, investments that are directly linked to improving the overall mobile experience, rather than just network performance, can reduce the risk of churn and increase data monetization opportunities. This is why it is critical to understand the performance of the network in terms of the subscriber experience.

Understanding Quality of Experience

The relationship between network performance and Quality of Experience (QoE) is shown below in Figure 1. QoE can be measured in terms of the Quality of Service (QoS), as defined by the ITU, matched against the expectations subscribers have for their service.

Take video streaming as an example: Imagine two operators A & B (shown in Figure 2 below) have applied a commonly quoted QoS KPI threshold of 5 Mbps download throughput to its offerings, recommended by Netflix for watching an HD stream. As shown, Operator A has the highest mean, median and peak download throughput performance and can offer a 4K experience to most of its subscribers. However, there are also a significant number of subscribers who will fail to experience Netflix in HD as their download throughput is below the required threshold. Operator B on the other hand has a narrower range of performance, but all of its subscribers are receiving download throughputs above 5 Mbps.

So which is more important: providing all subscribers a ‘good enough’ experience or having some subscribers with ‘great’ experiences at the cost of more with ‘poor’ ones? Ultimately the answer will depend on the operator(s) strategy; and by viewing the data through the lens of ‘subscriber experience’, it should better support this.

Download Throughput is often used as an indicator of mobile service quality but when it comes to subscriber experience KPIs such as latency and packet loss are becoming more significant. As we increasingly use our mobile devices to interact with each other we are also becoming more reliant on uplink performance in the network. Use cases such as gaming and content sharing continue to grow in popularity and video calling has become a critical service to many during the pandemic, all of which are increasing the demand for faster and more consistent upload speeds. So let us consider the consistency of operator performance in terms of providing a basic level of upload speed which will support a one-to-one video call, basic content sharing and a fair online gaming experience. Each of these services have varying requirements but an upload throughput of 1.5 Mbps, which is the requirement for a Skype video call in HD, is a common requirement for many of the apps in these use cases.

Comparing QoE across operators

When we look at our data, many operators unfortunately continue to push the top end performance without addressing the lower levels of experience, which are likely to be causing these customer satisfaction issues (Figure 3). The operator in this example has increased its median upload throughput over the 12 month period, yet the percentage of measurements below 1.5 Mbps remain at a similar level, resulting in minimal improvements to subscriber experience overall.

Figure 4 shows network performance in terms of both median upload speed and consistency for 194 operators around the world.

Based on this chart, LG U+ is one of the best networks globally, with the third highest median upload speed and the lowest percentage of measurements below 1.5 Mbps. SK Telecom and Singtel also show strong performance. There are also operators who have relatively low upload speeds, such as Deutsche Telekom and TPG but still provide relatively good consistency. Cellcom Avantel and Nextel on the other hand are amongst the worst operators for upload throughput consistency, and although Canadian operator Rogers recorded the 6th highest median upload speed of the group, it is ranked 95th in terms of measurements above 1.5 Mbps.

The increased demand for upload is challenging to operators due to the asynchronous nature of mobile networks, although those with TDD have the advantage of being able to adjust the allocation of time their networks support upload instead of download.

In some cases, performance is deliberately capped below performance levels known to give a poor experience - for example on ‘unlimited’ plans which have a data allowance at full speed and unlimited data at a capped speed, typically below 1.5Mbps. In other cases, flanker brands are used with lower speeds for a lower price, and subscribers on the main brand will be experiencing a better consistency of performance. Whether an operator is deliberately throttling performance or not, it remains critical for operators to understand the impact in QoE terms. In fact, this becomes more critical for operators utilizing throttling as part of a data monetization strategy in order to set performance at an optimal level based on the subscriber experience.

Here we have explored consistency of performance in terms of upload throughput, but the same principles can, and should, be applied to a combination of KPIs to reflect the QoE requirements for different subscriber use cases.

At Tutela we continue to pursue our mission of helping organizations in the mobile industry to understand and improve the world’s networks. Creating stronger links between crowdsourced data and subscriber expectations is just one of the ways in which we do this, please contact us for more information.