November 2020

At Tutela, our goal is to accurately capture and report customer experience on mobile networks. This is why Tutela developed the "consistent quality" measure, which assesses a network experience on whether it meets certain minimum thresholds for all of these KPIs. The minimum thresholds have been set using publicly available information published by content providers such as Google and Netflix and other research. The methodology is described in more detail here.

However, often it’s useful to compare user experience against a specific use case. In the current climate, with so many people working from home, the obvious priority is video conferencing.

In this study, we conducted some high-level analysis of operator performance against the requirements for group HD video calling set by popular platform Zoom, as well as experience standards for jitter, packet loss and latency which are integral for call consistency, and apply them to our vast crowdsourced dataset. 

Using these inputs, we performed a benchmarking analysis for the main UK operators, which demonstrated the below trend over time.

In Common Coverage Areas across the UK, EE led the UK operators in this measure of video conferencing, with Vodafone, O2 and 3 following in that order. In the high-level trend analysis, we can see that Vodafone suffered a dip in quality in August, while O2 has seen a steady increase in this measure over the course of the year.

However, an overarching benchmark just presents a headline overview. To get a deeper picture, we need to examine alternative lenses. For example, although EE performs well, a regional analysis shows the network experience is significantly affected in areas where we would expect to see more demand from holiday-makers, such as in the south-west and Wales:

As we can see, quality worsens in many parts of Exeter, as well as Torquay. By looking into the underlying network KPIs, we can also see significant drops in median download throughput, as well as some increases in average packet loss in these areas. 

Analyses such as this offer an alternative perspective on customer experience and allow operators to build and optimize networks in ways that deliver meaningful results for consumers. As we head into the age of 5G, it will be the operators that use the extra capacities and capabilities of the latest generation of network technology to support this that will end up succeeding. Tutela is investing in this area of research not only for video conferencing but also other popular and emerging applications and services in our bid to expand the way mobile networks can be measured and ultimately improved.