Gaming, streaming and conferencing over Wi-Fi: testing home routers’ QED is crucial
Latency and speed used to be the key performance indicators, but when dealing with many different kinds of activity simultaneously, QED provides a much better quality indicator. A Teams conference call requires a different approach than streaming The Mandalorian or taking names in Fortnite.
QED stands for Quality of Experience Delivered. It’s a metric that focuses on how satisfied your customers can expect to be with a product or service. You can read more about the concept of QED in one of our previous blog posts: https://www.excentis.com/blog/lag-or-bad-aim-in-counter-strike-qed-quality-of-experience-delivered/.
Wi-Fi has been (and continues to be) an easy target for unsatisfied users. An increasing effort now goes into finding the root causes why exactly they are unhappy, and what can be done to increase their sastifaction (and in turn lower churn and support costs).
Testing and comparing 11 Wi-Fi gateways
A customer approached us with a list of 11 competing Wi-Fi gateways. They selected these gateways to rigorously test and compare them, and find out how they each performed. We proposed not to test against the typical Wi-Fi performance indicators, and not to test for highest throughput rates or the largest coverage area, but instead a modern approach that sought to answer the following question:
“A family lives in a house where they are using the wireless network for gaming, video streaming and conferencing, simultaneously. Which gateway is best at providing all these services at the same time?”
How to test Wi-Fi performance
This probably won’t come as a surprise, but very relevant to testing: wireless users are mobile. For Wi-Fi connections, it matters if you are streaming your video in the living room next to the gateway, in your office two rooms away from the gateway or on a different floor.
Wi-Fi testing house upstairs bathroom
Wi-Fi testing house multiple devices
The same goes for any of the users in any of the rooms. So, a wide coverage of test results was constructed by testing many iterations with different users scattered around different rooms and floors.
With the wireless users connected and in place, we now needed to have simulated real traffic.
We analyzed the data profile of common use cases such as video streaming and gaming , so that these profiles could be mimicked by our ByteBlower Traffic Generator. These profiles describe characteristics as data rates, flow types (UDP or TCP) and packet sizes.
Wi-Fi test reports
Every time a setup was tested, a traffic report displayed the relevant results for all the users:
Video stream results showed the video buffer status and video playback:
Figure 1: Example gaps in incoming packets -> video buffer starts emptying
Gaming and conference call results show throughput and latency.
The example output shows large latency peaks higher than 160 ms during the short download burst of another wireless user:
Figure 2: Large latency peaks during short download burst of another user.
Which wi-fi router performed best?
After executing all tests for all 11 of the gateways, we collected and presented the results in a clear table that displayed the success rates (or lack thereof) of the gateways, and how they stacked against each-other. The winner was clear.
A happy client can guarantee the highest performance
We’re excited to state the client was particularly pleased with this outcome! Not only are they guaranteeing the highest possible performance, and lowering support costs, this is a solid message for existing and new customers: we have the best.
Increase customer satisfaction and lower costs: test your wi-fi router
Interested in discovering how your (candidate) CPE stacks up against competitors?
Let’s put it to the test!
