Huawei Tampers With Benchmark Results, Because Others Do It Too
Recently, certain smartphone companies have been caught tampering with Benchmark scores, and one of them remains unapologetic about it. In fact, this company wants to justify it.
Benchmark tests are used to gauge a device's capacity and performance. Consumers look to this number to see if any given device is worth paying for, no matter the price. But when benchmark scores are tampered, Anandtech says the trust placed on it erodes significantly.
Two such smartphone brands that are found to have manipulated benchmark scores are Huawei and Honor. Reviewers at Anandtech have discovered that newer models from these two brands, specifically the Huawei P20 and Honor Play, both are able to tamper with benchmarking software to chug out higher scores.
These devices, Anandtech found, appear to have some sort of benchmarking detection tool that allows them to run with a higher power limit and higher thermal headroom for their SoCs. While the result of this is heightened power consumption, shorter battery life, and a lowered efficiency level, it does allow the SoC to perform at a significantly higher level.
For example, the Kirin 970 that powers the Honor Play was able to produce a whopping 127.36 FPS with the benchmark detection set to 'on,' compared to only 66.54 FPS with the benchmark detection tool set to 'off.'
Speaking with Anandtech at the recent IFA, Dr. Wang Chenglu, the President of Software for Huawei's Consumer Business Group, expressed reasons why Huawei had to do such a thing.
Dr. Wang said benchmark tests do not necessarily reflect real-life usage, and expressed his preference for benchmark tests that are more akin to how real people in real-life situations.
Huawei's team expressed appreciation for standardized benchmark tests that measure features found throughout the mobile industry -- items such as call quality -- and that they, like other tech companies, work to improve their devices.
Benchmarking the Competition
Huawei told Phone Arena that their devices are designed for meeting user's satisfaction based on real-life applications, not merely hitting high scores on benchmark tests.
Dr. Wang, however, told Anandtech one very important thing: Huawei can't just do nothing while other Chinese smartphone companies continually brandish unrealistically high benchmark scores.
He indirectly admitted that while Huawei wants to put more of their devices in the hands of consumers, their competitors make it hard for them because they cheat using ballooned benchmark scores.
And so, they do it too.