B&C: Hi Jerry, first of all, thank you for this interview. Can you explain to our readers what your position and roles are at AnTuTu Labs?
Jerry XIA: Thank you, Gian Maria. I am Market Manager of AnTuTu. I am mainly responsible for contacts with overseas companies, such as ARM, Qualcomm, etc.
B&C: AnTuTu is a very popular benchmark for smartphone and tablet, and it's used by reviewers and famous IT sites, like Anandtech. We can say that AnTuTu is a reference in the benchmark world. When Jim McGregor posted this article, what did you think?
JX: AnTuTu Benchmark has over 5 million downloads on Google Play, and it has over 80 million users around the world. I am very confident that AnTuTu Benchmark has surpassed other benchmark apps through Google search results. As for the disputation of ARM beat Intel, I admit this is our mistake. That's why we perfect it. No app is perfect, including AnTuTu Benchmark. We have the determination and ability to make AnTuTu Benchmark more accurate.
B&C: Now we know that the problem of AnTuTu 3.3 was the Intel C++ Compiler (ICC), and the problem was fixed with the release 3.3.2. How could it happen? The difference between the results of the two version is not tiny.
JX: We have improved our standard, and made corrections to the previous compliers as well as algorithm. I am not an engineer, so I am afraid I can not explain this to you in details. AnTuTu V4 will apply a new standard. This standard will be very different from V2 or V3. Scores from V4 can not compare with the scores tested on V2 or V3 (such as AnTuTu 3.3.2).
B&C: Also we know that AnTuTu is written with two compiler: GCC (GNU Compiler Collection) and ICC (Intel C++ Compiler). Why didn't you use only GCC? E.g., Ubuntu is compiled by GCC.
JX: We have to take all aspects into consideration in benchmark test, i.e., 2 compliers can reveal different aspects in performance test better.
B&C: Here you write that the release 4 of AnTuTu will be “a return from ultimate hardware performance to user experience”. Can you disclose something to us?
JX: We have added more test projects in V4, such as Application Performance (including Dalvik virtual machine performance test) test. You know, naerly all Android apps contain Java-like Dalvik code, and 85% apps use only Dalvik code. So the performance of Dalvik virtual machine is of great importance to user experience. Also the working quality of thread will be very important in V4 while core number will be less important. Multitask test is an important test project in V4. For example, most apps use only one core. It is no doubt that you can have a better user experience when use these apps on a high-end dual-core CPU device than a low-end quad-core device. That's why Nexus 10 (has an excellent dual-core device) can get a higher score than Galaxy S3 in V4 multitask test.
B&C: Can you tell us what kind of test AnTuTu accomplish? In the developing of AnTuTu did you utilize preexisting tools or libraries set or did you create ones of your own?
JX: Actually, if you have notice the development of benchmarking apps, you will find that AnTuTu Benchmark is the first benchmark app in the world who supports dual-core test, quad-core test as well as octa-core test (in V4). Most algorithm in AnTuTu Benchmark are developed by ourselves, because we have different standard with our competitors. Maybe AnTuTu Benchmark is not the most professional one, but our standard is designed for users. We know what users want to see.
B&C: 04/03/2013: “3DMark for Android is just a gaudy joke. We do not think it can test the real performance of GPU. It is more like an app made by Qualcomm” (Link). Futuremark is a big company, present in the benchmark world since 1997. Now it is developing benchmark software for mobile devices: aren't you worried about this?
JX: 3DMark has a large user group around the world, and it is famous for its PC benchmarking software. But we should know that because of the screen size and bandwidth limitation of mobile devices, some aspects in the scene specification, such as vertex number, computation accuracy and complexity, are different from PCs’. Copying the benchmarking tests of PC to mobile devices are obvious unable to reveal the real performance. We take 3DMark as either a competitor or friend. We have something need to learn from them. Also we have surpassed them in many aspects. 3DMark still got a long way to come in Android benchmark market.
B&C: Futuremark, in the past, bundled its own software: e.g. 3DMark Vantage was bundled with these sticks of high-end RAM.Could you do the same with some high-end smarphones?
JX: AnTuTu Benchmark has the ability to test the real performance of high-end smartphones, such as Samsung Galaxy S4 (Octa-core). It needn't a special version for these devices. Besides, we do have a professional test app especially for manufacturers and hardware suppliers: AnTuTu Test Suite. This app has more professional functions which can test specific performance, such as CPU load. Since some of these data have no meaning to common users, we only applied part of the functions of Test Suite to AnTuTu Benchmark V4. Generally speaking, AnTuTu Benchmark does not have corresponding advance or pro version especially for high-end smartphones on Google Play like 3DMark, but we have AnTuTu Test Suite which is beyond other benchmark apps.
B&C: AnTuTu, day after day, has more and more competitors. Do you have in mind some alliance with the big companies, like Samsung or Motorola or Asus? Asus, in particular, has a lot of experience with the benchmarks in the PC area.
JX: Actually, our team has over 10 years of development experience on Windows (since 1998). We have developed lots of utilities for Windows, including benchmarking software for Windows. So we believe we have the ability to develop Android benchmarking app well and yields to none. We do have connections with manufacturers as well as hardware producers, and we welcome more coorperations in the future. They can provide us the support of new technologies and we can tell them the trend of users' requirements.
B&C: In this interview, Tamás Miklós, Lead Developer of AIDA64, says that the documentation for the developers is inadequate in the PC industry (and we agree with him). Do you have the same problem in the Mobile industry? ARM and Qualcomm support effectively the development community. Do Intel and nVidia the same?
JX: Yes, we have the same problem in Mobile industry. We have to develop some tools to do tests by ourselves. We will keep in touch with all the hardware suppliers, such as ARM, Qualcomm, Intel and nVidia for creat a fair AnTuTu Benchmark.
B&C: AMD will be in the Mobile SoC market soon. What kind of compiler will you use for these SoCs in case? What do you think about AMD tablet products?
JX: We haven't contact with AMD. So I do not know the details. Tablet is becoming more and more popular. I will be glad to see AMD can achieve a lot in tablet market. It is good for users anyway.
B&C: What do you think about Windows RT and the Surface Tablet? Is it a dead product or not?
JX: Though Windows RT and Surface Tablet performed poor on the market, I think it is not a dead product as long as Microsoft willing to invest more in system development. In any case, Microsoft has many years of system development experience. It has the ability to make good products.
B&C: In your opinion, the ARM SoCs could be in future a good choice to equip notebooks or desktops PC?
JX: ARM is famous for its efficiency, energy conservation as well as cost effective. It may be a good choice to equip notebooks in the future.
B&C: Thank you Jerry, it has been a pleasure talking with you.
JX: Thank you, Gian Maria. Please feel free to contact me if there are something more you want to know.