Linus Torvalds (torvalds@linux-foundation.org) on 1/25/12 wrote:
>
>Graphical applications end up doing several layers of
>indirection through libraries - something that you never
>see in Spec. In fact, most Spec runs are outright lying,
>in that they tend to flatten out even the one layer of
>libraries there is - inlining a ton, and usually statically
>linking the rest.


I fully agree, GUIs are the most visible and important performance component every daily user of any computing device encounters, but are essentially absent from all benchmarking. It is hit obliquely here and there within some Windows benchmarks that test "office" performance by having macros perform a given list of MS Office functions, but it is lost in the noise against stuff like disk I/O and calculations (Excel) and text searching/filtering (Word)

Perhaps this could be rectified to some extent by creating a benchmark to test the GUI, while eliminating other variables as much as possible. To wit, create an iso version of Linux that boots into a RAMdisk like the 'Live' versions do. Have it able to run a test using an xorg driver that's basically a /dev/null. Feed in a pregenerated sequence of mouse events to exercise GNOME and all the lower level libraries by running a few big nasty programs like GIMP, Firefox and Open Office through some simple paces that don't require any heavy computation on their own. Just starting up these applications, opening menus and moving windows around is probably sufficient.

This would exercise the hell out of the icache, branch predictors and so on by running through all the GUI layers, plus the various kernel memory management and I/O layers (but running off a RAMdisk, the disk I/O itself would be abstracted out) While the Windows OS is very different, I think the relative results between two different systems under this test on Linux would be fairly comparable to that which one would find if the same type of test could be legally constructed using Windows.

It may even be possible to create a mostly comparable version using the Android version of Linux as the kernel but the same userspace to give a halfway decent idea of how far apart the ARM CPUs used in Android phones are from the Intel CPUs used in laptops and desktops. It wouldn't be exact but it would be way better than the synthetic benchmarks we have now or the SPEC results we don't have.