Though my background is primarily on the software side, I am very curious about the effects that different processor designs and features have on system performance. Being somewhat less knowledgeable about CPU architecture than many others, I have been looking for some way to see these effects in a concrete manner. Benchmarks are supposed to provide this ‘visible’ evidence, but there are probably as many different application mixes and workloads as there are users. This isn’t a problem for a corporation that has the money to spend to evaluate how their applications and workloads will perform on a given platform – but the average user, such as myself, is left wondering what these benchmarks are really measuring so they can be applied to our own usage.
About a year ago, I decided to embark on a task that I have been thinking about for quite some time. This involves running a number of benchmarks on a variety of platforms in a very controlled manner. Eventually, I hoped to have a sufficient number of results so that some sense could be made out of all of the marketing claims and conflicting arguments. To this end, over the past year I have acquired several unlocked Intel and AMD processors through various means, which as allowed me to make some comparisons that would otherwise be impossible.
As a starting point for this research, I have focused on comparing various processors from both AMD and Intel at the same clock rate (1.2GHz). While I realize that some will complain that ‘clock-for-clock’ comparisons are not valid, I must emphasize that this is not meant to be an evaluation of which processor is ‘more efficient’ than another. It is also inevitable that some will want to point at specific results and claim the P4 is a bad design because of how it fares compared to PIII and Athlon processors at the same speed, but the same caveat applies. While 1.2GHz is fairly passe nowadays, it happens to be the ‘crossover’ point between the various architectures, so as many things can be ‘normalized’ as possible.
I have results from a number of other clock rates (from 800MHz thru 2.4GHz), but the amount of information at only 1.2GHz is fairly significant by itself. As it is, this will be a three part article. This first installment will focus on the processors themselves – their features and performance on CPU specific tests. The second installment will cover the memory and cache performance tests, while the third will look at system level tests.
My primary purpose here is to look at benchmarks, rather than to compare processors per se. It will be possible to come to some conclusions about the value of various features and implementations under specific situations, but a judgement of what is good or bad is probably best left to the individual user. The best processor for a given individual is not necessarily the one that scores higher on more benchmarks than another, but the one that performs best under the same type of tasks that the user will throw at it. What this means is that somehow we have to make a comparison of the benchmarks themselves to the types of resource usage that different applications will have. This should theoretically allow savvy users to choose the processor that satisfies their needs, rather than one that satisfies the assumptions of a reviewer.
This article focuses on CPU specific benchmarks. I’ve used three different benchmarks here – PCMark2002, SiSoft Sandra and SPEC CPU2000. For a background on PCMark2002, you can read the PCMark2002 – a First Look article on this site. Information on Sandra can be found on the SiSoft website (look in the Questions and Answers section under Benchmarks), and of course, information about SPEC can be found at http://www.spec.org/osg/cpu2000/#Info.
Discuss (One comment)