In order to get a better idea of the sensitivity these benchmarks have to various system changes, I have begun to create a profile for each. The data presented here is only a subset of the data I have collected, and therefore doesn’t provide a truly complete picture. In addition, there is much more data to collect before I can spot any real trends or anomalies – but for this first look it should be sufficient.
The idea here is to determine how much of an effect changing the memory size, processor speed, FSB and L2 cache size would have on the Winstone benchmarks. This was done by using unlocked Athlon and Duron processors to get a range of processor speeds without worrying about the effects of different steppings (please refer to the System Disclosure for details). Using this data, I can see how each benchmark reacts to such changes, which may be useful when comparing different components (or may not…).
As this graph clearly shows, both benchmarks benefit almost equally by increased processor speed. Business Winstone benefits just slightly more, with a 50.5% improvement from 600MHz to 1.2GHz (Content Creation Winstone gets about 47.5%), probably because it relies a bit less on I/O and memory. Remember, both of these benchmarks focus on those operations where users will wait, so these results apply mostly to those who are ‘power users’ of these office applications. But, it does show that the benchmarks are a reasonably good test of the effect of processor speed, and even at 1.2GHz there doesn’t seem to be any significant flattening of the curve yet.
This chart shows the effect of memory size on the two benchmarks. Obviously, the applications in Business Winstone have a smaller working set size, and are therefore less dependent upon memory. Beyond 256MB, there is little benefit for Business Winstone, but Content Creation Winstone seems to gain some even at 384MB. This points out the importance of understanding where your bottlenecks are. If, for example, one is comparing two processors using only 128MB of memory, the results are going to not reflect the true processor differences, and anything greater than 384MB won’t provide any real benefit. The difference between 384MB and 512MB is well within the 3% margin of error on both benchmarks.
This chart simply shows the difference between 100MHz FSB and 133MHz FSB. With only two data points it is difficult to come to any real conclusions, but we can see that Content Creation Winstone seems to get more benefit from the faster FSB than Business Winstone does. It would be interesting to see the effects of a 166MHz FSB, but the equipment used in this test would not allow it. This is one place where additional data is definitely needed.
Based upon the above information, we can reasonably assume that the relative scores will be better in Content Creation Winstone when using 133MHz FSB – but does it mean the scores will actually scale better? We can find out using the next set of charts…
It is apparent that going from 100MHz to 133MHZ FSB gives the Content Creation Winstone a greater boost than it does Business Winstone – but the processor scaling is not affected. In other words, Business Winstone still gains more percentage wise (25% vs. 23%), even at 133MHz FSB.
OK – so what about cache size? Using a Duron (Spitfire core) instead of an Athlon (Tbird core), we have limited the difference only to the cache size. The following two charts show the results…
The first chart shows that the additional cache of the Athlon is more important than memory size to the results for Business Winstone. Note that the Athlon at 128MB provides essentially the same performance as the Duron with 512MB. This is not the case for Content Creation Winstone, where memory size provides a greater boost than the additional cache. Also note that the Duron scores flatten out – or even drop – after 384MB in both Business and Content Creation Winstone.
The second chart shows that the scaling with processor speed is just slightly better with the Athlon in both benchmarks. Therefore, it might be possible that cache size becomes more important as processor speed increases, though not by a huge amount. Obviously, additional data using faster processor speeds and even different processors would be necessary to confirm this. In other words, is this a function of the specific components being tested or a function of how the applications use resources. This points out how difficult it can be to come to accurate conclusions without sufficient data points.
Be the first to discuss this article!