Benchmark Profile
As with the Winstone benchmarks, I have begun to create a profile for SYSmark 2001. The data presented here is only a subset of the data I have collected, but my next article will include the complete results and compare them to the complete set of Winstone results. It is important to note that I’ve only collected data for Athlon and Duron processors (not XP) to this point, and only with one motherboard/memory combination. There is much more data to collect in order to completely validate the information here, including tests with Pentium III and Pentium 4 processors. Unfortunately, these tests are quite time consuming, so it will be an ongoing effort to get a complete set of data.
Once again, these tests were run on an Abit KT7A (KT133A chipset) using the ‘Optimized Defaults’ BIOS settings (no tweaks), an IBM 75GXP 45GB HDD and a Diamond V770 Ultra (TNT2) video card w/32MB SDRAM. Windows 2000 SP2 is the operating system, using the VIA 4-in-1 drivers. All other drivers are the default drivers in Windows 2000. The processors used were and unlocked Athlon and Duron to get a range of processor speeds without worrying about the effects of different steppings. All memory modules used were Crucial Technology PC133 SDRAM (CL2). These results provide an indication of how the benchmark responds to differences in processor speeds, FSB speeds, memory sizes and cache sizes.
This chart shows that both scenarios (Office Productivity and Internet Content Creation) both benefit from increases in processor speed, but they scale a little differently. ICC starts with a 26% improvement in the first jump, followed by increases of about 18% and 15%. OP gets improvements of around 24%, 15% and 11% – so the benefit falls off a bit faster with increased processor speed. The difference between the slowest and fastest processors is about 58% for OP, and 71% for ICC. I believe that this is evidence of the ‘overstatement’ of the effect of processor speed vs. a more real world scenario that includes OS overhead. More on this in the next article comparing the Winstone and SYSmark benchmarks.
This is the chart that I think is the most interesting of all – the effect of memory size on performance. Don’t be fooled by the dropoff with 512MB, as the scaling I used overemphasizes the difference in scores. Note that they are all within 4 points of each other – or about 3% – which is within the margin of error for the benchmark. What makes this interesting is that the theory I presented earlier seems to be validated with this chart. There is absolutely no gain in performance when adding memory! Since multiple applications are being run simultaneously, according to the White Paper referenced earlier, it is not possible that virtually no swapping is occuring at 128MB (considering OS overhead, data and application size – particularly for the ICC test). The only possible explanation is that swap activity is not being measured with this benchmark, as suggested previously.
This also makes the interpretation of of SYSmark 2001 results somewhat interesting. On a system with 128MB of memory, I believe the effects of the processor and FSB speed on system performance will be overstated when compared with other benchmarks that include OS overhead in their measurements. This, in turn, can mislead consumers who would be expecting a greater improvement in system performance with a particular component upgrade.
This chart shows the difference between 100MHz FSB and 133MHz FSB. As mentioned in the previous article, only two data points it is difficult to come to any real conclusions. In this case, both scenarios benefit by almost exactly the same amount – about 8%.
Now we can see the scaling with processor speed and FSB speed in the Office Productivity (top chart) and Internet Content Creation (bottom chart). It is fairly obvious with these charts that the faster FSB seems to improve the performance of the processor at higher clock rates. This should not be surprising for a benchmark that is measuring the effects of processor and memory speeds while minimizing the effects of OS overhead that relies on I/O speed. Remember, performance will always be limited by the slowest part of the system, so eliminating that from the measurements will overstate the effects of the faster parts for purposes of determining the real world performance. This may not seem like much, but it should become more apparent when we compare the Winstone and SYSmark benchmarks later on.
The two charts above provide a glimpse of what effect cache size has on the benchmark. The first chart once again shows that memory size has no effect on performance, but gives us a very good look what cache size means to this benchmark – about 18% in OP and 30% in ICC. Again, I believe that this overstates the effect of cache in real world usage.
The second chart shows that the scaling with processor speed is better with the Athlon in both benchmarks. This is consistent with what one would expect, as the hit ratio in cache will be higher for the larger cache, so the faster clock rates provide greater value with the larger cache. OS overhead would not be a big factor here, I believe, because this is a test that tends to focus on the processor/memory subsystem interaction with no reliance on I/O activity.
Pages: « Prev 1 2 3 4 5 6 7 8 Next »
Discuss (15 comments)