In Round 1, the professional memory diagnostics from Ultra-X were finalists, as was the surprisingly robust freeware utility called MemTest86. This time, there are five new contenders and an even more difficult set of tests.
The first benchmarks I would like to look at are the eTesting Labs (formerly Ziff-Davis Benchmark Operation, or ZDBOp) Winstone benchmarks, specifically Business Winstone 2001 and Content Creation Winstone 2001. These are amongst the most widely used performance measurement tools, primarily because they are free, run under Windows, and are easily installed and used. ZDBOp used to also offer a subcription publication, called “Benchmark Insider”, which provided quite a few details about the design and implementation of their benchmarks, but the subscription fee of $235/yr seems to have discouraged all but the most serious benchmarking professionals. Based upon discussions with a few other publications, it is likely that only the largest actually subscribed (and the Ziff-Davis Media publications likely received theirs for free).
I had intended my next article in this column to be a look at the Winstone benchmarks (which is in the works), however the news and reaction to the SysMark 2001 ‘bug’ caught my attention. Essentially, SysMark 2001 has an inherent problem that seemingly puts the Athlon XP, Athlon MP and Duron (w/Morgan core) processors at a disadvantage in the benchmark. It turns out that Windows Media Encoder 7, which is used in the benchmark, checks specifically for the CPUID string of ‘GenuineIntel’ in order to determine if it should use the SSE instruction path or not (just as we thought developers had learned their lesson about such things).
A look at how hard disks and system file configuration affects performance on a Windows system.
This is the first of what I intend to be a series of articles on benchmarking, benchmarking issues and benchmarks themselves. I will attempt to perform evaluations, talk with experts and manufacturers, and provide whatever information I can about the benchmarking industry and the various tools and techniques used, not only in computer publications, but within the industry itself. I don’t expect to uncover any subversive plots or present any shocking revelations, but I do expect to learn and present useful information and methods for testing and interpreting benchmark results from various products. I also hope that reviewers and industry professionals will take interest and provide their own views and feedback that I can incorporate into future articles.
A short analysis of the Intel roadmap updates for 2001, and what they might reveal about the ongoing battle with AMD.