Lies, Damn Lies, and Benchmarks
With regard to DRDRAM, there are powerful business interests currently lined up against each other. One side, led by Intel and Rambus Inc., wants to promote the widespread adoption of Rambus memories in personal computers, while the opposing side (which apparently includes most memory manufacturers) would like to see this process prevented or at the very least slowed down. A propaganda war has already started and both sides will continue to release a flurry of benchmarks, white papers, and endorsements from “independent” market analysts in their favor.
Do the early adopters of Rambus-based PCs get value for their money, or simply get expensive memory with high clock rates but little to show for it? To a large extent that depends on how you use your computer. My personal opinion is that for the vast majority of users the evolutionary path (PC100 to PC133 to DDR) will be both cheaper and offer better performance for conventionally architected PCs (i.e. chipset based systems with powerful graphics cards incorporating large, high bandwidth local memories). However, I will present a pair of benchmark examples, one in favour of Rambus and one against Rambus, and dissect them to show what critical observers should look for and consider in weighing such evidence to make up their own mind.
The Enemy Within
One famous example is the set of Microsoft Office benchmarks attributed to a presentation at the September 1999 Intel Developer Forum (IDF) by Jay Bell, a senior fellow at Dell Computer. Considering how intimately aligned Dell Computer is to Intel’s interests, the results were surprisingly negative for Rambus. As widely reported by a particular analyst, the Dell benchmarks reportedly compared a 440BX chipset based system using PC100 SDRAM against an 820 chipset based system employing 800 Mbps DRDRAM. Both PCs were outfitted with identical 500 MHz processors. The Rambus-based PC was described as being 17% slower on PowerPoint 2000, 44% slower on Excel 2000, and 15% slower on Word 2000.
What can we make of these results? For starters, the 15% and 17% figures might be considered plausible if you imagine large personal productivity applications spend most of their time making extensive traversals of linked data structures too large to fit into the cache hierarchy. However, the 44% worse performance on Excel is impossible to explain strictly on the basis of differences in read latency. For most programs the performance difference should be about 5% worse for Rambus (as shown by my simple model). An asymptotic worst case of 20 to 25% lower performance might be seen for programs that are entirely pointer chasing exercises. This should be a warning flag that the Rambus based system was operating far below expected performance levels. The 820 chipset has a great deal of flexibility in setting parameters for power management and throttling strategies for the DRDRAMs under its care. The 44% worse result for Rambus strongly suggests that the 820 based PC was configured in an extremely conservative manner, and performance was strongly impacted by memory throttling and premature retirement of ACTIVE state DRDRAMs into the B-pool. As such it places the validity of the other two benchmark results into doubt. Buyer beware!
Be the first to discuss this article!