Benchmark Discussion and Summary
For this review, five benchmarks were used to evaluate the performance of the upcoming Bensley platform against current DP servers. Two of the benchmarks were rendering applications, one was a kernel of a financial problem, another was a financial application and the last was a Java-based commercial server benchmark. The relative improvement for each benchmark is shown below in Figure 6 and ranges from a little over 40% to slightly under 120%. The relative improvement was calculated as the execution time of the Nocona system divided by the execution time on the Bensley system, minus one (in the case of SPECjbb2005, it is simply the Bensley score divided by the Nocona score, which is equivalent since throughput is the inverse of execution time).
Figure 6 – Summary of Bensley Performance Relative to Nocona
Each of these benchmarks reflects a different sort of workload. The most relevant for the server market is clearly SPECjbb2005, while SunGard and Black-Scholes are good for measuring performance for certain financial applications. Cinebench and POV-Ray fall more into the workstation or Digital Content Creation category; but we are leery of the results. Rendering is an embarrassingly parallel application that is done on clusters; if a workload can efficiently be done on a cluster, then it should scale in a near linear fashion for a shared memory machine. Yet, we observed (and several other reviewers as well) that the scaling falls off after four threads. The most likely explanation is that the algorithms used in these applications were designed only to scale to a certain point. In the past, systems with four threads were the largest commonly sold; larger computers typically were proprietary and rather expensive. Hence it is natural that the authors might choose techniques that are most effective on systems with one to four threads.
These benchmarks supplement Intel’s performance estimates by providing realistic measurements. All companies, Intel included, make performance projections for highly tuned systems. Not all servers are so carefully configured, and so our realistic measurements help to elaborate on official claims and estimates. We found a better than 100% performance increase in SPECjbb2005 with two modestly and similarly configured systems. In light of that, Intel’s predictions for a 70-90% increase in performance for Java, OLTP and other commercial workloads seem eminently reasonable. Judging from our other benchmark results, the performance improvement is not just limited to commercial server workloads either. Ultimately, our results confirm that the Bensley platform is Intel’s first real step into the world of multiple core MPUs, and will provide substantial benefits to many users. As to how it compares with competing offerings, perhaps we will revisit the subject in Q1 next year.