The SunGard Adaptiv Credit Risk (ACR) benchmark, which was used in our analysis of Dempsey last year, has now been ported to 64 bits. This benchmark was provided by Intel and unfortunately, the binaries are still not publicly available.
ACR is an application from SunGard that uses a proprietary Monte Carlo engine to evaluate a hypothetical portfolio of assets. The benchmark portfolio consists of 1428 deals, primarily interest rate swaps and foreign exchange forwards; the portfolio is examined in conjunction with 29 price factors and 19 risk factors. Monte Carlo techniques are quite amenable to parallelization, since they usually partition problems into independent workloads. As a result, the benchmark should scale relatively well.
Unlike the rendering benchmarks, SunGard ACR has built in threading control. The user may set the number of threads, so that both performance and scaling can be observed. When a multithreading aware OS is used, preference will be giving towards physical processors. In this situation, we can very clearly see the performance gain from multithreading for Dempsey, as it is the speed up from 4 to 8 threads.
Figure 6 – SunGard ACR Performance
While some have objected that any benchmark provided by a vendor cannot possibly be unbiased, SunGard ACR is an application, not a toy benchmark. As a general rule of thumb, applications are somewhat less susceptible to tampering than other more artificial benchmarks. Perhaps the most ironic part of this review is that in the SunGard benchmark, which was provided by Intel, Dempsey is just a shade behind Woodcrest. This can almost certainly be chalked up to Dempsey’s multithreading, which brings the execution time down from 763 seconds to a little over 500. Going from 1 to 8 threads reduces the execution time by 4.7x for Dempsey, another instance of super linear scaling. Despite that, Woodcrest is not far behind, with run times dropping by a factor of 3.9x, nearly perfect scaling.
The standard deviations for the Woodcrest system, going from 1 thread to 4 were 1.26, 1.65, and 4.77 seconds. The Dempsey system had standard deviations of 2.66, 8.89, 2.91 and 3.97 seconds. Most of these variations are small relative to the absolute scores, but the Woodcrest system running with 4 threads had roughly a 10% standard deviation, which could be improved. However, Woodcrest’s performance lead was significant by any measure.