Conclusion and Future Directions
The bottom line is that the SPECpower committee has done an excellent job of both putting out two key deliverables: the most comprehensive framework for measuring power and performance and creating an implementation of that framework that targets server-side Java workloads with graduated utilization. In the long term, the methodology that SPEC has established will probably prove to be the most essential element of their work, as it will shape the way other benchmarks are formulated. However, the benchmark itself will have the most influence on the industry in the short term. Ultimately the goal of SPECpower is to help the industry reduce energy consumption and as the famous saying goes “If you cannot measure it, you cannot improve it.” SPECpower will provide a target for engineers who design servers, motherboards, power supplies and MPU, and will reward those vendors who produce more efficient products. To truly be effective, OEMs must participate in SPECpower and consumers must be able to trust the results enough to incorporate them into their decision making process.
That brings up one of the more interesting points regarding SPECpower: future evolution of power efficiency benchmarks. As the SPEC website clearly explains, SPECpower_ssj2008 is only intended to be representative of server side Java applications – not webservers, file servers, HPC or transactional or analytic databases. One of the future goals for SPECpower is to expand the portfolio so to speak, and incorporate a greater number of workloads and more form factors. For instance, the Graphics and Workstation Performance Group has already announced that they are working to adapt SPECpower to support the relevant workstation applications.
It is unlikely that SPEC will try and address database workloads, as those tend to be the province of the Transaction Processing Council (TPC). TPC has an even longer gestation period for their benchmarks than SPEC (TPC-C is 15 years old and still going – positively ancient by any standard), and while we can hope for a quick adoption of a power efficiency framework, it seems unlikely as TPC-E was just finalized and still has few submissions, and TPC-DS has yet to be completed. A version of SPECpower that can handle virtualization is much more likely, especially since virtualization is one of the key technologies that IT departments will use in the future to increase utilization which increases efficiency. Many server OEMs are touting blade servers as more efficient computing platforms; a SPECpower methodology that can be applied to blade servers would be a valuable tool indeed for IT purchasers (especially if it can establish comparisons to rackmount or pedestal servers) and another candidate for future development. Ultimately, this is the first step along a path towards more power efficient system and MPU design and there is still quite a bit more work to look forward to.