By: Howard Chu (email@example.com), October 31, 2008 1:39 am
Room: Moderated Discussions
Given the flakiness of the tools, it would have been worthwhile to hack the codepath selection in each of the programs-under-test, to force identical codepaths on both platforms. Then you could assess just how badly the tools were lying. As it is, you have a variable propagating systematically through all your results and no way to account for it.