David Kanter (dkanter@realworldtech.com) on 11/2/08 wrote:
---------------------------
>Howard Chu (hyc@symas.com) on 10/31/08 wrote:
>---------------------------
>>Given the flakiness of the tools, it would have been worthwhile to hack the codepath
>>selection in each of the programs-under-test, to force identical codepaths on both
>>platforms.
>
>I'm not entirely sure how I could do that without faking CPUID, which I'm sure
>is possible, but I don't know how to do it, and I suspect it might consume a lot of time.
>
>> Then you could assess just how badly the tools were lying. >As it is,
>>you have a variable propagating systematically through all >your results and no way to account for it.
>
>Yes and no. I trust the VTune results - I'm sure they are numerically accurate.

I remember VTune lying on me. It reported "64K Aliasing Conflicts" when in fact it was something else, most probably running out of store buffers.
I only used VTune a couple of times, so hitting one serious bug is a lot percentage-wise.