Peer review

Article: A Preview of Intel's Bensley Platform (Part II)
By: Bill Todd (billtodd.delete@this.metrocast.net), December 1, 2005 10:14 pm
Room: Moderated Discussions
David Kanter (dkanter@realworldtech.com) on 12/1/05 wrote:

...

David opened the door to an entirely reasonable question by divulging (as
>>well he should have) that Intel provided some of the >benchmarks he had used.
>
>There's providing and there is providing. Would using a version of Linpack supplied
>by Intel be influencing me?

If without any doubt whatsoever you would have used (or not used) Linpack anyway, then the answer is a definite "No".

If Intel's providing Linpack caused you to use it when you otherwise might have chosen something different, then the answer is a definite "Yes" - and the next question then becomes whether there's any reason to suspect that Linpack produced results more to Intel's liking than some other choice might have (i.e., whether the fact that you were influenced in any way affected the article's observations or whether the influence, real though it was, was irrelevant).

> It's the same source code as what Dongarra and his fellows provide...

I should hope so: otherwise, providing it would be out-right cheating, not just potentially influencing.

>
>That seems to me to be rather innocent then.

Whether it is innocent or not depends mostly upon whether the benchmark provided by Intel was likely to produce results considered more desirable by Intel than other benchmarks you might have chosen on your own likely would have.

>
>>Determining
>>the degree to which this might or might not have influenced >the article's conclusions
>>was not insulting - in fact, it was something which a truly >experienced author would
>>have addressed explicitly (eliminating the need for the >question) because it's so important.
>
>Why don't you tell me how you would have done this?

I'm in no way suggesting that I personally would have done a better job - just that an author with a lot of experience doing this kind of thing would have.

I think that disclosing the
>source of the benchmark and explaining the nature of the benchmark ought to be sufficient.

Then you are somewhat naive. Just by observing how companies like HP (and, to be fair, Sun is another fine example) play benchmarks in their own PR one can understand how easily they can be abused - even inadvertently. Full disclosure (which you did pretty well at) is only the first step: when any *potential* for bias exists (as is the case when you elect to use benchmarks provided by a vendor rather than select all of them completely on your own), addressing that possibility explicitly such that it can be ruled out is a good idea - and failing that it is entirely reasonable to expect questions about the issue.

Not that I'm under the delusion that most technical articles do any better a job in such areas than yours did, mind you: it seemed well done in any relative sense even if it fell a bit short in an absolute sense in this particular area.

- bill
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Bensley Platform Preview (Part II) OnlineDavid Kanter2005/11/29 01:45 AM
  Bensley Platform Preview (Part II) OnlineTemp2005/11/29 06:25 AM
    Bensley Platform Preview (Part II) OnlineDavid Kanter2005/11/29 11:55 AM
      Bensley Platform Preview (Part II) OnlineTemp2005/11/29 02:29 PM
        Bensley Platform Preview (Part II) Onlinerwessel2005/11/29 02:53 PM
    Bensley Platform Preview (Part II) OnlineDean Kent2005/11/29 12:01 PM
      Bensley Platform Preview (Part II) OnlineWilliam Campbell2005/11/29 12:48 PM
        Bensley Platform Preview (Part II) OnlineDavid Kanter2005/11/29 01:37 PM
          Well said! (NT)savantu2005/11/29 01:44 PM
          Peer reviewWilliam Campbell2005/11/29 04:12 PM
            To clarify intentWilliam Campbell2005/11/29 04:19 PM
            Peer reviewDavid Kanter2005/11/29 04:21 PM
              Peer reviewWilliam Campbell2005/11/29 06:13 PM
                Peer reviewnick2005/11/29 11:09 PM
                  Peer reviewWilliam Campbell2005/11/30 12:39 AM
                    Peer reviewDavid Kanter2005/11/30 01:21 AM
                Peer reviewDavid Kanter2005/11/29 11:25 PM
                  Yes please (NT)William Campbell2005/11/30 12:28 AM
                    Yes please (NT)David Kanter2005/11/30 06:19 PM
                      Thank youWilliam Campbell2005/11/30 08:51 PM
                        Thank youDavid Kanter2005/11/30 10:29 PM
            Peer reviewDean Kent2005/11/29 07:12 PM
              Peer reviewWilliam Campbell2005/11/29 07:50 PM
                Peer reviewDean Kent2005/11/30 05:16 AM
                  Peer reviewWilliam Campbell2005/11/30 08:49 PM
                    Peer reviewTemp2005/12/01 03:02 AM
                      Peer reviewWilliam Campbell2005/12/01 04:54 AM
                        Peer reviewTemp2005/12/01 05:11 AM
                  Peer reviewTemp2005/12/01 03:03 AM
                    Peer reviewDean Kent2005/12/01 07:55 AM
                      Peer reviewBill Todd2005/12/01 08:26 PM
                        Peer reviewDavid Kanter2005/12/01 09:52 PM
                          Peer reviewBill Todd2005/12/01 10:14 PM
                            Peer reviewDavid Kanter2005/12/01 11:04 PM
                              Peer reviewBill Todd2005/12/02 12:13 AM
                            Peer reviewDean Kent2005/12/02 07:02 AM
                              You lost this one.Ray2005/12/02 11:54 AM
                                You lost.tecate2005/12/02 02:55 PM
                                  I second that (NT)savantu2005/12/02 03:22 PM
                                  I wasn't in the game.Ray2005/12/02 04:19 PM
                                    I wasn't in the game.Dean Kent2005/12/02 10:20 PM
                                  You lost.Bill Todd2005/12/02 05:28 PM
                                    You lost.Anonymous2005/12/02 08:27 PM
                                      You lost.Bill Todd2005/12/02 08:56 PM
                                        You lost.Dean Kent2005/12/02 10:37 PM
                                          You lost.Bill Todd2005/12/03 12:08 AM
                                            All about the contextDavid Kanter2005/12/03 02:27 PM
                                              All about the contextBill Todd2005/12/03 02:51 PM
                                                All about the contextDavid Kanter2005/12/03 04:29 PM
                                    You lost.Ray2005/12/02 09:15 PM
                                      You lost.Bill Todd2005/12/02 10:00 PM
                                        You lost.Ray2005/12/02 11:09 PM
                                        You lost.anonymous2005/12/03 02:42 AM
                                          You lost.Bill Todd2005/12/03 02:45 PM
                                            Well...David Kanter2005/12/03 03:51 PM
                                            You lost.Ray2005/12/03 05:54 PM
                                              Bill is a self loathing AmericanNIKOLAS2005/12/03 06:25 PM
                                                Bill is a self loathing AmericanBill Todd2005/12/03 09:40 PM
                                                  Bill is a self loathing AmericanBill Todd2005/12/03 09:48 PM
                                                Bill is a self loathing AmericanDavid Kanter2005/12/03 09:48 PM
                                                  Bill is a self loathing AmericanBill Todd2005/12/03 11:17 PM
                                                    Bill is a self loathing AmericanDavid Kanter2005/12/04 12:37 AM
                                                      Bill is a self loathing AmericanBill Todd2005/12/04 01:19 AM
                                                        This whole thread is a symptom...Dean Kent2005/12/04 09:43 AM
                                                          This whole thread is a symptom...tecate2005/12/04 01:17 PM
                                                            This whole thread is a symptom...mas2005/12/04 02:02 PM
                                                              This whole thread is a symptom...tecate2005/12/05 06:21 AM
                                                          This whole thread is a symptom...tecate2005/12/04 01:18 PM
                                                          ...Temp2005/12/04 03:38 PM
                                                            ...Dean Kent2005/12/04 05:25 PM
                                                              Once more, alasTemp2005/12/05 02:23 AM
                                                                Once more, alasDean Kent2005/12/05 08:23 AM
                                                                  ByeTemp2005/12/05 10:47 AM
                                                                  Once more, alasBill Todd2005/12/05 10:58 AM
                                                              Sungard as a benchmarkTemp2005/12/05 03:42 AM
                                                                Sungard as a benchmarkDean Kent2005/12/05 10:06 AM
                                                                Sungard as a benchmarkDavid Kanter2005/12/05 08:08 PM
                                                                  Sungard as a benchmarkTemp2005/12/06 01:45 AM
                                                                    More info about SungardTemp2005/12/06 03:20 PM
                                                                      More info about SungardDavid Kanter2005/12/06 04:25 PM
                                                                        More info about SungardTemp2005/12/07 12:40 AM
                                                                        More info about SungardDean Kent2005/12/07 07:52 AM
                                                                      More info about SungardDean Kent2005/12/06 07:22 PM
                                                          This whole thread is a symptom...Bill Todd2005/12/04 09:31 PM
                                                            This whole thread is a symptom...Dean Kent2005/12/04 09:51 PM
                                              You lost.Bill Todd2005/12/03 11:14 PM
                                                You lost.Ray2005/12/04 01:06 AM
                                                  You lost.Bill Todd2005/12/04 01:54 AM
                                                    Enough with the politics... (NT)David Kanter2005/12/04 03:41 AM
                                            You lost.anonymous2005/12/04 04:03 AM
                                              Well Said! (NT)Anonymous2005/12/04 04:48 AM
                                              You lost.savantu2005/12/04 06:47 AM
                                              You lost.Bill Todd2005/12/04 09:39 PM
                                                You lost.anonymous2005/12/05 02:51 AM
                                You lost this one.Dean Kent2005/12/02 09:41 PM
                                  You lost this one.Leonov2005/12/03 12:55 AM
                                    You lost this one.tecate2005/12/03 05:27 AM
                                      You lost this one.Leonov2005/12/03 06:33 AM
                                        You lost this one.savantu2005/12/03 10:19 AM
                                          You lost this one.Leonov2005/12/03 12:19 PM
                                        For god sake.Anonymous2005/12/04 04:28 AM
                                          It's sadsav2005/12/04 06:43 AM
                                            It's sadmas2005/12/04 07:09 AM
                                              It's sadMichael S2005/12/04 07:33 AM
                                              PerfectNo one you'd know2005/12/04 10:52 AM
                                                Perfectmas2005/12/04 12:32 PM
                                                  PerfectDean Kent2005/12/04 12:50 PM
                                                    Perfectmas2005/12/04 01:16 PM
                                                      PerfectDean Kent2005/12/04 04:22 PM
                                                        Posts deleted, topic not open for discussionDavid Kanter2005/12/05 02:05 PM
                                                          Posts deleted, topic not open for discussionKeith Fiske2005/12/05 05:03 PM
                                              This will not be toleratedDavid Kanter2005/12/04 04:32 PM
                                          For god sake.Leonov2005/12/05 07:10 AM
                                            Back on track...Dean Kent2005/12/05 12:35 PM
                                              Back on track...Leonov2005/12/06 03:08 AM
                                  You lost this one.Temp2005/12/03 04:16 AM
                        Peer reviewDean Kent2005/12/02 06:22 AM
                          Peer reviewTemp2005/12/02 12:01 PM
      Bensley Platform Preview (Part II) Onlinean2005/11/29 01:17 PM
        Bensley Platform Preview (Part II) OnlineDavid Kanter2005/11/29 02:17 PM
          Bensley Platform Preview (Part II) Onlinean2005/11/30 07:52 AM
            Bensley Platform Preview (Part II) OnlineDavid Kanter2005/11/30 10:42 PM
        Bensley Platform Preview (Part II) OnlineDean Kent2005/11/29 04:11 PM
        Bensley Platform Preview (Part II) Onlineanonymous2005/11/29 05:38 PM
          It's calledWilliam Campbell2005/11/29 06:17 PM
      Bensley Platform Preview (Part II) OnlineTemp2005/11/29 02:41 PM
        Bensley Platform Preview (Part II) OnlineDavid Kanter2005/11/29 03:02 PM
        Bensley Platform Preview (Part II) OnlineDean Kent2005/11/29 07:41 PM
  2 small nitpicksan2005/11/29 02:03 PM
    2 small nitpicksDaniel Bizó2005/11/29 03:27 PM
      2 small nitpicksan2005/11/30 07:40 AM
        2 small nitpicksDaniel Bizó2005/11/30 11:17 AM
          2 small nitpicksan2005/11/30 12:30 PM
            2 small nitpicksDavid Kanter2005/11/30 02:32 PM
              2 small nitpicksan2005/11/30 02:49 PM
  Minor Comment about CineBenchRakesh Malik2005/11/29 02:22 PM
  Bensley Platform Preview (Part II) OnlinePiedPiper2005/11/29 08:04 PM
    Bensley Platform Preview (Part II) OnlinePiedPiper2005/11/29 08:08 PM
      Bensley Platform Preview (Part II) OnlineDavid Kanter2005/11/30 02:05 AM
        Bensley Platform Preview (Part II) OnlinePiedPiper2005/11/30 07:58 PM
          Bensley Platform Preview (Part II) OnlineDavid Kanter2005/12/01 01:45 AM
  Why no 64-bit tests?PiedPiper2005/11/29 08:37 PM
    Why no 64-bit tests?David Kanter2005/11/30 02:07 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?