TPC-H is not a good benchmark

By: David Kanter (dkanter.delete@this.realworldtech.com), September 28, 2011 5:02 pm
Room: Moderated Discussions
>Its funny how when IBM shows a significantly higher benchmark result, like on SPEC
>CPU2006 or even SAP SD 2-Tier, certain folks here say *wow*, this is an amazing
>CPU

Yes, because SPECcpu2006 is the standard for CPU performance and SAP SD 2-tier is probably the best server benchmark out there (real application, not under control of a HW vendor, many results, etc.).

>and dont even question the fact that maybe IBM has cracked the benchmark yet
>here, SPARC T4 does incredibly well on TPC-H, an >independently audited benchmark,

Independent auditing is meaningless for TPC-H. Some components of SPECcpu2006 are 'cracked', but you can just ignore those and look at GCC if you want. Or you can assume that everyone will crack them reasonably well and that it all nets out at the end.

>much more real world benchmark (than spec cpu will ever >be), and where IBM and HP
>have recently published results so therefore Oracle must >have cracked the benchmark?

TPC-H is about as 'real world' as the illegitimate love child of the Easter Bunny, Santa Claus and the Wicked Witch of the West. The queries are 20 years out of date, the data set is ridiculous.

For data warehousing, TPC-H is utterly unreflective of modern workloads.

David
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
On the UltraSPARC T4Gabriele Svelto09/27/11 01:32 PM
  "perceptron-based" based branch predictorMark Roulo09/27/11 02:17 PM
  On the UltraSPARC T4DaveC09/27/11 02:22 PM
    On the UltraSPARC T4Gabriele Svelto09/27/11 03:28 PM
      On the UltraSPARC T4DaveC09/27/11 03:59 PM
    TPC-H benchmark resultMichael S09/27/11 03:44 PM
      TPC-H benchmark resultJesper Frimann09/28/11 01:43 AM
        TPC-H benchmark resultMichael S09/28/11 02:39 AM
          TPC-H benchmark resultJesper Frimann09/28/11 03:49 AM
            TPC-H benchmark resultGabriele Svelto09/28/11 09:44 AM
        TPC-H benchmark resultPhil09/28/11 05:56 AM
          TPC-H benchmark resultMichael S09/28/11 06:23 AM
          TPC-H benchmark resultsomeone09/28/11 06:54 AM
          TPC-H benchmark resultJesper Frimann09/28/11 07:39 AM
            cracking benchmarksanon09/28/11 02:10 PM
              cracking benchmarksJesper Frimann09/28/11 09:39 PM
          TPC-H is not a good benchmarkDavid Kanter09/28/11 05:02 PM
        TPC-H benchmark resultsomeone09/28/11 06:50 AM
          TPC-H benchmark resultMichael S09/28/11 07:11 AM
          TPC-H benchmark result - the high priced spreadblaine09/28/11 07:22 PM
            TPC-H benchmark result - the high priced spreadJesper Frimann09/28/11 09:41 PM
              TPC-H benchmark result - the high priced spread-SPARC T4 still comes out ahead!Phil09/29/11 12:06 AM
                TPC-H benchmark result - the high priced spread-SPARC T4 still comes out ahead!Jesper Frimann09/29/11 01:49 AM
                  TPC-H benchmark result - the high priced spread-SPARC T4 still comes out ahead!Phil09/29/11 02:13 AM
                    TPC-H benchmark result - the high priced spread-SPARC T4 still comes out ahead!Jesper Frimann09/29/11 03:24 AM
        TPC-H benchmark resultAC10/10/11 01:11 AM
          TPC-H benchmark resultJesper Frimann10/10/11 02:24 AM
            But woulnd't that make TPC-H meaningless? (NT)Daniel Bizo10/11/11 12:29 AM
              not very meaningful alreadyMichael S10/11/11 01:09 AM
                OLTP vs. analytic DBMSDavid Kanter10/11/11 02:44 PM
      software or hardware sp4mysf09/29/11 06:14 AM
    On the UltraSPARC T4Thu Nguyen09/27/11 04:40 PM
    On the UltraSPARC T4Brett09/27/11 08:21 PM
      On the UltraSPARC T4David Kanter09/27/11 08:33 PM
        On the UltraSPARC T4anon09/27/11 08:56 PM
          TERA MTAMark Roulo09/28/11 08:36 AM
            TERA MTAanon09/28/11 03:11 PM
              TERA MTAMichael S09/29/11 12:58 PM
                TERA MTAanon09/29/11 01:46 PM
            TERA MTABrett09/29/11 02:33 PM
              Are you sure?Michael S09/29/11 03:19 PM
                Are you sure?Brett09/29/11 05:31 PM
                  I trust The Register morePaul A. Clayton09/30/11 10:15 AM
              TERA MTAmpx09/30/11 05:33 AM
    On the UltraSPARC T4anon09/27/11 08:28 PM
      On the UltraSPARC T4Phil09/28/11 12:23 AM
        On the UltraSPARC T4David Kanter09/28/11 12:57 AM
          On the UltraSPARC T4Gabriele Svelto09/28/11 09:22 AM
            Database clusteringDavid Kanter09/28/11 04:55 PM
              Database clusteringPhil09/29/11 12:15 AM
                Database clusteringDavid Kanter09/29/11 09:12 AM
                  Database clusteringAlex Fatkulin09/29/11 11:14 AM
                    Database clusteringDavid Kanter09/29/11 09:11 PM
                      Database clusteringAnon09/30/11 12:32 AM
                        Database clusteringDavid Kanter09/30/11 01:23 AM
                          Database clusteringmpx10/05/11 01:12 AM
                        Database clusteringJesper Frimann09/30/11 02:14 AM
                          Database clusteringAnon09/30/11 10:14 PM
                            Database clusteringJesper Frimann10/01/11 04:24 AM
                              Database clusteringAnon10/01/11 05:27 PM
                                Real system level : )David Kanter10/02/11 06:59 PM
                                  Real system level : )Anon10/05/11 11:36 PM
                              Database clusteringanon10/02/11 05:34 PM
                                Database clusteringJesper Frimann10/03/11 01:56 AM
                                  Database clusteringanon10/03/11 08:55 AM
                                    Database clusteringanon10/03/11 11:41 AM
                                      Database clusteringJesper Frimann10/03/11 02:24 PM
                                        Google...EduardoS10/03/11 03:24 PM
                                          Google...Aaron Spink10/04/11 01:02 AM
                                        Database clusteringanon10/03/11 04:04 PM
                                          Database clusteringJesper Frimann10/04/11 02:42 AM
                                            Database clusteringanon10/04/11 08:13 AM
                                              Database clusteringEduardoS10/04/11 08:29 AM
                                                Database clusteringanon10/04/11 08:45 AM
                                              Database clusteringbakaneko10/04/11 09:11 AM
                                              Database clusteringGabriele Svelto10/04/11 10:39 AM
                                                Database clusteringanon10/04/11 10:51 AM
                                                  Database clusteringGabriele Svelto10/04/11 01:27 PM
                                                    Database clusteringanon10/04/11 01:39 PM
                                              Database clusteringJesper Frimann10/04/11 11:26 AM
                                                Database clusteringanon10/04/11 01:37 PM
                              Database clusteringanonymous10/02/11 10:27 PM
                            Database clusteringmpx10/01/11 04:40 AM
                      Database clusteringAlex Fatkulin09/30/11 05:42 AM
                        Database clusteringJesper Frimann09/30/11 06:19 AM
                          Database clusteringAlex Fatkulin09/30/11 06:33 AM
                            Database clusteringJesper Frimann09/30/11 08:44 AM
                              Database clusteringAlex Fatkulin09/30/11 09:01 AM
                                Database clusteringJesper Frimann09/30/11 09:55 AM
                                  Database clusteringAlex Fatkulin09/30/11 10:07 AM
                                    Database clusteringJesper Frimann09/30/11 11:11 AM
                                      Database clusteringAlex Fatkulin09/30/11 11:33 AM
                                        Database clusteringJesper Frimann09/30/11 03:00 PM
                                          Database clusteringanon10/01/11 09:18 AM
                                      Database clusteringMichael S10/01/11 10:33 AM
                                        Database clusteringJesper Frimann10/01/11 10:22 PM
                                          Database clusteringJesper Frimann10/12/11 11:14 PM
                                            Warning: Bad pun (no content)Paul A. Clayton10/12/11 11:32 PM
                                              Good one *8^) (NT)Jesper Frimann10/12/11 11:56 PM
                              Database clusteringAlex Fatkulin10/04/11 08:09 AM
                                Database clusteringJesper Frimann10/04/11 11:35 AM
                                  Database clusteringGabriele Svelto10/04/11 01:40 PM
                                    Database clusteringDavid Kanter10/04/11 02:45 PM
                                      Database clusteringJesper Frimann10/05/11 12:09 AM
                            Database clusteringEduardoS09/30/11 02:13 PM
                              Database clusteringAlex Fatkulin10/04/11 05:56 AM
                          Database clusteringmpx09/30/11 10:39 AM
                            Database clusteringEduardoS09/30/11 02:07 PM
                              Database clusteringAlex Fatkulin10/04/11 06:01 AM
                                Database clusteringmpx10/05/11 01:02 AM
                            Database clusteringJesper Frimann10/01/11 01:32 AM
                    Database clusteringmpx09/30/11 12:20 AM
                      Database clusteringEduardoS09/30/11 02:09 PM
        TPC-C kingMichael S09/28/11 02:31 AM
        On the UltraSPARC T4someone09/28/11 11:23 AM
  Flash vs. disk on SPECjEnterprise2010?Paul A. Clayton09/27/11 03:29 PM
    Flash vs. disk on SPECjEnterprise2010?Gabriele Svelto09/27/11 10:23 PM
      Flash vs. disk on SPECjEnterprise2010?Jesper Frimann09/28/11 12:04 AM
    Flash vs. disk on SPECjEnterprise2010?Phil09/28/11 12:32 AM
      Are you trolling?David Kanter09/28/11 01:07 AM
        memory bandwidth clarification etc..Michael S09/28/11 02:23 AM
          memory bandwidth clarification etc..Jesper Frimann09/28/11 04:13 AM
            memory bandwidth clarification etc..Phil09/28/11 06:06 AM
              memory bandwidth clarification etc..Jesper Frimann09/28/11 09:25 AM
        There is one (another one)...EduardoS09/28/11 02:54 AM
          There is one (another one)...Michael S09/28/11 03:59 AM
        Are you trolling? Yes, trolling for substantiation!!Phil09/28/11 05:47 AM
          Are you trolling? Yes, trolling for substantiation!!Jesper Frimann09/28/11 11:01 AM
            Are you trolling? Yes, trolling for substantiation!!Phil09/29/11 12:26 AM
              Are you trolling? Yes, trolling for substantiation!!Jesper Frimann09/29/11 02:27 AM
                Are you trolling? Yes, trolling for substantiation!!Phil09/29/11 04:46 AM
          Are you trolling? Yes, trolling for substantiation!!David Kanter09/28/11 05:15 PM
            Are you trolling? Yes, trolling for substantiation!!Phil09/29/11 12:53 AM
              Are you trolling? Yes, trolling for substantiation!!Jesper Frimann09/29/11 07:43 AM
          PricingDavid Kanter09/28/11 05:28 PM
            PricingJesper Frimann09/28/11 09:56 PM
            PricingMichael S09/29/11 01:42 PM
        T4 better at cryptographic workloads?Paul A. Clayton09/28/11 05:49 AM
          T4 better at cryptographic workloads?Gabriele Svelto09/28/11 09:36 AM
            T4 better at cryptographic workloads?Jesper Frimann09/28/11 11:10 AM
              T4 better at cryptographic workloads?Gabriele Svelto09/28/11 12:56 PM
                T4 better at cryptographic workloads?Max09/29/11 06:53 AM
            Instruction- vs. coprocessor-based extensionsPaul A. Clayton09/28/11 04:48 PM
            Memory versioning?anon10/02/11 08:48 PM
              Memory versioning?David Kanter10/02/11 09:31 PM
                Memory versioning--not just for TM?Paul A. Clayton10/09/11 02:37 PM
                  Memory versioning--not just for TM?Gabriele Svelto10/09/11 10:11 PM
                    Memory versioning--not just for TM?David Kanter10/11/11 03:15 PM
          Point well takenDavid Kanter09/28/11 05:18 PM
          T4 better at cryptographic workloads?EduardoS09/28/11 05:36 PM
        Are you trolling?Gabriele Svelto09/28/11 09:49 AM
        An addendum on I/ODavid Kanter09/28/11 11:29 AM
          An addendum on I/OPhil01/27/12 08:19 AM
            An addendum on I/Oanon01/27/12 11:04 AM
            An addendum on I/ODavid Kanter01/27/12 03:47 PM
              An addendum on I/OKira01/27/12 04:23 PM
                Meant POWER7 (NT)David Kanter01/27/12 06:55 PM
            An addendum on I/OJesper Frimann01/29/12 07:58 AM
        Are you trolling?anon09/28/11 01:03 PM
      Flash vs. disk on SPECjEnterprise2010?mpx09/29/11 01:05 AM
  Well done!David Kanter09/27/11 07:10 PM
    Well done!Gabriele Svelto09/27/11 10:39 PM
  On the UltraSPARC T4mpx09/29/11 12:56 AM
  How much memory it supports?EduardoS10/11/11 01:37 AM
    How much memory it supports?Michael S10/11/11 04:14 AM
  Big power miss on T4?David Kanter07/16/12 09:24 PM
    Big power miss on T4?Gabriele Svelto07/17/12 11:26 PM
      Big power miss on T4?David Kanter07/19/12 04:04 PM
        Big power miss on T4?Kira07/19/12 05:44 PM
          Big power miss on T4?EduardoS07/19/12 06:19 PM
            Big power miss on T4?Kira07/19/12 09:37 PM
              Big power miss on T4?David Kanter07/19/12 10:15 PM
                 T4->T5blaine07/21/12 08:39 AM
              Big power miss on T4?EduardoS07/20/12 08:18 AM
            Big power miss on T4?mpx07/20/12 01:26 AM
              Big power miss on T4?EduardoS07/20/12 08:26 AM
        Big power miss on T4?Gabriele Svelto07/21/12 12:38 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell blue?