Benefit of dynamic hammock predication

By: Patrick Chase (patrickjchase.delete@this.gmai.com), July 7, 2015 10:49 am
Room: Moderated Discussions
Linus Torvalds (torvalds.delete@this.linux-foundation.org) on July 7, 2015 9:23 am wrote:
> Patrick Chase (patrickjchase.delete@this.gmai.com) on July 6, 2015 10:31 pm wrote:
> > And yet you yourself have (very effectively, with real data) made the argument that
> > cmov seldom pays on x86.
>
> Absolutely. I think cmov a often a bad idea, because it leaves those data dependencies.
> And because it's often a bad idea, it's probably under-utilized in some cases (and also
> probably over-utilized in other cases).

Back when I was mentoring people who did a lot of DSP-ish coding I saw a common pattern: There would inevitably come a time when cmov/select was the right solution for a performance issue, so I would show them the appropriate idioms to convince the compiler to emit it (or intrinsic, or asm directive). Most of them would then go batsh*t crazy and use selects in all sorts of inappropriate places. Modern branch predictors are pretty good, and over-utilization ends up being the bigger problem in my experience.

> And that's part of my point - I think it would be interesting if hardware turned it
> into a predicted move, exactly to remove the data dependencies when there is a strong
> reason to believe it's the right thing to do. That's exactly the kind of information the
> CPU branch predictor already has (well, most of them do - not just predicting which way
> a branch goes, but also how likely it is).
>
> So hardware has the potential to offer the best of best worlds: keep the data
> dependency when it makes dynamic sense, and break it when it is likely the right thing
> to do.
>
> That's the kind of choice you can make at a hardware level. Doing it at the
> software level is really really problematic, for all the reasons outlined.
>
> See my argument?

Ah, understood. A couple thoughts:

1. In all but the most trivial cases the HW would also have to issue and execute all of the instructions on the not-predicted side of the branch. You might end up needing to do a hyperblock cache to mitigate impacts to I-fetch (i.e. an L0 I$ that stores pre-predicated hyperblocks, along the lines of but even less efficient than P4's superblock cache [*]).

2. Do existing branch predictors really have that information? I thought that even modern ones still tended to keep a fairly small amount of per-context state (where "context" is a catch-all for the branch address, histories, etc).

[*] It was never a trace cache, Intel's terminology notwithstanding. P4's L1 I$ contained single-entry, multi-exit EBBs, so the correct term is "superblock". Traces are multi-entry, multi-exit.

> That said, I also have to say that
>
> (a) I think cmov on x86 has improved. It used to have pretty bad latencies, afaik
> they've improved. So you still do have the data dependencies, but for many cases it
> probably doesn't matter that much.

Better but not free. 2 uops in Haswell/Broadwell.

> (b) there are clearly pretty big gotchas with using predictors too, and it may well be
> the case that it's not worth it. I wouldn't be surprised if this has been simulated, and
> real hw architects have come to the conclusion that the mispredicts just kill you.
>
> (c) since people and compilers have been taught to try to avoid cmov for well-predicted
> stuff, and some of those judgments are probably quite correct, the existing use may well
> be skewed enough towards "unpredictable" that the upsides are even smaller.
>
> So I'm certainly not claiming it's a no-brainer. I just think it would be interesting,
> and potentially something that hardware could do better (and it would allow software
> to maybe do better too, by making cmov more generically useful).

Yeah, agreed. The thing that I wrestle with here is that the easy-to-implement approach doesn't cover all that many real-world use cases IMO, while the general solution appears likely to be so expensive that it might be better to spend those gates on better predictors.

< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Cyclone Macro-op fusionMaynard Handley2015/07/05 06:01 PM
  Cyclone Macro-op fusionGabriele Svelto2015/07/06 02:11 AM
    Cyclone Macro-op fusionMaynard Handley2015/07/06 10:25 AM
      Cyclone Macro-op fusionWilco2015/07/06 11:17 AM
        Cyclone Macro-op fusionMaynard Handley2015/07/06 12:07 PM
          Cyclone Macro-op fusionMaynard Handley2015/07/06 12:55 PM
            Cyclone Macro-op fusionGabriele Svelto2015/07/07 02:49 AM
      Power8 branch conversionSHK2015/07/06 12:41 PM
        Power8 branch conversionMaynard Handley2015/07/06 01:01 PM
          Power8 branch conversionMaynard Handley2015/07/06 01:09 PM
        Benefit of dynamic hammock predicationPaul A. Clayton2015/07/06 03:02 PM
          Benefit of dynamic hammock predicationMaynard Handley2015/07/06 03:34 PM
            Benefit of dynamic hammock predicationEduardoS2015/07/06 07:56 PM
            Benefit of dynamic hammock predicationPatrick Chase2015/07/06 10:30 PM
          Benefit of dynamic hammock predicationLinus Torvalds2015/07/06 03:59 PM
            Benefit of dynamic hammock predicationWilco2015/07/06 04:35 PM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/06 05:25 PM
                Benefit of dynamic hammock predicationanon2015/07/06 09:02 PM
            Benefit of dynamic hammock predicationPatrick Chase2015/07/06 10:31 PM
              Benefit of dynamic hammock predicationanon2015/07/06 11:02 PM
              Benefit of dynamic hammock predicationMichael S2015/07/07 01:07 AM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/07 09:23 AM
                Benefit of dynamic hammock predicationPatrick Chase2015/07/07 10:49 AM
                  Benefit of dynamic hammock predicationSHK2015/07/07 11:34 AM
                  Benefit of dynamic hammock predicationMaynard Handley2015/07/07 12:00 PM
                    Benefit of dynamic hammock predicationExophase2015/07/07 01:26 PM
                      Benefit of dynamic hammock predicationWilco2015/07/07 04:54 PM
                      Benefit of dynamic hammock predicationMaynard Handley2015/07/07 05:25 PM
                    Benefit of dynamic hammock predicationPatrick Chase2015/07/07 02:32 PM
                    Benefit of dynamic hammock predicationMichael S2015/07/08 01:57 AM
                  Benefit of dynamic hammock predicationSylvain Collange2015/07/08 06:30 AM
                    Benefit of dynamic hammock predicationMaynard Handley2015/07/08 09:46 AM
                      Benefit of dynamic hammock predicationSylvain Collange2015/07/08 10:32 AM
                        Benefit of dynamic hammock predicationMaynard Handley2015/07/08 10:57 AM
                        Sadly, skewed associativity has not been broadly adoptedPaul A. Clayton2015/07/08 02:00 PM
                        Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/08 08:23 PM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)David Kanter2015/07/09 07:20 AM
                            Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/09 10:04 AM
                              Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Tim McCaffrey2015/07/09 10:41 AM
                                Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 01:08 PM
                                  Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Sylvain Collange2015/07/10 02:03 AM
                              Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)David Kanter2015/07/09 04:15 PM
                                Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/09 05:21 PM
                            Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 12:44 PM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 09:43 AM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)someone2015/07/10 05:43 AM
            Benefit of dynamic hammock predicationKonrad Schwarz2015/07/07 07:54 AM
            Benefit of dynamic hammock predicationdmcq2015/07/07 02:38 PM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/08 08:23 AM
                Benefit of dynamic hammock predicationdmcq2015/07/08 10:11 AM
                  Benefit of dynamic hammock predicationLinus Torvalds2015/07/08 01:38 PM
                    Example of memory ordering causing excitement in the real worldMark Roulo2015/07/08 04:54 PM
                      Example of memory ordering causing excitement in the real worlddmcq2015/07/09 04:37 AM
                        Example of memory ordering causing excitement in the real worldMark Roulo2015/07/09 08:24 AM
                          Example of memory ordering causing excitement in the real worlddmcq2015/07/09 10:09 AM
                          Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/09 10:12 AM
                            Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/09 12:14 PM
                              Example of memory ordering causing excitement in the real worlddmcq2015/07/09 04:24 PM
                                Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/10 01:07 PM
                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/10 03:49 PM
                                    Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/12 01:02 PM
                                      Example of memory ordering causing excitement in the real worlddmcq2015/07/13 05:03 AM
                                        Example of memory ordering causing excitement in the real worldMichael S2015/07/13 05:19 AM
                                        Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/14 01:21 PM
                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/14 02:54 PM
                                            Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/15 01:13 PM
                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/15 03:43 PM
                                                Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/16 12:41 PM
                                                  Example of memory ordering causing excitement in the real worldWilco2015/07/17 01:57 PM
                                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/17 04:29 PM
                                                      Example of memory ordering causing excitement in the real worlddmcq2015/07/18 04:07 AM
                                          Example of memory ordering causing excitement in the real worldJouni Osmala2015/07/16 03:52 AM
                                            Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/16 01:05 PM
                                              Example of memory ordering causing excitement in the real worldJouni Osmala2015/07/17 01:18 AM
                                                ISA support for bound checkingMichael S2015/07/17 04:51 AM
                                                  ISA support for bound checkingEtienne2015/07/17 06:39 AM
                                                  MPX on SkylakeMark Roulo2015/07/17 08:19 AM
                                                    MPX on SkylakeMichael S2015/07/17 08:44 AM
                                                      MPX on SkylakeMark Roulo2015/07/17 09:03 AM
                                  Example of memory ordering causing excitement in the real worldanon2015/07/12 03:42 AM
                                    Example of memory ordering causing excitement in the real worlddmcq2015/07/12 10:27 AM
                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 11:24 AM
                                      Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 11:53 AM
                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 01:07 PM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/12 07:52 PM
                                            Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/13 11:05 AM
                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/13 12:20 PM
                                                Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/13 01:46 PM
                                                  Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/13 02:10 PM
                                                    Example of memory ordering causing excitement in the real worldanon2015/07/13 08:51 PM
                                                    Simple cores justifying weak ordering?Paul A. Clayton2015/07/14 09:01 AM
                                                      David's Paul A. Clayton2015/07/14 09:37 AM
                                                      David's 2015-07-12 10:34 post confirmed some of that (NT)Paul A. Clayton2015/07/14 09:39 AM
                                                    Example of memory ordering causing excitement in the real worldEtienne2015/07/17 01:15 AM
                                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/13 02:14 PM
                                                  Reality is even worseTerry Gray2015/07/13 06:05 PM
                                                    Reality is even worseKonrad Schwarz2015/07/21 12:13 AM
                                                  A simple CAS/MCAS semantics2015/07/15 12:54 PM
                                                    A simple CAS/MCAS semanticsLinus Torvalds2015/07/15 01:55 PM
                                                    A simple CAS/MCAS semanticsEduardoS2015/07/15 02:31 PM
                                                Example of memory ordering causing excitement in the real worldsylt2015/07/13 02:19 PM
                                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/13 03:24 PM
                                                    Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 04:48 PM
                                                      Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/13 07:18 PM
                                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 07:57 PM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/13 09:01 PM
                                                    What high level languages do you have in mind?Mark Roulo2015/07/13 05:00 PM
                                                      What high level languages do you have in mind?dmcq2015/07/14 06:54 AM
                                                  Example of memory ordering causing excitement in the real worldWilco2015/07/13 04:36 PM
                                                    Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 04:56 PM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/13 06:43 PM
                                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 08:11 PM
                                                          Example of memory ordering causing excitement in the real worldanon2015/07/13 09:07 PM
                                      Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/12 12:32 PM
                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 01:35 PM
                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 01:38 PM
                                          Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 06:47 PM
                                            Example of memory ordering causing excitement in the real worldNoSpammer2015/07/13 12:34 PM
                                              Example of memory ordering causing excitement in the real worldWilco2015/07/13 04:50 PM
                                                Example of memory ordering causing excitement in the real worldNoSpammer2015/07/14 02:02 PM
                                                  Example of memory ordering causing excitement in the real worldanon2015/07/14 09:37 PM
                                                    Example of memory ordering causing excitement in the real worldMichael S2015/07/15 01:19 AM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/15 02:26 AM
                                                        Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 07:05 AM
                                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/15 08:04 AM
                                                            Example of memory ordering causing excitement in the real worldMichael S2015/07/15 08:29 AM
                                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/15 10:09 AM
                                                              Can string ops use microarchitectural state?Paul A. Clayton2015/07/16 09:33 AM
                                                                Can string ops use microarchitectural state?rwessel2015/07/16 01:00 PM
                                                                  Can string ops use microarchitectural state?Paul A. Clayton2015/07/16 06:16 PM
                                                                    Can string ops use microarchitectural state?rwessel2015/07/16 09:34 PM
                                                                      Can string ops use microarchitectural state?Michael S2015/07/17 05:39 AM
                                                                        Can string ops use microarchitectural state?rwessel2015/07/17 09:02 AM
                                                                Consider interrupt followed by task preemptionMichael S2015/07/16 04:31 PM
                                                                  Consider interrupt followed by task preemptionPaul A. Clayton2015/07/16 06:23 PM
                                                            Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 12:50 PM
                                                              Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 02:27 PM
                                                                Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 03:14 PM
                                                                  Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 05:42 PM
                                                      Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 02:44 AM
                                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 02:17 PM
                                                      Example of memory ordering causing excitement in the real worldWilco2015/07/15 02:43 PM
                                                        Example of memory ordering causing excitement in the real worldanon2015/07/15 05:45 PM
                                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 06:35 PM
                                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/16 02:46 AM
                                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/16 02:49 AM
                                        Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 07:15 AM
                                      Example of memory ordering causing excitement in the real worldanon2015/07/12 07:42 PM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/12 09:48 PM
                                        Example of memory ordering causing excitement in the real worldDavid Kanter2015/07/12 10:34 PM
                                          Example of memory ordering causing excitement in the real worldMichael S2015/07/13 01:51 AM
                                            Example of memory ordering causing excitement in the real worldDavid Kanter2015/07/13 08:31 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/13 06:05 AM
                                            Example of memory ordering causing excitement in the real worldGabriele Svelto2015/07/17 05:40 AM
                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/13 08:09 AM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/13 07:46 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/14 04:12 AM
                                        Example of memory ordering causing excitement in the real worlddmcq2015/07/15 03:51 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/15 04:38 AM
                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/15 06:21 AM
                                      Store queues in POWER/PowerPC processorsGabriele Svelto2015/07/16 01:55 AM
                                        Not sure about that conclusionDavid Kanter2015/07/16 07:34 AM
                                          The POWER4 had the same structureGabriele Svelto2015/07/17 05:20 AM
                                        Committing stores in program orderPaul A. Clayton2015/07/16 01:35 PM
                                          Committing stores in program orderMaynard Handley2015/07/17 12:52 PM
                                      Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/20 04:44 AM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/20 07:29 AM
                                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 12:08 AM
                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/21 01:27 AM
                                            Example of memory ordering causing excitement in the real worldMichael S2015/07/21 03:11 AM
                                              Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 05:24 AM
                                            Example of memory ordering causing excitement in the real worldanon2015/07/21 06:22 AM
                                              Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 07:36 AM
                                                Example of memory ordering causing excitement in the real worldanon2015/07/21 08:25 AM
                                              Example of memory ordering causing excitement in the real worldanon2015/07/21 08:17 AM
                                                Example of memory ordering causing excitement in the real worldanon2015/07/21 09:50 AM
                                                Example of memory ordering causing excitement in the real worldGabriele Svelto2015/07/21 12:46 PM
                                                Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/22 12:35 AM
                                                  Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/22 01:37 AM
                                                  Example of memory ordering causing excitement in the real worldanon2015/07/23 03:17 AM
                                                    Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 04:59 AM
                                                      Example of memory ordering causing excitement in the real worldMichael S2015/07/23 07:31 AM
                                                        Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 03:38 PM
                                                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 11:44 PM
                                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/24 01:08 AM
                                    Example of memory ordering causing excitement in the real worlddmcq2015/07/13 07:27 AM
                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/16 05:11 AM
                      Example of memory ordering causing excitement in the real worldAnonymousse2015/07/16 05:07 AM
          Benefit of dynamic hammock predicationsomeotherdude2015/07/06 05:12 PM
            Benefit of dynamic hammock predicationMichael S2015/07/07 01:19 AM
          Benefit of dynamic hammock predicationPatrick Chase2015/07/06 10:20 PM
            SIMD predication is also helpful, of course (NT)Paul A. Clayton2015/07/07 07:45 AM
        Power8 branch conversionGabriele Svelto2015/07/07 02:50 AM
          isel works for storesPaul A. Clayton2015/07/07 07:43 AM
            isel works for storesPatrick Chase2015/07/07 11:08 AM
            isel DOES NOT work for storessomeotherdude2015/07/07 04:01 PM
              isel DOES NOT work for storesWilco2015/07/07 04:40 PM
              Yes it doesPatrick Chase2015/07/07 11:14 PM
                Yes it doesvvid2015/07/08 09:51 AM
            You don't need isel eitherGabriele Svelto2015/07/08 02:01 AM
              You don't need isel eitherPaul A. Clayton2015/07/08 04:46 AM
                You don't need isel eitherGabriele Svelto2015/07/08 05:22 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?