Example of memory ordering causing excitement in the real world

By: anon (anon.delete@this.anon.com), July 12, 2015 6:52 pm
Room: Moderated Discussions
Linus Torvalds (torvalds.delete@this.linux-foundation.org) on July 12, 2015 1:07 pm wrote:
> EduardoS (no.delete@this.spam.com) on July 12, 2015 11:53 am wrote:
> > >
> > > See? No memory barriers. No nothing. Just that same model of "load early and mark".
> >
> > As a programmer I think it is easy too, but nobody does it, not even x86.
>
> Actually, from what I can tell, that's pretty much exactly what modern big-core Intel CPU's
> are doing, it's just that write buffering is explicitly visible to software and thus reordering
> reads vs earlier local writes happens without that whole extra checking.
>
> Basically, delaying a store isn't considered "speculatively reordering" it.
> So no, x86 is not sequentially consistent, because of the delayed stores.
>
> Some people call it "x86-TSO".
>
> So the example I gave was designed to illustrate the point about re-ordering and how barriers
> make sense - and how they don't. Not so much designed to show what x86 does.
>
> The normal x86 model lets loads go ahead of local stores without any extra work (or put another
> way: the stores can be buffered over later loads). And that's actually the only really common
> case that you really absolutely need to re-order very aggressively for good performance.
>
> So x86 cheats and doesn't do full sequential ordering. It does what it does for the usual historical reasons:
> there was always a store buffer, so the whole "stores can be delayed" has been there since day one. Even back
> when there were no caches at all, the store buffer still meant that stores would be delayed, and that was "visible"
> both for DMA and for SMP (yes, some people - notably Sequent - did SMP on 80386s with no caches).
>
> So the x86 "delayed but in-order stores" is not some kind of smart design by superhuman minds that knew
> it was a good idea, it's a historical accident like pretty much all tech issues are. Then when caches
> got added, so as not to break anything (since there were applications that would break, even back then),
> everything else was done fairly strictly ordered. Which doesn't really say much, since the cores were
> in-order, the caches weren't horribly aggressive (I think they were originally blocking), and it was
> just all on a common frontside bus anyway. So there was a lot of inherent ordering there.
>
> With the P6 and OoO you had the first situation where Intel really almost could
> get some ordering issues, and they did in fact have a few SMP bugs.
>
> But, when pretty much everybody else says "we'll re-order anything against anything" (ie
> weak ordering) because they thought they were clever and had learnt from other peoples
> mistakes, x86 for historical reasons basically said "we'll only buffer writes".

Interestingly, Intel for a time was very hesitant to commit to their current memory consistency model by formalizing it in their ISA documents. IIRC this finally did happen around Core2(?) timeframe.

Presumably their cores were not implementing weaker consistency, but I'm sure there would have been questions over the issue of whether this stronger consistency would end up costing them significant performance. I guess they implemented memory disambiguation and load/load speculation around Core2 timeframe and decided that they can commit to strong load/load and store/store ordering because it will not hamper their performance significantly. (All this is purely conjecture on my part -- it's possible other reasons such as they found significant legacy code breakage, or that it may hurt them a bit but it would hurt competitors more, etc etc).
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Cyclone Macro-op fusionMaynard Handley2015/07/05 05:01 PM
  Cyclone Macro-op fusionGabriele Svelto2015/07/06 01:11 AM
    Cyclone Macro-op fusionMaynard Handley2015/07/06 09:25 AM
      Cyclone Macro-op fusionWilco2015/07/06 10:17 AM
        Cyclone Macro-op fusionMaynard Handley2015/07/06 11:07 AM
          Cyclone Macro-op fusionMaynard Handley2015/07/06 11:55 AM
            Cyclone Macro-op fusionGabriele Svelto2015/07/07 01:49 AM
      Power8 branch conversionSHK2015/07/06 11:41 AM
        Power8 branch conversionMaynard Handley2015/07/06 12:01 PM
          Power8 branch conversionMaynard Handley2015/07/06 12:09 PM
        Benefit of dynamic hammock predicationPaul A. Clayton2015/07/06 02:02 PM
          Benefit of dynamic hammock predicationMaynard Handley2015/07/06 02:34 PM
            Benefit of dynamic hammock predicationEduardoS2015/07/06 06:56 PM
            Benefit of dynamic hammock predicationPatrick Chase2015/07/06 09:30 PM
          Benefit of dynamic hammock predicationLinus Torvalds2015/07/06 02:59 PM
            Benefit of dynamic hammock predicationWilco2015/07/06 03:35 PM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/06 04:25 PM
                Benefit of dynamic hammock predicationanon2015/07/06 08:02 PM
            Benefit of dynamic hammock predicationPatrick Chase2015/07/06 09:31 PM
              Benefit of dynamic hammock predicationanon2015/07/06 10:02 PM
              Benefit of dynamic hammock predicationMichael S2015/07/07 12:07 AM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/07 08:23 AM
                Benefit of dynamic hammock predicationPatrick Chase2015/07/07 09:49 AM
                  Benefit of dynamic hammock predicationSHK2015/07/07 10:34 AM
                  Benefit of dynamic hammock predicationMaynard Handley2015/07/07 11:00 AM
                    Benefit of dynamic hammock predicationExophase2015/07/07 12:26 PM
                      Benefit of dynamic hammock predicationWilco2015/07/07 03:54 PM
                      Benefit of dynamic hammock predicationMaynard Handley2015/07/07 04:25 PM
                    Benefit of dynamic hammock predicationPatrick Chase2015/07/07 01:32 PM
                    Benefit of dynamic hammock predicationMichael S2015/07/08 12:57 AM
                  Benefit of dynamic hammock predicationSylvain Collange2015/07/08 05:30 AM
                    Benefit of dynamic hammock predicationMaynard Handley2015/07/08 08:46 AM
                      Benefit of dynamic hammock predicationSylvain Collange2015/07/08 09:32 AM
                        Benefit of dynamic hammock predicationMaynard Handley2015/07/08 09:57 AM
                        Sadly, skewed associativity has not been broadly adoptedPaul A. Clayton2015/07/08 01:00 PM
                        Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/08 07:23 PM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)David Kanter2015/07/09 06:20 AM
                            Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/09 09:04 AM
                              Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Tim McCaffrey2015/07/09 09:41 AM
                                Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 12:08 PM
                                  Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Sylvain Collange2015/07/10 01:03 AM
                              Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)David Kanter2015/07/09 03:15 PM
                                Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/09 04:21 PM
                            Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 11:44 AM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 08:43 AM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)someone2015/07/10 04:43 AM
            Benefit of dynamic hammock predicationKonrad Schwarz2015/07/07 06:54 AM
            Benefit of dynamic hammock predicationdmcq2015/07/07 01:38 PM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/08 07:23 AM
                Benefit of dynamic hammock predicationdmcq2015/07/08 09:11 AM
                  Benefit of dynamic hammock predicationLinus Torvalds2015/07/08 12:38 PM
                    Example of memory ordering causing excitement in the real worldMark Roulo2015/07/08 03:54 PM
                      Example of memory ordering causing excitement in the real worlddmcq2015/07/09 03:37 AM
                        Example of memory ordering causing excitement in the real worldMark Roulo2015/07/09 07:24 AM
                          Example of memory ordering causing excitement in the real worlddmcq2015/07/09 09:09 AM
                          Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/09 09:12 AM
                            Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/09 11:14 AM
                              Example of memory ordering causing excitement in the real worlddmcq2015/07/09 03:24 PM
                                Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/10 12:07 PM
                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/10 02:49 PM
                                    Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/12 12:02 PM
                                      Example of memory ordering causing excitement in the real worlddmcq2015/07/13 04:03 AM
                                        Example of memory ordering causing excitement in the real worldMichael S2015/07/13 04:19 AM
                                        Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/14 12:21 PM
                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/14 01:54 PM
                                            Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/15 12:13 PM
                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/15 02:43 PM
                                                Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/16 11:41 AM
                                                  Example of memory ordering causing excitement in the real worldWilco2015/07/17 12:57 PM
                                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/17 03:29 PM
                                                      Example of memory ordering causing excitement in the real worlddmcq2015/07/18 03:07 AM
                                          Example of memory ordering causing excitement in the real worldJouni Osmala2015/07/16 02:52 AM
                                            Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/16 12:05 PM
                                              Example of memory ordering causing excitement in the real worldJouni Osmala2015/07/17 12:18 AM
                                                ISA support for bound checkingMichael S2015/07/17 03:51 AM
                                                  ISA support for bound checkingEtienne2015/07/17 05:39 AM
                                                  MPX on SkylakeMark Roulo2015/07/17 07:19 AM
                                                    MPX on SkylakeMichael S2015/07/17 07:44 AM
                                                      MPX on SkylakeMark Roulo2015/07/17 08:03 AM
                                  Example of memory ordering causing excitement in the real worldanon2015/07/12 02:42 AM
                                    Example of memory ordering causing excitement in the real worlddmcq2015/07/12 09:27 AM
                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 10:24 AM
                                      Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 10:53 AM
                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 12:07 PM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/12 06:52 PM
                                            Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/13 10:05 AM
                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/13 11:20 AM
                                                Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/13 12:46 PM
                                                  Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/13 01:10 PM
                                                    Example of memory ordering causing excitement in the real worldanon2015/07/13 07:51 PM
                                                    Simple cores justifying weak ordering?Paul A. Clayton2015/07/14 08:01 AM
                                                      David's Paul A. Clayton2015/07/14 08:37 AM
                                                      David's 2015-07-12 10:34 post confirmed some of that (NT)Paul A. Clayton2015/07/14 08:39 AM
                                                    Example of memory ordering causing excitement in the real worldEtienne2015/07/17 12:15 AM
                                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/13 01:14 PM
                                                  Reality is even worseTerry Gray2015/07/13 05:05 PM
                                                    Reality is even worseKonrad Schwarz2015/07/20 11:13 PM
                                                  A simple CAS/MCAS semantics2015/07/15 11:54 AM
                                                    A simple CAS/MCAS semanticsLinus Torvalds2015/07/15 12:55 PM
                                                    A simple CAS/MCAS semanticsEduardoS2015/07/15 01:31 PM
                                                Example of memory ordering causing excitement in the real worldsylt2015/07/13 01:19 PM
                                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/13 02:24 PM
                                                    Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 03:48 PM
                                                      Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/13 06:18 PM
                                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 06:57 PM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/13 08:01 PM
                                                    What high level languages do you have in mind?Mark Roulo2015/07/13 04:00 PM
                                                      What high level languages do you have in mind?dmcq2015/07/14 05:54 AM
                                                  Example of memory ordering causing excitement in the real worldWilco2015/07/13 03:36 PM
                                                    Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 03:56 PM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/13 05:43 PM
                                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 07:11 PM
                                                          Example of memory ordering causing excitement in the real worldanon2015/07/13 08:07 PM
                                      Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/12 11:32 AM
                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 12:35 PM
                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 12:38 PM
                                          Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 05:47 PM
                                            Example of memory ordering causing excitement in the real worldNoSpammer2015/07/13 11:34 AM
                                              Example of memory ordering causing excitement in the real worldWilco2015/07/13 03:50 PM
                                                Example of memory ordering causing excitement in the real worldNoSpammer2015/07/14 01:02 PM
                                                  Example of memory ordering causing excitement in the real worldanon2015/07/14 08:37 PM
                                                    Example of memory ordering causing excitement in the real worldMichael S2015/07/15 12:19 AM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/15 01:26 AM
                                                        Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 06:05 AM
                                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/15 07:04 AM
                                                            Example of memory ordering causing excitement in the real worldMichael S2015/07/15 07:29 AM
                                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/15 09:09 AM
                                                              Can string ops use microarchitectural state?Paul A. Clayton2015/07/16 08:33 AM
                                                                Can string ops use microarchitectural state?rwessel2015/07/16 12:00 PM
                                                                  Can string ops use microarchitectural state?Paul A. Clayton2015/07/16 05:16 PM
                                                                    Can string ops use microarchitectural state?rwessel2015/07/16 08:34 PM
                                                                      Can string ops use microarchitectural state?Michael S2015/07/17 04:39 AM
                                                                        Can string ops use microarchitectural state?rwessel2015/07/17 08:02 AM
                                                                Consider interrupt followed by task preemptionMichael S2015/07/16 03:31 PM
                                                                  Consider interrupt followed by task preemptionPaul A. Clayton2015/07/16 05:23 PM
                                                            Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 11:50 AM
                                                              Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 01:27 PM
                                                                Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 02:14 PM
                                                                  Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 04:42 PM
                                                      Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 01:44 AM
                                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 01:17 PM
                                                      Example of memory ordering causing excitement in the real worldWilco2015/07/15 01:43 PM
                                                        Example of memory ordering causing excitement in the real worldanon2015/07/15 04:45 PM
                                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 05:35 PM
                                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/16 01:46 AM
                                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/16 01:49 AM
                                        Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 06:15 AM
                                      Example of memory ordering causing excitement in the real worldanon2015/07/12 06:42 PM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/12 08:48 PM
                                        Example of memory ordering causing excitement in the real worldDavid Kanter2015/07/12 09:34 PM
                                          Example of memory ordering causing excitement in the real worldMichael S2015/07/13 12:51 AM
                                            Example of memory ordering causing excitement in the real worldDavid Kanter2015/07/13 07:31 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/13 05:05 AM
                                            Example of memory ordering causing excitement in the real worldGabriele Svelto2015/07/17 04:40 AM
                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/13 07:09 AM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/13 06:46 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/14 03:12 AM
                                        Example of memory ordering causing excitement in the real worlddmcq2015/07/15 02:51 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/15 03:38 AM
                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/15 05:21 AM
                                      Store queues in POWER/PowerPC processorsGabriele Svelto2015/07/16 12:55 AM
                                        Not sure about that conclusionDavid Kanter2015/07/16 06:34 AM
                                          The POWER4 had the same structureGabriele Svelto2015/07/17 04:20 AM
                                        Committing stores in program orderPaul A. Clayton2015/07/16 12:35 PM
                                          Committing stores in program orderMaynard Handley2015/07/17 11:52 AM
                                      Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/20 03:44 AM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/20 06:29 AM
                                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/20 11:08 PM
                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/21 12:27 AM
                                            Example of memory ordering causing excitement in the real worldMichael S2015/07/21 02:11 AM
                                              Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 04:24 AM
                                            Example of memory ordering causing excitement in the real worldanon2015/07/21 05:22 AM
                                              Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 06:36 AM
                                                Example of memory ordering causing excitement in the real worldanon2015/07/21 07:25 AM
                                              Example of memory ordering causing excitement in the real worldanon2015/07/21 07:17 AM
                                                Example of memory ordering causing excitement in the real worldanon2015/07/21 08:50 AM
                                                Example of memory ordering causing excitement in the real worldGabriele Svelto2015/07/21 11:46 AM
                                                Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 11:35 PM
                                                  Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/22 12:37 AM
                                                  Example of memory ordering causing excitement in the real worldanon2015/07/23 02:17 AM
                                                    Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 03:59 AM
                                                      Example of memory ordering causing excitement in the real worldMichael S2015/07/23 06:31 AM
                                                        Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 02:38 PM
                                                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 10:44 PM
                                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/24 12:08 AM
                                    Example of memory ordering causing excitement in the real worlddmcq2015/07/13 06:27 AM
                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/16 04:11 AM
                      Example of memory ordering causing excitement in the real worldAnonymousse2015/07/16 04:07 AM
          Benefit of dynamic hammock predicationsomeotherdude2015/07/06 04:12 PM
            Benefit of dynamic hammock predicationMichael S2015/07/07 12:19 AM
          Benefit of dynamic hammock predicationPatrick Chase2015/07/06 09:20 PM
            SIMD predication is also helpful, of course (NT)Paul A. Clayton2015/07/07 06:45 AM
        Power8 branch conversionGabriele Svelto2015/07/07 01:50 AM
          isel works for storesPaul A. Clayton2015/07/07 06:43 AM
            isel works for storesPatrick Chase2015/07/07 10:08 AM
            isel DOES NOT work for storessomeotherdude2015/07/07 03:01 PM
              isel DOES NOT work for storesWilco2015/07/07 03:40 PM
              Yes it doesPatrick Chase2015/07/07 10:14 PM
                Yes it doesvvid2015/07/08 08:51 AM
            You don't need isel eitherGabriele Svelto2015/07/08 01:01 AM
              You don't need isel eitherPaul A. Clayton2015/07/08 03:46 AM
                You don't need isel eitherGabriele Svelto2015/07/08 04:22 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?