Example of memory ordering causing excitement in the real world

By: Jouni Osmala (josmala.delete@this.cc.hut.fi), July 17, 2015 12:18 am
Room: Moderated Discussions

> > > Given that option 1 is slow even when people get it right, you'd expect software engineers
> > > who care about performance at all to write option 2. However, option 2 is simply "is the input
> > > aligned? If so, use fast instruction for aligned read, else use slow instruction for unaligned
> > > read" - and I don't see why that burden can't be passed down to the hardware.
> >
> > Here's the problem. In the pipeline decision what to execute next happens several cycles
> > before the address is calculated, so you cannot really take advantage of random speed
> > ups on low cycle count instructions, like a load with L1 cache hit should be.
> > So well pipelined hardware on modern processes cannot really make that decision efficiently. Of course they
> > could assume its always aligned and reissue all the dependent
> > instructions when guess is wrong, which should
> > give multi cycle penalty on all unaligned accesses instead of that single cycle, and waste execution slots
> > from other instructions. And having a predictor is also pretty costly solution for minimal potential gains,
> > and in the end its still slower than just put everything through
> > slow path and try make that as fast as possible,
> > and assume every access is unaligned but doesn't cross cache-lines for scheduling purpose.
> >
>
> Of course hardware isn't perfect, and usually for good reasons. My argument is that these deficiencies
> (like no real world CPU offering sequential consistency) are just that - deficiencies that software
> has to live with because there are real world limits on the hardware, not hardware features that
> software should embrace. In a similar vein, it'd be nice if CPUs could offer zero-cost bounds
> checking (when accesses are in bounds - only pay a penalty if the bounds check fails); again,
> I understand why CPUs don't offer this, but it would still be nice to have.
>
> Equally, because these are things that would be nice to have, I can hope that a hardware
> designer has the sort of epiphany that (for example) enabled them to replace clustered message
> passing systems with NUMA, or that enabled Intel's SNB and later CPUs to have no penalty
> for unaligned accesses. Maybe one day, a designer will come up with an innovative way to
> have sequential consistency in a large system, or to get bounds checking at no penalty.

I would say that penalty for unaligned accesses is there but its eaten by aligned accesses too. Intel has pretty high L1Dcache latency for its size and that the cost of all the features Intel puts in its memory accesses.

Additional hardware is only "almost free" if its parallel to existing hardware and small, and doesn't require additional inputs from registers, and doesn't issue dependency between earlier pipeline stage and later pipeline stage, and you can fit it inside an existing execution unit.

I'd say that I'm no expert of physical limitations of hardware. Most problems in software peoples ideas are obvious from logical design. The common pitfalls for pure software people is lets force O(n^2) algorithm to have always 2n instead of n as input because it makes few things percentage point or two cheaper in software, or assumes earlier pipeline stage has same information as execute stage, or just adds more sequential hardware operations to a common operation.

Another thing that is forgotten is that hardware building blocks has to handle worst case situation, not average case situation.

I believe haswell made the best possible addition to speed up bounds checking. Increasing the branch unit from one to two. Having already compare and branch as fused operation means that at execution stage it doesn't take more than one additional operation.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Cyclone Macro-op fusionMaynard Handley2015/07/05 05:01 PM
  Cyclone Macro-op fusionGabriele Svelto2015/07/06 01:11 AM
    Cyclone Macro-op fusionMaynard Handley2015/07/06 09:25 AM
      Cyclone Macro-op fusionWilco2015/07/06 10:17 AM
        Cyclone Macro-op fusionMaynard Handley2015/07/06 11:07 AM
          Cyclone Macro-op fusionMaynard Handley2015/07/06 11:55 AM
            Cyclone Macro-op fusionGabriele Svelto2015/07/07 01:49 AM
      Power8 branch conversionSHK2015/07/06 11:41 AM
        Power8 branch conversionMaynard Handley2015/07/06 12:01 PM
          Power8 branch conversionMaynard Handley2015/07/06 12:09 PM
        Benefit of dynamic hammock predicationPaul A. Clayton2015/07/06 02:02 PM
          Benefit of dynamic hammock predicationMaynard Handley2015/07/06 02:34 PM
            Benefit of dynamic hammock predicationEduardoS2015/07/06 06:56 PM
            Benefit of dynamic hammock predicationPatrick Chase2015/07/06 09:30 PM
          Benefit of dynamic hammock predicationLinus Torvalds2015/07/06 02:59 PM
            Benefit of dynamic hammock predicationWilco2015/07/06 03:35 PM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/06 04:25 PM
                Benefit of dynamic hammock predicationanon2015/07/06 08:02 PM
            Benefit of dynamic hammock predicationPatrick Chase2015/07/06 09:31 PM
              Benefit of dynamic hammock predicationanon2015/07/06 10:02 PM
              Benefit of dynamic hammock predicationMichael S2015/07/07 12:07 AM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/07 08:23 AM
                Benefit of dynamic hammock predicationPatrick Chase2015/07/07 09:49 AM
                  Benefit of dynamic hammock predicationSHK2015/07/07 10:34 AM
                  Benefit of dynamic hammock predicationMaynard Handley2015/07/07 11:00 AM
                    Benefit of dynamic hammock predicationExophase2015/07/07 12:26 PM
                      Benefit of dynamic hammock predicationWilco2015/07/07 03:54 PM
                      Benefit of dynamic hammock predicationMaynard Handley2015/07/07 04:25 PM
                    Benefit of dynamic hammock predicationPatrick Chase2015/07/07 01:32 PM
                    Benefit of dynamic hammock predicationMichael S2015/07/08 12:57 AM
                  Benefit of dynamic hammock predicationSylvain Collange2015/07/08 05:30 AM
                    Benefit of dynamic hammock predicationMaynard Handley2015/07/08 08:46 AM
                      Benefit of dynamic hammock predicationSylvain Collange2015/07/08 09:32 AM
                        Benefit of dynamic hammock predicationMaynard Handley2015/07/08 09:57 AM
                        Sadly, skewed associativity has not been broadly adoptedPaul A. Clayton2015/07/08 01:00 PM
                        Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/08 07:23 PM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)David Kanter2015/07/09 06:20 AM
                            Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/09 09:04 AM
                              Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Tim McCaffrey2015/07/09 09:41 AM
                                Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 12:08 PM
                                  Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Sylvain Collange2015/07/10 01:03 AM
                              Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)David Kanter2015/07/09 03:15 PM
                                Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Maynard Handley2015/07/09 04:21 PM
                            Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 11:44 AM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)Patrick Chase2015/07/09 08:43 AM
                          Prediction quality of A8 and Intel (was Benefit of dynamic hammock predication)someone2015/07/10 04:43 AM
            Benefit of dynamic hammock predicationKonrad Schwarz2015/07/07 06:54 AM
            Benefit of dynamic hammock predicationdmcq2015/07/07 01:38 PM
              Benefit of dynamic hammock predicationLinus Torvalds2015/07/08 07:23 AM
                Benefit of dynamic hammock predicationdmcq2015/07/08 09:11 AM
                  Benefit of dynamic hammock predicationLinus Torvalds2015/07/08 12:38 PM
                    Example of memory ordering causing excitement in the real worldMark Roulo2015/07/08 03:54 PM
                      Example of memory ordering causing excitement in the real worlddmcq2015/07/09 03:37 AM
                        Example of memory ordering causing excitement in the real worldMark Roulo2015/07/09 07:24 AM
                          Example of memory ordering causing excitement in the real worlddmcq2015/07/09 09:09 AM
                          Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/09 09:12 AM
                            Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/09 11:14 AM
                              Example of memory ordering causing excitement in the real worlddmcq2015/07/09 03:24 PM
                                Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/10 12:07 PM
                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/10 02:49 PM
                                    Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/12 12:02 PM
                                      Example of memory ordering causing excitement in the real worlddmcq2015/07/13 04:03 AM
                                        Example of memory ordering causing excitement in the real worldMichael S2015/07/13 04:19 AM
                                        Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/14 12:21 PM
                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/14 01:54 PM
                                            Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/15 12:13 PM
                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/15 02:43 PM
                                                Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/16 11:41 AM
                                                  Example of memory ordering causing excitement in the real worldWilco2015/07/17 12:57 PM
                                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/17 03:29 PM
                                                      Example of memory ordering causing excitement in the real worlddmcq2015/07/18 03:07 AM
                                          Example of memory ordering causing excitement in the real worldJouni Osmala2015/07/16 02:52 AM
                                            Example of memory ordering causing excitement in the real worldSimon Farnsworth2015/07/16 12:05 PM
                                              Example of memory ordering causing excitement in the real worldJouni Osmala2015/07/17 12:18 AM
                                                ISA support for bound checkingMichael S2015/07/17 03:51 AM
                                                  ISA support for bound checkingEtienne2015/07/17 05:39 AM
                                                  MPX on SkylakeMark Roulo2015/07/17 07:19 AM
                                                    MPX on SkylakeMichael S2015/07/17 07:44 AM
                                                      MPX on SkylakeMark Roulo2015/07/17 08:03 AM
                                  Example of memory ordering causing excitement in the real worldanon2015/07/12 02:42 AM
                                    Example of memory ordering causing excitement in the real worlddmcq2015/07/12 09:27 AM
                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 10:24 AM
                                      Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 10:53 AM
                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 12:07 PM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/12 06:52 PM
                                            Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/13 10:05 AM
                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/13 11:20 AM
                                                Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/13 12:46 PM
                                                  Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/13 01:10 PM
                                                    Example of memory ordering causing excitement in the real worldanon2015/07/13 07:51 PM
                                                    Simple cores justifying weak ordering?Paul A. Clayton2015/07/14 08:01 AM
                                                      David's Paul A. Clayton2015/07/14 08:37 AM
                                                      David's 2015-07-12 10:34 post confirmed some of that (NT)Paul A. Clayton2015/07/14 08:39 AM
                                                    Example of memory ordering causing excitement in the real worldEtienne2015/07/17 12:15 AM
                                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/13 01:14 PM
                                                  Reality is even worseTerry Gray2015/07/13 05:05 PM
                                                    Reality is even worseKonrad Schwarz2015/07/20 11:13 PM
                                                  A simple CAS/MCAS semantics2015/07/15 11:54 AM
                                                    A simple CAS/MCAS semanticsLinus Torvalds2015/07/15 12:55 PM
                                                    A simple CAS/MCAS semanticsEduardoS2015/07/15 01:31 PM
                                                Example of memory ordering causing excitement in the real worldsylt2015/07/13 01:19 PM
                                                  Example of memory ordering causing excitement in the real worlddmcq2015/07/13 02:24 PM
                                                    Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 03:48 PM
                                                      Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/13 06:18 PM
                                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 06:57 PM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/13 08:01 PM
                                                    What high level languages do you have in mind?Mark Roulo2015/07/13 04:00 PM
                                                      What high level languages do you have in mind?dmcq2015/07/14 05:54 AM
                                                  Example of memory ordering causing excitement in the real worldWilco2015/07/13 03:36 PM
                                                    Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 03:56 PM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/13 05:43 PM
                                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/13 07:11 PM
                                                          Example of memory ordering causing excitement in the real worldanon2015/07/13 08:07 PM
                                      Example of memory ordering causing excitement in the real worldMaynard Handley2015/07/12 11:32 AM
                                        Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 12:35 PM
                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/12 12:38 PM
                                          Example of memory ordering causing excitement in the real worldEduardoS2015/07/12 05:47 PM
                                            Example of memory ordering causing excitement in the real worldNoSpammer2015/07/13 11:34 AM
                                              Example of memory ordering causing excitement in the real worldWilco2015/07/13 03:50 PM
                                                Example of memory ordering causing excitement in the real worldNoSpammer2015/07/14 01:02 PM
                                                  Example of memory ordering causing excitement in the real worldanon2015/07/14 08:37 PM
                                                    Example of memory ordering causing excitement in the real worldMichael S2015/07/15 12:19 AM
                                                      Example of memory ordering causing excitement in the real worldanon2015/07/15 01:26 AM
                                                        Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 06:05 AM
                                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/15 07:04 AM
                                                            Example of memory ordering causing excitement in the real worldMichael S2015/07/15 07:29 AM
                                                              Example of memory ordering causing excitement in the real worlddmcq2015/07/15 09:09 AM
                                                              Can string ops use microarchitectural state?Paul A. Clayton2015/07/16 08:33 AM
                                                                Can string ops use microarchitectural state?rwessel2015/07/16 12:00 PM
                                                                  Can string ops use microarchitectural state?Paul A. Clayton2015/07/16 05:16 PM
                                                                    Can string ops use microarchitectural state?rwessel2015/07/16 08:34 PM
                                                                      Can string ops use microarchitectural state?Michael S2015/07/17 04:39 AM
                                                                        Can string ops use microarchitectural state?rwessel2015/07/17 08:02 AM
                                                                Consider interrupt followed by task preemptionMichael S2015/07/16 03:31 PM
                                                                  Consider interrupt followed by task preemptionPaul A. Clayton2015/07/16 05:23 PM
                                                            Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 11:50 AM
                                                              Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 01:27 PM
                                                                Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 02:14 PM
                                                                  Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 04:42 PM
                                                      Example of memory ordering causing excitement in the real worldNoSpammer2015/07/15 01:44 AM
                                                    Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 01:17 PM
                                                      Example of memory ordering causing excitement in the real worldWilco2015/07/15 01:43 PM
                                                        Example of memory ordering causing excitement in the real worldanon2015/07/15 04:45 PM
                                                        Example of memory ordering causing excitement in the real worldLinus Torvalds2015/07/15 05:35 PM
                                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/16 01:46 AM
                                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/16 01:49 AM
                                        Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 06:15 AM
                                      Example of memory ordering causing excitement in the real worldanon2015/07/12 06:42 PM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/12 08:48 PM
                                        Example of memory ordering causing excitement in the real worldDavid Kanter2015/07/12 09:34 PM
                                          Example of memory ordering causing excitement in the real worldMichael S2015/07/13 12:51 AM
                                            Example of memory ordering causing excitement in the real worldDavid Kanter2015/07/13 07:31 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/13 05:05 AM
                                            Example of memory ordering causing excitement in the real worldGabriele Svelto2015/07/17 04:40 AM
                                          Example of memory ordering causing excitement in the real worlddmcq2015/07/13 07:09 AM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/13 06:46 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/14 03:12 AM
                                        Example of memory ordering causing excitement in the real worlddmcq2015/07/15 02:51 AM
                                          Example of memory ordering causing excitement in the real worldanon2015/07/15 03:38 AM
                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/15 05:21 AM
                                      Store queues in POWER/PowerPC processorsGabriele Svelto2015/07/16 12:55 AM
                                        Not sure about that conclusionDavid Kanter2015/07/16 06:34 AM
                                          The POWER4 had the same structureGabriele Svelto2015/07/17 04:20 AM
                                        Committing stores in program orderPaul A. Clayton2015/07/16 12:35 PM
                                          Committing stores in program orderMaynard Handley2015/07/17 11:52 AM
                                      Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/20 03:44 AM
                                        Example of memory ordering causing excitement in the real worldanon2015/07/20 06:29 AM
                                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/20 11:08 PM
                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/21 12:27 AM
                                            Example of memory ordering causing excitement in the real worldMichael S2015/07/21 02:11 AM
                                              Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 04:24 AM
                                            Example of memory ordering causing excitement in the real worldanon2015/07/21 05:22 AM
                                              Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 06:36 AM
                                                Example of memory ordering causing excitement in the real worldanon2015/07/21 07:25 AM
                                              Example of memory ordering causing excitement in the real worldanon2015/07/21 07:17 AM
                                                Example of memory ordering causing excitement in the real worldanon2015/07/21 08:50 AM
                                                Example of memory ordering causing excitement in the real worldGabriele Svelto2015/07/21 11:46 AM
                                                Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/21 11:35 PM
                                                  Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/22 12:37 AM
                                                  Example of memory ordering causing excitement in the real worldanon2015/07/23 02:17 AM
                                                    Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 03:59 AM
                                                      Example of memory ordering causing excitement in the real worldMichael S2015/07/23 06:31 AM
                                                        Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 02:38 PM
                                                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/23 10:44 PM
                                                            Example of memory ordering causing excitement in the real worlddmcq2015/07/24 12:08 AM
                                    Example of memory ordering causing excitement in the real worlddmcq2015/07/13 06:27 AM
                          Example of memory ordering causing excitement in the real worldKonrad Schwarz2015/07/16 04:11 AM
                      Example of memory ordering causing excitement in the real worldAnonymousse2015/07/16 04:07 AM
          Benefit of dynamic hammock predicationsomeotherdude2015/07/06 04:12 PM
            Benefit of dynamic hammock predicationMichael S2015/07/07 12:19 AM
          Benefit of dynamic hammock predicationPatrick Chase2015/07/06 09:20 PM
            SIMD predication is also helpful, of course (NT)Paul A. Clayton2015/07/07 06:45 AM
        Power8 branch conversionGabriele Svelto2015/07/07 01:50 AM
          isel works for storesPaul A. Clayton2015/07/07 06:43 AM
            isel works for storesPatrick Chase2015/07/07 10:08 AM
            isel DOES NOT work for storessomeotherdude2015/07/07 03:01 PM
              isel DOES NOT work for storesWilco2015/07/07 03:40 PM
              Yes it doesPatrick Chase2015/07/07 10:14 PM
                Yes it doesvvid2015/07/08 08:51 AM
            You don't need isel eitherGabriele Svelto2015/07/08 01:01 AM
              You don't need isel eitherPaul A. Clayton2015/07/08 03:46 AM
                You don't need isel eitherGabriele Svelto2015/07/08 04:22 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?