Avoiding ping pong

By: Maynard Handley (name99.delete@this.name99.org), December 9, 2014 11:33 am
Room: Moderated Discussions
Linus Torvalds (torvalds.delete@this.linux-foundation.org) on December 8, 2014 8:08 pm wrote:
> Jouni Osmala (josmala.delete@this.cc.hut.fi) on December 8, 2014 3:47 pm wrote:
> >
> > There is enough scaling left to turn somewhat better than current high end xeons to client
> > CPU:s
>
> Maybe. And probably not. Given the choice between 16 cores and 4, I suspect most people
> will take 4, and prefer more cache, graphics, and integrated networking etc.
>
> > Especially when stacked dram becomes standard and L3 cache becomes redundant due to
> > lower latency on main memory, and there would be enormous bandwidth available to CPU:s.
>
> Again, unlikely. Especially with all those cores. You want the L3 as a synchronization point for all your
> theoretical parallel programs that would otherwise have totally unacceptable synchronization overheads.
>
> Stacked RAM may help throughput, but it won't be running at CPU frequencies,
> and latency of communication between cores is pretty primary.
>
> Of course, that all assumes that the parallel solutions exist at all, which is still an
> open question (aka "not solved in the last few decades, probably not solvable at all")

Linus, might I respectfully suggest that you are blinded by two points:

(a) The space you work in strives for the absolute maximum performance possible. But a consequence of this is that parallel for you IS really hard, when your life is a constant stream of trying to reason about RCU on different architectures with different memory model, trying to figure out just how aggressively you can reorder some instructions to gain an extra .1% of throughput, how to create new lock-free data structures in the face of the limited guarantees different HW provides, etc.

But that is NOT parallel programming for most programmers. They are/will be using canned data structures and algorithms, and compiler and language support. Yes, this route may leave 10 or 15% of possible performance on the table. But if it allows the code to run 2x faster on a quad-core CPU, that's still progress.

(b) You are probably unrealistic about the speed at which change does (or does not) happen as you move from smaller more controlled environments (CPU) to larger (OS frameworks and APIs) to larger (compilers, changes to existing languages) to largest of all (new languages, which need to be learned, evangelized, have their infrastructure written, have their texbooks written, have to move into the college curriculum).

There may well be severe limits to what we can do if we insist on writing every parallel program in K&R C with pthreads. But we are slowly fumbling our way to better abstractions. Blocks/lambdas/futures are still only a few years old, and where they have been retrofitted to existing languages the edges are still pretty obvious (horribly so in the case of C++, just ugly in the case of Objective C or C#).
Relying on mutability as an important organizing principle for structuring, understanding, and performance, is even more leading edge. Actors and most functional programming concepts are even on the radar of the C-derived languages.

Fifty years ago, stacks and pointers were a leading edge concept, and the mainstream programming language (Fortran) provided no mechanisms that relied on them --- so no recursion, no sophisticated data structures. Today these concepts are taken for granted --- stacks are just invisible in the background of how function calls work, pointers have become invisible except for those working in C and C++.

Thirty years ago it was common (REALLY COMMON) practice to write code with a mess of globals. That was just how things were done by most people --- create this huge pool of globals at the top of a file and then have every function rummage around in it, rather than passing explicit variables/structures/objects between functions. I began programming at the tail-end of this, so I've no idea exactly what the driving force of this was --- maybe it was considered easier to debug, or more convenient (because you didn't have to alter the prototypes of a chain of functions in order to pass in a new parameter), or maybe it was considered (and actually was?) faster because function parameter marshaling was slow on those CPUs?
Point is anybody who sees this today would be horrified by that style of code; but it took 20+ years for it to go away in the early 2000s or so, and I'm guessing there are plenty of places where it still lives on, in people learning programming today by looking at older code and older books, who at some point will have to (hopefully) re-educated by their teachers and colleagues.

We have not even really started on this re-education of programmers so that they do things in a better way (where "better" means using abstractions that are a better match for parallel programming). Our languages, APIs, and tools are still in an abysmal state, like we're using Fortran and so trying to force recursion or pointers into the language is like pulling teeth. Our tools don't have the sort of refactoring that today makes us not care about needing to add a new parameter to a chain of function calls. etc etc.

It MAY be true that parallelism is a dead end. But claiming so on the basis that programming the Linux kernel is crazy hard, that C++14 is a godawful mess that pretty much no human being fully understands, and that the OSX/Windows APIs have piecemeal support for concurrency strikes me as premature. These things take time --- and the time scale may well be thirty years, where the clock really only started ticking in about 2004.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
ARMv8 getting atomic operationsdmcq12/02/14 05:32 PM
  ARMv8 getting atomic operationsMaynard Handley12/02/14 07:33 PM
    ARMv8 getting atomic operationsDoug S12/02/14 10:30 PM
      ARMv8 getting atomic operationsdmcq12/03/14 03:16 AM
      ARMv8 getting atomic operationsMaynard Handley12/03/14 09:20 AM
      ARMv8 getting atomic operationsBrett12/03/14 04:46 PM
    ARMv8 getting atomic operationsAndreas12/03/14 06:51 AM
      ARMv8 getting atomic operationsLinus Torvalds12/03/14 11:15 AM
        ARMv8 getting atomic operationsanon12/03/14 05:08 PM
          Guaranteed transactionsPaul A. Clayton12/03/14 08:04 PM
            Guaranteed transactionsanon12/03/14 08:38 PM
              Avoiding ping pongPaul A. Clayton12/04/14 09:11 AM
                Avoiding ping ponganon12/04/14 10:15 AM
                  OoO window is limitedPaul A. Clayton12/04/14 01:06 PM
                Avoiding ping pongAaron Spink12/04/14 12:01 PM
                  Avoiding ping pongKonrad Schwarz12/04/14 01:10 PM
                    Avoiding ping pongAaron Spink12/04/14 02:31 PM
                    Avoiding ping pongGabriele Svelto12/04/14 02:49 PM
                      Avoiding ping pongKonrad Schwarz12/04/14 11:08 PM
                        Avoiding ping pongGabriele Svelto12/05/14 12:04 AM
                          Avoiding ping pongEric Bron nli12/05/14 02:28 AM
                            Avoiding ping pongKonrad Schwarz12/05/14 03:37 AM
                              Avoiding ping pongEric Bron nli12/05/14 04:23 AM
                                Avoiding ping pongKlimax12/05/14 05:47 AM
                                  Avoiding ping pongEric Bron12/05/14 06:24 AM
                              Avoiding ping pongGabriele Svelto12/05/14 10:38 AM
                                Avoiding ping pongKonrad Schwarz12/07/14 02:28 PM
                                  Avoiding ping pongGabriele Svelto12/08/14 07:10 PM
                                    Avoiding ping pongKonrad Schwarz12/09/14 05:12 AM
                                      Avoiding ping pongGabriele Svelto12/09/14 07:31 AM
                                        Avoiding ping ponganon12/09/14 11:24 PM
                            Avoiding ping pongGabriele Svelto12/05/14 10:17 AM
                              Avoiding ping pongEric Bron12/05/14 10:32 AM
                                Avoiding ping pongGabriele Svelto12/05/14 12:45 PM
                                  Avoiding ping pongEric Bron12/06/14 02:20 AM
                                    Avoiding ping pongnksingh12/06/14 03:42 AM
                                      Avoiding ping pongEric Bron12/06/14 04:04 AM
                                        Avoiding ping pongGiGNiC12/06/14 06:27 AM
                                          Avoiding ping pongEric Bron nli12/06/14 06:44 AM
                                          Avoiding ping pongEric Bron12/06/14 07:07 AM
                                            Avoiding ping pongnksingh12/07/14 04:06 PM
                                              Avoiding ping pongEric Bron12/08/14 04:17 AM
                                                Avoiding ping pongGiGNiC12/08/14 11:53 AM
                                                Avoiding ping pongnksingh12/08/14 05:53 PM
                                                  Avoiding ping pongEric Bron12/09/14 01:33 AM
                                    Avoiding ping pongdmsc12/06/14 04:12 AM
                                      Avoiding ping pongEric Bron12/06/14 04:25 AM
                                        Avoiding ping pongKlimax12/06/14 05:49 AM
                                          Avoiding ping pongrwessel12/07/14 02:34 AM
                                        Avoiding ping pongdmsc12/06/14 07:39 AM
                                        Avoiding ping pongKonrad Schwarz12/07/14 02:37 PM
                                          Avoiding ping pongMichael S12/07/14 04:37 PM
                                            Avoiding ping pongKonrad Schwarz12/08/14 04:35 AM
                          Avoiding ping pongKonrad Schwarz12/05/14 03:30 AM
                        Avoiding ping pongLinus Torvalds12/05/14 12:58 PM
                          Avoiding ping pongEric Bron12/06/14 02:42 AM
                            Avoiding ping pongnksingh12/06/14 03:51 AM
                              Avoiding ping pongEric Bron12/06/14 04:08 AM
                            Avoiding ping pongLinus Torvalds12/06/14 01:25 PM
                              Avoiding ping pongnksingh12/07/14 03:26 PM
                                Avoiding ping pongEric Bron12/08/14 04:35 AM
                                  Avoiding ping pongBrett12/08/14 10:00 AM
                                    Avoiding ping pongEric Bron12/08/14 10:48 AM
                                    Avoiding ping pongrwessel12/08/14 12:52 PM
                                      Avoiding ping pongBrett12/08/14 01:58 PM
                                      Avoiding ping pongDoug S12/08/14 02:04 PM
                              Avoiding ping pongJouni Osmala12/08/14 02:45 AM
                                Avoiding ping ponganon12/08/14 05:44 AM
                                  Avoiding ping pongJouni Osmala12/08/14 01:10 PM
                                    Avoiding ping pongLinus Torvalds12/08/14 01:34 PM
                                      Avoiding ping pongJouni Osmala12/08/14 03:47 PM
                                        Avoiding ping pongLinus Torvalds12/08/14 08:08 PM
                                          Avoiding ping pongGabriele Svelto12/09/14 07:48 AM
                                            Avoiding ping pongMaynard Handley12/09/14 11:41 AM
                                              Avoiding ping pongPatrick Chase12/09/14 01:06 PM
                                              Avoiding ping pongGabriele Svelto12/09/14 01:52 PM
                                                Avoiding ping pongPatrick Chase12/09/14 02:08 PM
                                            Why read RWT or Reddit when you can get journalists to do it for you?Rob Thorpe01/02/15 08:20 AM
                                              Why read RWT or Reddit when you can get journalists to do it for you?juanrga01/02/15 11:21 AM
                                                Why read RWT or Reddit when you can get journalists to do it for you?EduardoS01/02/15 11:37 AM
                                                  Why read RWT or Reddit when you can get journalists to do it for you?juanrga01/03/15 12:00 PM
                                                Why read RWT or Reddit when you can get journalists to do it for you?Eric Bron nli01/02/15 02:28 PM
                                                  Why read RWT or Reddit when you can get journalists to do it for you?juanrga01/03/15 12:02 PM
                                                    Why read RWT or Reddit when you can get journalists to do it for you?Michael S01/03/15 12:36 PM
                                                      Why read RWT or Reddit when you can get journalists to do it for you?juanrga01/03/15 01:11 PM
                                                        Why read RWT or Reddit when you can get journalists to do it for you?Michael S01/03/15 01:30 PM
                                                          Why read RWT or Reddit when you can get journalists to do it for you?Eric Bron01/03/15 02:57 PM
                                                            KNL cacheDavid Kanter01/03/15 07:36 PM
                                                              KNL cacheEric Bron01/04/15 03:34 AM
                                                                KNL cacheMichael S01/04/15 04:11 AM
                                                                  KNL cacheEric Bron01/04/15 04:57 AM
                                                                    KNL cacheMichael S01/04/15 05:21 AM
                                                                      KNL cacheEric Bron01/04/15 05:58 AM
                                                          Why read RWT or Reddit when you can get journalists to do it for you?juanrga01/07/15 05:47 AM
                                                            Why read RWT or Reddit when you can get journalists to do it for you?Michael S01/07/15 08:27 AM
                                                              Manycores vs multicoresjuanrga01/10/15 04:10 PM
                                                                Manycores vs multicoresAaron Spink01/10/15 05:32 PM
                                                                  Manycores vs multicoresjuanrga01/10/15 06:32 PM
                                                                    Manycores vs multicoresExophase01/10/15 06:49 PM
                                                                      Manycores vs multicoresjuanrga01/10/15 08:21 PM
                                                                        Manycores vs multicoresExophase01/10/15 08:51 PM
                                                                        Manycores vs multicoresAaron Spink01/10/15 09:03 PM
                                                                    Manycores vs multicoresAaron Spink01/10/15 07:21 PM
                                                                      Manycores vs multicoresjuanrga01/10/15 08:25 PM
                                                                        Manycores vs multicoresAaron Spink01/10/15 09:11 PM
                                                                          Manycores vs multicoresJouni Osmala01/11/15 04:50 AM
                                                                            Manycores vs multicoresjuanrga01/11/15 08:58 AM
                                                                            Manycores vs multicorescoppice01/12/15 10:01 PM
                                                                              Manycores vs multicoresJouni Osmala01/13/15 04:38 AM
                                                                        Manycores vs multicoresanon01/11/15 03:19 AM
                                                                      Manycores vs multicoresMichael S01/11/15 05:44 AM
                                                                        Manycores vs multicoresAaron Spink01/11/15 05:55 PM
                                                                          Manycores vs multicoresMichael S01/12/15 04:41 AM
                                                                            Manycores vs multicoresEric Bron01/12/15 06:29 AM
                                                                              Manycores vs multicoresEric Bron01/12/15 06:30 AM
                                                        Why read RWT or Reddit when you can get journalists to do it for you?Eric Bron01/03/15 02:54 PM
                                                          Why read RWT or Reddit when you can get journalists to do it for you?juanrga01/07/15 05:48 AM
                                                            Why read RWT or Reddit when you can get journalists to do it for you?Eric Bron01/07/15 07:41 AM
                                                              Manycores vs multicoresjuanrga01/10/15 04:14 PM
                                                    Why read RWT or Reddit when you can get journalists to do it for you?Eric Bron01/03/15 02:42 PM
                                                      Why read RWT or Reddit when you can get journalists to do it for you?juanrga01/07/15 06:03 AM
                                                        Why read RWT or Reddit when you can get journalists to do it for you?Eric Bron01/07/15 07:45 AM
                                                        Why read RWT or Reddit when you can get journalists to do it for you?Linus Torvalds01/08/15 03:09 PM
                                                          Pink unicorns for salejuanrga01/10/15 05:09 PM
                                                        Intentionally picking a competitors slow part is cheating ...Mark Roulo01/08/15 06:37 PM
                                                          Intentionally picking a competitors slow part is cheating ...coppice01/08/15 11:38 PM
                                                            Intentionally picking a competitors slow part is cheating ...Mark Roulo01/09/15 09:13 AM
                                                              Intentionally picking a competitors slow part is cheating ...Anon01/10/15 02:00 AM
                                                              Intentionally picking a competitors slow part is cheating ...David Hess01/11/15 01:03 PM
                                                            Intentionally picking a competitors slow part is cheating ...someone01/09/15 10:31 AM
                                                              Intentionally picking a competitors slow part is cheating ...coppice01/12/15 09:45 PM
                                                                Intentionally picking a competitors slow part is cheating ...coppice01/12/15 09:47 PM
                                                                  Intentionally picking a competitors slow part is cheating ...Michael S01/13/15 07:53 AM
                                                                    Intentionally picking a competitors slow part is cheating ...coppice01/13/15 09:44 AM
                                                                      Intentionally picking a competitors slow part is cheating ...Michael S01/13/15 10:01 AM
                                                                        Intentionally picking a competitors slow part is cheating ...coppice01/13/15 08:35 PM
                                                                      Core sizesjuanrga01/13/15 12:28 PM
                                                          NVIDIA'S FIRST CPU IS A WINNER (Linley Gwennap)juanrga01/10/15 04:34 PM
                                                        Why read RWT or Reddit when you can get journalists to do it for you?Patrick Chase01/08/15 07:02 PM
                                                        Why read RWT or Reddit when you can get journalists to do it for you?coppice01/08/15 10:18 PM
                                                          Why read RWT or Reddit when you can get journalists to do it for you?Patrick Chase01/09/15 11:54 AM
                                                            Why read RWT or Reddit when you can get journalists to do it for you?Mark Roulo01/09/15 12:59 PM
                                                              Why read RWT or Reddit when you can get journalists to do it for you?Patrick Chase01/09/15 03:20 PM
                                                                Why read RWT or Reddit when you can get journalists to do it for you?Eric Bron01/09/15 03:30 PM
                                                            Alternatives to OOOE (again)juanrga01/10/15 04:50 PM
                                                              Alternatives to OOOE (again)David Kanter01/11/15 12:10 AM
                                                                Alternatives to OOOE (again)juanrga01/11/15 08:30 AM
                                                            Why read RWT or Reddit when you can get journalists to do it for you?Gabriele Svelto01/11/15 12:53 AM
                                              Why read RWT or Reddit when you can get journalists to do it for you?Fake Linus Torvalds01/03/15 12:14 PM
                                                Why read RWT or Reddit when you can get journalists to do it for you?Rob Thorpe01/03/15 08:25 PM
                                          Avoiding ping pongMaynard Handley12/09/14 11:33 AM
                                            Avoiding ping pongPatrick Chase12/09/14 01:54 PM
                                              Avoiding ping pongMaynard Handley12/09/14 06:56 PM
                                      Avoiding ping pongSalvatore De Dominicis12/09/14 08:51 AM
                                        Avoiding ping pongPatrick Chase12/09/14 02:00 PM
                                      Avoiding ping pongook12/11/14 03:31 AM
                                      Avoiding ping pongArt Scott12/19/14 10:19 PM
                                        Avoiding ping pongEric Bron nli12/20/14 04:05 AM
                                      What about specialization?Troll?01/02/15 07:55 AM
                                        What about specialization?Ungo01/04/15 03:27 PM
                                      Avoiding ping pongfewwef01/05/15 08:16 PM
                                      Avoiding ping pongV.Krishn01/08/15 06:11 AM
                                    Avoiding ping pongGabriele Svelto12/08/14 07:32 PM
                                    Avoiding ping ponganon12/08/14 11:37 PM
                            Avoiding ping pongKonrad Schwarz12/10/14 06:23 AM
                              Avoiding ping pongLinus Torvalds12/10/14 11:56 AM
                          Object reference lockingDavid W12/08/14 11:36 PM
                            Object reference lockingPatrick Chase12/09/14 04:52 PM
                              Object reference lockingDavid W12/11/14 05:18 AM
                    ISA != interface for "most programmers"Paul A. Clayton12/04/14 03:34 PM
                      ISA != interface for "most programmers"rwessel12/04/14 07:50 PM
                  Interesting! (exporting hot lines/cache-aware ISA); "Please sir, I want some more" (NT)Paul A. Clayton12/04/14 02:26 PM
                  Avoiding ping pongMichael S12/06/14 03:48 PM
          ARMv8 getting atomic operationsLinus Torvalds12/04/14 12:05 PM
            LL/SC idiom recognition is not admitting RMW superiorityPaul A. Clayton12/04/14 02:34 PM
            ARMv8 getting atomic operationsanon12/04/14 10:17 PM
    ARMv8 getting atomic operationsPatrick Chase12/03/14 12:09 PM
  limited ordernksingh12/04/14 10:17 PM
    I didn't understand this either. (NT)Konrad Schwarz12/04/14 10:32 PM
    limited orderdmcq12/05/14 02:13 AM
    limited orderbakaneko12/05/14 09:11 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?