Sharing between servers and phones

By: Linus Torvalds (torvalds.delete@this.linux-foundation.org), January 28, 2017 11:49 am
Room: Moderated Discussions
David Kanter (dkanter.delete@this.realworldtech.com) on January 27, 2017 6:46 am wrote:
>
> The TLB is a good example. It is larger in servers do deal with
> larger data footprints. But TLBs are power-hungry and large.

I'm not actually sure that the TLB is all that good an example.

There are various trade-offs wrt TLB's, and "larger data footprints" is only one issue.

Another (big) issue for TLB's is interface and usage, and latency of the inevitable misses (and lots of misses are compulsory, not capacity).

To be specific, for Intel the traditional TLB lookup model was (and that traditional model still affects things today, even if it's slightly modified):

(a) no address space ID's

(b) very frequent TLB flushes

where (a) is simply from historical interface reasons and (b) is from how the traditional win32 GDI subsystem worked (and other workarounds for things like PAE) which just caused the TLB to be flushed all the time.

Now, the naive assumption is that "if the TLB gets flushed very often, then a small TLB makes sense". But that's not actually really all that happened.

What happened is that because of the constant TLB flushes, the TLB re-population simply got a lot more critical for Intel (and AMD) CPU's than it tends to be for the "server only" CPU's that you mention.

If you are a POWER architect, the easy way to make the TLB more effective is to just make it much bigger. You are running only big jobs that almost never flush the TLB, so making the TLB bigger is a simply no-brainer. Same is true on zArch.

But I really want to stress that "no brainer" part. It's not a "clever" approach. It's also not something that should be seen as a good thing and be seen as "big box serious hardware for real men and women". It's the stupid approach that just happens to work pretty well under the kinds of boring and big - but otherwise pretty well-behaved - applications that those CPU's were mostly running.

The thing is, the POWER page tables are traditionally horrible nasty shit, and TLB misses take a long time and need software support to fill in the hashed page tables. The thing is just bad. It's a really bad design, and the big TLB's are pure brawn with no brains. I'm not as familiar with s390, so I'm not going to say horrible things about that, but I do want to stress that "big TLB" is not necessarily a good thing in itself.

Because the other side of the coin really is how well you can fill that TLB.

And if you are better at filling the TLB, you may simply need a smaller TLB in practice.

Put another way: I'd rather take a smaller, smarter, low-latency cache that is coupled with a smart memory subsystem that can have multiple outstanding misses and does a good job of prefetching and not stalling the pipeline in inconvenient situations.

And I think you'd say "Duh! Of course" if you thought about it from the standpoint of the regular L1D$.

I'm saying that the exact same thing is true of the TLB. Size is absolutely not everything. "Big" does not automatically equal "good". If everything else is equal, big is obviously better, but everything else is very much not equal.

Server loads are in many ways simpler than desktop and mobile loads. A lot of traditional server loads can be handled by just putting "more" of everything. More cores. Bigger caches and TLB's. More memory. More, more, more. But seldom "clever". Brute force over nimble and smart. I don't think people call some of those things dinosaurs just because they are old.

Linus
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Cray to Develop ARM-based Isambard Supercomputer for UK Met Officegallier201/20/17 07:48 AM
  Cray to Develop ARM-based Isambard Supercomputer for UK Met Officegallier201/20/17 07:48 AM
    Ignore second post (NT)gallier201/20/17 07:49 AM
  ARM-based supercomputersDavid Kanter01/20/17 02:55 PM
    ARM-based supercomputersMaynard Handley01/20/17 08:55 PM
      ARM-based supercomputers lolRobert David Graham01/21/17 05:34 PM
        ARM-based supercomputers lolnone01/21/17 06:44 PM
          ARM-based supercomputers lolMaynard Handley01/21/17 07:36 PM
            ARM-based supercomputers lolMichael S01/22/17 02:07 AM
            What did you learn?Innocent Bystander01/22/17 08:26 AM
          ARM-based supercomputers lolanon01/22/17 02:24 AM
            ARM-based supercomputers lolGabriele Svelto01/23/17 12:53 AM
              ARM-based supercomputers lolanon01/23/17 06:40 AM
            ARM-based supercomputers loljuanrga01/29/17 05:01 AM
              ARM-based supercomputers lolanon01/29/17 09:35 AM
        ARM-based supercomputers lolBrendan01/21/17 07:36 PM
         Dumb belief? ROFL (NT)juanrga01/22/17 02:59 AM
          Dumb belief? ROFLwumpus01/22/17 11:31 AM
            Dumb belief? ROFLjuanrga01/29/17 05:13 AM
        ARM-based supercomputers lolDavid Kanter01/22/17 08:20 AM
    ARM-based supercomputersRichardC01/22/17 07:54 AM
      ARM-based supercomputersAaron Spink01/22/17 12:03 PM
        ARM-based supercomputersNiels Jørgen Kruse01/22/17 12:16 PM
          ARM-based supercomputersAaron Spink01/23/17 07:01 PM
            ARM-based supercomputersNiels Jørgen Kruse01/23/17 10:41 PM
              ARM-based supercomputerstarlinian01/24/17 08:58 AM
                ARM-based supercomputersMichael S01/24/17 09:05 AM
                  ARM-based supercomputersAaron Spink01/24/17 10:38 AM
        ARM-based supercomputersdmcq01/22/17 01:03 PM
          A73 does not have L1 ECCDavid Kanter01/22/17 09:50 PM
            A73 does not have L1 ECCdmcq01/23/17 03:32 PM
              A73 does not have L1 ECCMichael S01/23/17 03:50 PM
              A73 does not have L1 ECCdmcq01/24/17 08:15 AM
                A73 does not have L1 ECCMichael S01/24/17 08:47 AM
          ARM SOCs with ECC DRAMMichael S01/23/17 03:17 AM
          ARM-based supercomputersAaron Spink01/23/17 07:02 PM
            X-Gene 3 supports 1TB of memoryvvid01/24/17 01:55 AM
              X-Gene 3 supports 1TB of memoryWilco01/24/17 02:13 AM
                You seem to be confusing "SoC" with "CPU core" (NT)Anon01/24/17 03:12 AM
                X-Gene 3 supports 1TB of memoryMichael S01/24/17 05:26 AM
                  X-Gene 3 supports 1TB of memorydmcq01/24/17 08:28 AM
                X-Gene 3 supports 1TB of memoryAaron Spink01/24/17 06:46 AM
                  X-Gene 3 supports 1TB of memoryWilco01/24/17 03:10 PM
                    X-Gene 3 supports 1TB of memoryDavid Kanter01/24/17 07:31 PM
                X-Gene 3 supports 1TB of memorySimon Farnsworth01/26/17 02:23 AM
              X-Gene 3 supports 1TB of memoryAaron Spink01/24/17 06:44 AM
                X-Gene 3 supports 1TB of memoryAnon01/24/17 01:54 PM
                  X-Gene 3 supports 1TB of memoryWilco01/24/17 03:07 PM
                    X-Gene 3 supports 1TB of memoryDavid Kanter01/24/17 07:34 PM
                      X-Gene 3 supports 1TB of memorynone01/24/17 10:48 PM
                        X-Gene 3 supports 1TB of memoryMichael S01/25/17 01:32 AM
                        X-Gene 3 supports 1TB of memoryDavid Hess01/25/17 09:47 PM
                          X-Gene 3 supports 1TB of memoryMichael S01/26/17 12:55 AM
                            Really that different?Daniel B01/26/17 03:37 AM
                              Really that different?none01/26/17 05:39 AM
                                Really that different?itsmydamnation01/26/17 03:55 PM
                                  Really that different?none01/26/17 11:14 PM
                      X-Gene 3 supports 1TB of memoryWilco01/25/17 03:22 AM
                        Sharing between servers and phonesDavid Kanter01/25/17 07:15 AM
                          Sharing between servers and phonesWilco01/25/17 04:41 PM
                            Sharing between servers and phonesDavid Kanter01/25/17 06:10 PM
                              Sharing between servers and phonesGabriele Svelto01/26/17 03:15 AM
                                Sharing between servers and phonesDavid Kanter01/26/17 07:33 AM
                                  Sharing between servers and phoneswumpus01/26/17 03:25 PM
                                    Sharing between servers and phonesDavid Kanter01/27/17 06:46 AM
                                      Any idea why Intel doesn't ship server chips with eDRAM?Mark Roulo01/27/17 09:02 AM
                                        Any idea why Intel doesn't ship server chips with eDRAM?Per Hesselgren01/28/17 06:49 AM
                                          Any idea why Intel doesn't ship server chips with eDRAM?Simon Farnsworth01/28/17 07:06 AM
                                          Any idea why Intel doesn't ship server chips with eDRAM?Michael S01/29/17 02:43 AM
                                            off die eDRAM?wumpus01/29/17 06:57 AM
                                              off die eDRAM?anon01/29/17 09:46 AM
                                      Sharing between servers and phonesrwessel01/27/17 10:36 PM
                                      Sharing between servers and phonesLinus Torvalds01/28/17 11:49 AM
                                        Sharing between servers and phonesrwessel01/29/17 08:56 PM
                                          Sharing between servers and phonesLinus Torvalds01/30/17 10:01 AM
                                            Sharing between servers and phonesrwessel01/31/17 12:29 AM
                                              Sharing between servers and phonesIreland01/31/17 07:55 AM
                                                Please keep your posts on topicDavid Kanter01/31/17 11:32 AM
                                                  Technology that can survive in harsh conditionsIreland01/31/17 12:51 PM
                                                    Stay on topic, you have been warnedDavid Kanter01/31/17 06:56 PM
                                                On topic summary and my thoughts on it. Jouni Osmala01/31/17 01:10 PM
                                                  On topic summary and my thoughts on it. Ireland01/31/17 01:27 PM
                                              Sharing between servers and phonesLinus Torvalds01/31/17 01:01 PM
                                                Sharing between servers and phonesLinus Torvalds01/31/17 01:49 PM
                                                  Sharing between servers and phonesDoug S01/31/17 02:12 PM
                                                Sharing between servers and phonesrwessel01/31/17 05:54 PM
                                                  Sharing between servers and phonesLinus Torvalds02/01/17 09:17 AM
                                                    Sharing between servers and phonesrwessel02/02/17 03:40 PM
                                                  Sharing between servers and phonesjoncmu02/01/17 01:36 PM
                                                    Sharing between servers and phonesChristian Borntraeger02/02/17 03:46 AM
            ARM-based supercomputersRichardC01/24/17 05:50 AM
              ARM-based supercomputersAaron Spink01/24/17 07:20 AM
                ARM-based supercomputersdmcq01/24/17 08:44 AM
                  ARM-based supercomputersnone01/24/17 09:10 AM
                  ARM-based supercomputersjuanrga01/29/17 05:33 AM
                    ARM-based supercomputershobel01/30/17 02:35 AM
                ARM-based supercomputersRichardC 01/24/17 09:34 AM
                  ARM-based supercomputersdmcq01/24/17 10:10 AM
                  ARM-based supercomputersAaron Spink01/24/17 11:01 AM
                    ARM-based supercomputersRichardC01/24/17 04:06 PM
                      ARM-based supercomputersIreland01/24/17 05:16 PM
                      ARM-based supercomputersAaron Spink01/24/17 07:43 PM
                    video renderingRichardC01/24/17 05:08 PM
                      video renderingIreland01/24/17 05:26 PM
                      video renderingAaron Spink01/24/17 07:54 PM
                        video renderingRichardC01/25/17 04:26 AM
                          display bandwidthRichardC01/25/17 05:30 AM
                          video renderingIreland01/25/17 08:11 AM
                            You can keep a coal furnace fed all the time. (NT)anon01/25/17 03:27 PM
                              You can keep a coal furnace fed all the time.Ireland01/25/17 03:36 PM
                                You can keep a coal furnace fed all the time.anon01/27/17 03:29 AM
                                  You can keep a coal furnace fed all the time.Michael S01/27/17 07:22 AM
                                    You can keep a coal furnace fed all the time.Ireland01/27/17 10:59 AM
                                    You can keep a coal furnace fed all the time.anon01/27/17 03:09 PM
                                      Information and Super Materials Ireland01/28/17 11:13 AM
                            video renderingRichardC01/26/17 11:39 AM
                              video renderingIreland01/26/17 12:49 PM
                                video renderingIreland01/26/17 12:58 PM
                                video renderingRichardC01/26/17 03:24 PM
                                  video renderingIreland01/26/17 05:09 PM
                          video renderingGabriele Svelto01/25/17 08:16 AM
                            video renderingIreland01/25/17 08:33 AM
                              The challenge at Pixar Ireland01/25/17 08:40 AM
                                Pixar story & technologyRichard Cownie01/25/17 11:36 AM
                                  Pixar story & technologyIreland01/25/17 11:58 AM
                          video renderingAaron Spink01/25/17 02:49 PM
                            video renderingIreland01/25/17 03:26 PM
                            cloud network infrastructureRichardC01/26/17 10:47 AM
                              cloud network infrastructureAaron Spink01/26/17 07:37 PM
                ARM-based supercomputersGabriele Svelto01/24/17 12:40 PM
                  ARM-based supercomputersAaron Spink01/24/17 08:00 PM
        ARM-based supercomputersRichardC01/22/17 09:45 PM
          ARM-based supercomputersGabriele Svelto01/23/17 01:03 AM
            ARM-based supercomputersRichardC01/23/17 06:57 AM
      ARM-based supercomputersGabriele Svelto01/23/17 03:23 AM
    Mont blanc project and ARM HPC in generaljuanrga01/29/17 04:42 AM
      Juan, why do you have to be such an arrogant twat to the person who provides this forum? (NT)Annoyed01/30/17 02:32 AM
      Still idioticDavid Kanter01/30/17 07:49 AM
        Still idioticMr. Camel01/30/17 08:16 PM
        Still idioticjuanrga02/02/17 08:11 AM
      Mont blanc project and ARM HPC in generalIreland01/31/17 04:53 PM
        Four Things to Consider Ireland01/31/17 05:11 PM
          Four Things to Consider tarlinian01/31/17 06:38 PM
            Four Things to Consider Ireland01/31/17 06:58 PM
              no evidence that it goes on a shipRichardC02/01/17 06:05 AM
                no evidence that it goes on a shipIreland02/02/17 01:57 PM
                  no evidence that it goes on a shipRichardC02/03/17 06:04 AM
                    no evidence that it goes on a shipIreland02/03/17 08:02 AM
              A better place to site a supercomputerAnon02/01/17 06:57 AM
                A better place to site a supercomputerIreland02/01/17 06:37 PM
                  A better Ireland..Anon02/01/17 07:52 PM
                    A better Ireland..slacker02/01/17 10:32 PM
                    A better Ireland..Ireland02/02/17 04:06 AM
                    He is manic (NT)anonymo02/02/17 12:21 PM
                      or a botanon02/02/17 02:17 PM
                        or a botanonymou502/02/17 03:46 PM
                        or a botnone02/02/17 10:45 PM
                          or a botanon02/03/17 09:30 AM
                        Better bot example: amanfrommars1Doug S02/03/17 10:44 AM
            Make allowances, the man never sleeps..Anon01/31/17 08:22 PM
        Mont blanc project and ARM HPC in generaletudiant02/02/17 08:39 AM
          Mont blanc project and ARM HPC in generalIreland02/02/17 11:12 AM
            Connection between two different modelsIreland02/02/17 11:18 AM
              Connection between two different modelsetudiant02/04/17 02:40 PM
                Connection between two different modelsRichardC02/06/17 08:36 AM
                  Connection between two different modelsdmcq02/06/17 10:07 AM
                  Connection between two different modelsIreland02/06/17 11:17 AM
                    probably not the right forum ...RichardC02/06/17 12:31 PM
                      probably not the right forum ...Ireland02/06/17 01:53 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?