Sharing between servers and phones

By: Linus Torvalds (torvalds.delete@this.linux-foundation.org), January 28, 2017 12:49 pm
Room: Moderated Discussions
David Kanter (dkanter.delete@this.realworldtech.com) on January 27, 2017 6:46 am wrote:
>
> The TLB is a good example. It is larger in servers do deal with
> larger data footprints. But TLBs are power-hungry and large.

I'm not actually sure that the TLB is all that good an example.

There are various trade-offs wrt TLB's, and "larger data footprints" is only one issue.

Another (big) issue for TLB's is interface and usage, and latency of the inevitable misses (and lots of misses are compulsory, not capacity).

To be specific, for Intel the traditional TLB lookup model was (and that traditional model still affects things today, even if it's slightly modified):

(a) no address space ID's

(b) very frequent TLB flushes

where (a) is simply from historical interface reasons and (b) is from how the traditional win32 GDI subsystem worked (and other workarounds for things like PAE) which just caused the TLB to be flushed all the time.

Now, the naive assumption is that "if the TLB gets flushed very often, then a small TLB makes sense". But that's not actually really all that happened.

What happened is that because of the constant TLB flushes, the TLB re-population simply got a lot more critical for Intel (and AMD) CPU's than it tends to be for the "server only" CPU's that you mention.

If you are a POWER architect, the easy way to make the TLB more effective is to just make it much bigger. You are running only big jobs that almost never flush the TLB, so making the TLB bigger is a simply no-brainer. Same is true on zArch.

But I really want to stress that "no brainer" part. It's not a "clever" approach. It's also not something that should be seen as a good thing and be seen as "big box serious hardware for real men and women". It's the stupid approach that just happens to work pretty well under the kinds of boring and big - but otherwise pretty well-behaved - applications that those CPU's were mostly running.

The thing is, the POWER page tables are traditionally horrible nasty shit, and TLB misses take a long time and need software support to fill in the hashed page tables. The thing is just bad. It's a really bad design, and the big TLB's are pure brawn with no brains. I'm not as familiar with s390, so I'm not going to say horrible things about that, but I do want to stress that "big TLB" is not necessarily a good thing in itself.

Because the other side of the coin really is how well you can fill that TLB.

And if you are better at filling the TLB, you may simply need a smaller TLB in practice.

Put another way: I'd rather take a smaller, smarter, low-latency cache that is coupled with a smart memory subsystem that can have multiple outstanding misses and does a good job of prefetching and not stalling the pipeline in inconvenient situations.

And I think you'd say "Duh! Of course" if you thought about it from the standpoint of the regular L1D$.

I'm saying that the exact same thing is true of the TLB. Size is absolutely not everything. "Big" does not automatically equal "good". If everything else is equal, big is obviously better, but everything else is very much not equal.

Server loads are in many ways simpler than desktop and mobile loads. A lot of traditional server loads can be handled by just putting "more" of everything. More cores. Bigger caches and TLB's. More memory. More, more, more. But seldom "clever". Brute force over nimble and smart. I don't think people call some of those things dinosaurs just because they are old.

Linus
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Cray to Develop ARM-based Isambard Supercomputer for UK Met Officegallier22017/01/20 08:48 AM
  Cray to Develop ARM-based Isambard Supercomputer for UK Met Officegallier22017/01/20 08:48 AM
    Ignore second post (NT)gallier22017/01/20 08:49 AM
  ARM-based supercomputersDavid Kanter2017/01/20 03:55 PM
    ARM-based supercomputersMaynard Handley2017/01/20 09:55 PM
      ARM-based supercomputers lolRobert David Graham2017/01/21 06:34 PM
        ARM-based supercomputers lolnone2017/01/21 07:44 PM
          ARM-based supercomputers lolMaynard Handley2017/01/21 08:36 PM
            ARM-based supercomputers lolMichael S2017/01/22 03:07 AM
            What did you learn?Innocent Bystander2017/01/22 09:26 AM
          ARM-based supercomputers lolanon2017/01/22 03:24 AM
            ARM-based supercomputers lolGabriele Svelto2017/01/23 01:53 AM
              ARM-based supercomputers lolanon2017/01/23 07:40 AM
            ARM-based supercomputers loljuanrga2017/01/29 06:01 AM
              ARM-based supercomputers lolanon2017/01/29 10:35 AM
        ARM-based supercomputers lolBrendan2017/01/21 08:36 PM
         Dumb belief? ROFL (NT)juanrga2017/01/22 03:59 AM
          Dumb belief? ROFLwumpus2017/01/22 12:31 PM
            Dumb belief? ROFLjuanrga2017/01/29 06:13 AM
        ARM-based supercomputers lolDavid Kanter2017/01/22 09:20 AM
    ARM-based supercomputersRichardC2017/01/22 08:54 AM
      ARM-based supercomputersAaron Spink2017/01/22 01:03 PM
        ARM-based supercomputersNiels Jørgen Kruse2017/01/22 01:16 PM
          ARM-based supercomputersAaron Spink2017/01/23 08:01 PM
            ARM-based supercomputersNiels Jørgen Kruse2017/01/23 11:41 PM
              ARM-based supercomputerstarlinian2017/01/24 09:58 AM
                ARM-based supercomputersMichael S2017/01/24 10:05 AM
                  ARM-based supercomputersAaron Spink2017/01/24 11:38 AM
        ARM-based supercomputersdmcq2017/01/22 02:03 PM
          A73 does not have L1 ECCDavid Kanter2017/01/22 10:50 PM
            A73 does not have L1 ECCdmcq2017/01/23 04:32 PM
              A73 does not have L1 ECCMichael S2017/01/23 04:50 PM
              A73 does not have L1 ECCdmcq2017/01/24 09:15 AM
                A73 does not have L1 ECCMichael S2017/01/24 09:47 AM
          ARM SOCs with ECC DRAMMichael S2017/01/23 04:17 AM
          ARM-based supercomputersAaron Spink2017/01/23 08:02 PM
            X-Gene 3 supports 1TB of memoryvvid2017/01/24 02:55 AM
              X-Gene 3 supports 1TB of memoryWilco2017/01/24 03:13 AM
                You seem to be confusing "SoC" with "CPU core" (NT)Anon2017/01/24 04:12 AM
                X-Gene 3 supports 1TB of memoryMichael S2017/01/24 06:26 AM
                  X-Gene 3 supports 1TB of memorydmcq2017/01/24 09:28 AM
                X-Gene 3 supports 1TB of memoryAaron Spink2017/01/24 07:46 AM
                  X-Gene 3 supports 1TB of memoryWilco2017/01/24 04:10 PM
                    X-Gene 3 supports 1TB of memoryDavid Kanter2017/01/24 08:31 PM
                X-Gene 3 supports 1TB of memorySimon Farnsworth2017/01/26 03:23 AM
              X-Gene 3 supports 1TB of memoryAaron Spink2017/01/24 07:44 AM
                X-Gene 3 supports 1TB of memoryAnon2017/01/24 02:54 PM
                  X-Gene 3 supports 1TB of memoryWilco2017/01/24 04:07 PM
                    X-Gene 3 supports 1TB of memoryDavid Kanter2017/01/24 08:34 PM
                      X-Gene 3 supports 1TB of memorynone2017/01/24 11:48 PM
                        X-Gene 3 supports 1TB of memoryMichael S2017/01/25 02:32 AM
                        X-Gene 3 supports 1TB of memoryDavid Hess2017/01/25 10:47 PM
                          X-Gene 3 supports 1TB of memoryMichael S2017/01/26 01:55 AM
                            Really that different?Daniel B2017/01/26 04:37 AM
                              Really that different?none2017/01/26 06:39 AM
                                Really that different?itsmydamnation2017/01/26 04:55 PM
                                  Really that different?none2017/01/27 12:14 AM
                      X-Gene 3 supports 1TB of memoryWilco2017/01/25 04:22 AM
                        Sharing between servers and phonesDavid Kanter2017/01/25 08:15 AM
                          Sharing between servers and phonesWilco2017/01/25 05:41 PM
                            Sharing between servers and phonesDavid Kanter2017/01/25 07:10 PM
                              Sharing between servers and phonesGabriele Svelto2017/01/26 04:15 AM
                                Sharing between servers and phonesDavid Kanter2017/01/26 08:33 AM
                                  Sharing between servers and phoneswumpus2017/01/26 04:25 PM
                                    Sharing between servers and phonesDavid Kanter2017/01/27 07:46 AM
                                      Any idea why Intel doesn't ship server chips with eDRAM?Mark Roulo2017/01/27 10:02 AM
                                        Any idea why Intel doesn't ship server chips with eDRAM?Per Hesselgren2017/01/28 07:49 AM
                                          Any idea why Intel doesn't ship server chips with eDRAM?Simon Farnsworth2017/01/28 08:06 AM
                                          Any idea why Intel doesn't ship server chips with eDRAM?Michael S2017/01/29 03:43 AM
                                            off die eDRAM?wumpus2017/01/29 07:57 AM
                                              off die eDRAM?anon2017/01/29 10:46 AM
                                      Sharing between servers and phonesrwessel2017/01/27 11:36 PM
                                      Sharing between servers and phonesLinus Torvalds2017/01/28 12:49 PM
                                        Sharing between servers and phonesrwessel2017/01/29 09:56 PM
                                          Sharing between servers and phonesLinus Torvalds2017/01/30 11:01 AM
                                            Sharing between servers and phonesrwessel2017/01/31 01:29 AM
                                              Sharing between servers and phonesIreland2017/01/31 08:55 AM
                                                Please keep your posts on topicDavid Kanter2017/01/31 12:32 PM
                                                  Technology that can survive in harsh conditionsIreland2017/01/31 01:51 PM
                                                    Stay on topic, you have been warnedDavid Kanter2017/01/31 07:56 PM
                                                On topic summary and my thoughts on it. Jouni Osmala2017/01/31 02:10 PM
                                                  On topic summary and my thoughts on it. Ireland2017/01/31 02:27 PM
                                              Sharing between servers and phonesLinus Torvalds2017/01/31 02:01 PM
                                                Sharing between servers and phonesLinus Torvalds2017/01/31 02:49 PM
                                                  Sharing between servers and phonesDoug S2017/01/31 03:12 PM
                                                Sharing between servers and phonesrwessel2017/01/31 06:54 PM
                                                  Sharing between servers and phonesLinus Torvalds2017/02/01 10:17 AM
                                                    Sharing between servers and phonesrwessel2017/02/02 04:40 PM
                                                  Sharing between servers and phonesjoncmu2017/02/01 02:36 PM
                                                    Sharing between servers and phonesChristian Borntraeger2017/02/02 04:46 AM
            ARM-based supercomputersRichardC2017/01/24 06:50 AM
              ARM-based supercomputersAaron Spink2017/01/24 08:20 AM
                ARM-based supercomputersdmcq2017/01/24 09:44 AM
                  ARM-based supercomputersnone2017/01/24 10:10 AM
                  ARM-based supercomputersjuanrga2017/01/29 06:33 AM
                    ARM-based supercomputershobel2017/01/30 03:35 AM
                ARM-based supercomputersRichardC2017/01/24 10:34 AM
                  ARM-based supercomputersdmcq2017/01/24 11:10 AM
                  ARM-based supercomputersAaron Spink2017/01/24 12:01 PM
                    ARM-based supercomputersRichardC2017/01/24 05:06 PM
                      ARM-based supercomputersIreland2017/01/24 06:16 PM
                      ARM-based supercomputersAaron Spink2017/01/24 08:43 PM
                    video renderingRichardC2017/01/24 06:08 PM
                      video renderingIreland2017/01/24 06:26 PM
                      video renderingAaron Spink2017/01/24 08:54 PM
                        video renderingRichardC2017/01/25 05:26 AM
                          display bandwidthRichardC2017/01/25 06:30 AM
                          video renderingIreland2017/01/25 09:11 AM
                            You can keep a coal furnace fed all the time. (NT)anon2017/01/25 04:27 PM
                              You can keep a coal furnace fed all the time.Ireland2017/01/25 04:36 PM
                                You can keep a coal furnace fed all the time.anon2017/01/27 04:29 AM
                                  You can keep a coal furnace fed all the time.Michael S2017/01/27 08:22 AM
                                    You can keep a coal furnace fed all the time.Ireland2017/01/27 11:59 AM
                                    You can keep a coal furnace fed all the time.anon2017/01/27 04:09 PM
                                      Information and Super Materials Ireland2017/01/28 12:13 PM
                            video renderingRichardC2017/01/26 12:39 PM
                              video renderingIreland2017/01/26 01:49 PM
                                video renderingIreland2017/01/26 01:58 PM
                                video renderingRichardC2017/01/26 04:24 PM
                                  video renderingIreland2017/01/26 06:09 PM
                          video renderingGabriele Svelto2017/01/25 09:16 AM
                            video renderingIreland2017/01/25 09:33 AM
                              The challenge at Pixar Ireland2017/01/25 09:40 AM
                                Pixar story & technologyRichard Cownie2017/01/25 12:36 PM
                                  Pixar story & technologyIreland2017/01/25 12:58 PM
                          video renderingAaron Spink2017/01/25 03:49 PM
                            video renderingIreland2017/01/25 04:26 PM
                            cloud network infrastructureRichardC2017/01/26 11:47 AM
                              cloud network infrastructureAaron Spink2017/01/26 08:37 PM
                ARM-based supercomputersGabriele Svelto2017/01/24 01:40 PM
                  ARM-based supercomputersAaron Spink2017/01/24 09:00 PM
        ARM-based supercomputersRichardC2017/01/22 10:45 PM
          ARM-based supercomputersGabriele Svelto2017/01/23 02:03 AM
            ARM-based supercomputersRichardC2017/01/23 07:57 AM
      ARM-based supercomputersGabriele Svelto2017/01/23 04:23 AM
    Mont blanc project and ARM HPC in generaljuanrga2017/01/29 05:42 AM
      Juan, why do you have to be such an arrogant twat to the person who provides this forum? (NT)Annoyed2017/01/30 03:32 AM
      Still idioticDavid Kanter2017/01/30 08:49 AM
        Still idioticMr. Camel2017/01/30 09:16 PM
        Still idioticjuanrga2017/02/02 09:11 AM
      Mont blanc project and ARM HPC in generalIreland2017/01/31 05:53 PM
        Four Things to Consider Ireland2017/01/31 06:11 PM
          Four Things to Consider tarlinian2017/01/31 07:38 PM
            Four Things to Consider Ireland2017/01/31 07:58 PM
              no evidence that it goes on a shipRichardC2017/02/01 07:05 AM
                no evidence that it goes on a shipIreland2017/02/02 02:57 PM
                  no evidence that it goes on a shipRichardC2017/02/03 07:04 AM
                    no evidence that it goes on a shipIreland2017/02/03 09:02 AM
              A better place to site a supercomputerAnon2017/02/01 07:57 AM
                A better place to site a supercomputerIreland2017/02/01 07:37 PM
                  A better Ireland..Anon2017/02/01 08:52 PM
                    A better Ireland..slacker2017/02/01 11:32 PM
                    A better Ireland..Ireland2017/02/02 05:06 AM
                    He is manic (NT)anonymo2017/02/02 01:21 PM
                      or a botanon2017/02/02 03:17 PM
                        or a botanonymou52017/02/02 04:46 PM
                        or a botnone2017/02/02 11:45 PM
                          or a botanon2017/02/03 10:30 AM
                        Better bot example: amanfrommars1Doug S2017/02/03 11:44 AM
            Make allowances, the man never sleeps..Anon2017/01/31 09:22 PM
        Mont blanc project and ARM HPC in generaletudiant2017/02/02 09:39 AM
          Mont blanc project and ARM HPC in generalIreland2017/02/02 12:12 PM
            Connection between two different modelsIreland2017/02/02 12:18 PM
              Connection between two different modelsetudiant2017/02/04 03:40 PM
                Connection between two different modelsRichardC2017/02/06 09:36 AM
                  Connection between two different modelsdmcq2017/02/06 11:07 AM
                  Connection between two different modelsIreland2017/02/06 12:17 PM
                    probably not the right forum ...RichardC2017/02/06 01:31 PM
                      probably not the right forum ...Ireland2017/02/06 02:53 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?