Sharing between servers and phones

By: rwessel (robertwessel.delete@this.yahoo.com), January 31, 2017 5:54 pm
Room: Moderated Discussions
Linus Torvalds (torvalds.delete@this.linux-foundation.org) on January 31, 2017 1:01 pm wrote:
> rwessel (robertwessel.delete@this.yahoo.com) on January 31, 2017 12:29 am wrote:
> > Linus Torvalds (torvalds.delete@this.linux-foundation.org) on January 30, 2017 10:01 am wrote:
> >
> > > Yeah, I'm, aware of the high-level details, the oddities are in the low-level issues
> > > (very strange TLB flushing, iirc, due to some odd dirty bit handling and other rules,
> > > I only see the patches flow by, I've never used it or looked all that closely).
> >
> >
> > Well, given Linux's heritage, the dirty and referenced bits on S/360 are certainly odd
> > (as they're attached to the physical page and not the TLB entry). Having some of the protection
> > bits attached to the physical page probably also annoys the Linux kernel...
>
> So the reason the dirty bit was somewhat annoying is that we actually do maintain it on a physical
> page basis as well in the kernel (because in the end, when you do IO, you don't ask yourself "is
> this virtual address dirty?" - you are writing out a particular physical copy, of course).
>
> So I see where the hardware designer comes from: he's trying to make it easy for the OS.
> And it probably did help MVS, which presumably was the driving factor for that design.


As a first order approximation, everything in the S/360 line has been done to support OS/MVS/zOS. Sure there have been exceptions (not least a number of non-trivial ISA additions to support Linux), but that's certainly been the primary driver for the last 50 years. IBM has even tried to kill VM and VSE (DOS) a number of times over the years.


> But the fact that we do end up maintaining a physical dirty bit doesn't make the need for the per-virtual-mapping
> dirty bit go away: you still end up having several places that want to know if a write has been done through
> a particular mapping. It's not all that common (so the s390 people actually got away without a proper dirty bit
> per page table entry for a long while), but there really are cases where it is more than just useful.


They've gotten away with it since 1965 when the 360/67 shipped (although there were some changes, almost all of the virtual memory/machine functions in the 67 are the basis of what was added for S/370).

But I'm curious, when is it handy to know which PTE dirtied (or referenced) a page? All I'm coming up with is pretty contrived. My best idea is to generate data to decide to migrate a page to a better NUMA node (or conversely, manage thread-to-node affinity).


> In fact, a per-virtual-mapping dirty bit is so useful that we end up not just tracking the
> usual dirty state (that most CPU's give us in hardware), we end up having a second sw-only
> dirty state that we call the "soft dirty" state for tracking things like "has this page been
> changed since we last looked at it", which is useful for things like checkpointing.


MVS does the same at the physical page level, for similar reasons. For the reference bit as well.


> > But yes, the biggest annoyance was just that we share the VM(code) No disagreement.


Of course, the number of Linux images zArch in use is unlikely to drive major internal changes to the kernel.

BTW, I'm sure your post went wonky after the (code) (where you used angle brackets), probably up to some point where he found something else that looked like a tag. Never put anything that looks remotely like a tag here, unless you really mean one of the basic HTML tags that are supported. Or at least edit the angle brackets to ampersand-< and ampersand-gt; (although there seems to be a bug in the handling of those as well if the forum software get ahold of them more than once). "Not my favorite interface ever". Check.


> > Although I personally find that to be more of a SW design issue: people
> > > who design their calling conventions to be about cross-process boundaries are crazy.
> >
> > It has its moment, though. As an example, if you've done a DB2 SELECT, each of the subsequent
> > FETCH's is just a wrapper around a PC (call-gate to DB2's address space). DB2, can then deposit
> > the row right into your address space (there being pretty good cross address space support in
> > zArch). Other solutions can reduce the number of cross address space calls (for example, buffering
> > multiple rows), but at the expense of complexity and copies of the data.
>
> As an OS person, I've often wanted the ability to just have the hardware write to different address
> spaces. Screw segments, I'd like to just have access to multiple address spaces. Sparc kind of had it
> (user vs kernel vs IO address space or whatever), but that doesn't handle the multi-process case.
>
> But since almost nobody has that hardware support, it's useless to a portable OS, and we end
> up doing the "copy from one address space to another" by walking the page tables by hand.
>
> Even on s390.


I'm surprised that you can't make use of some of that under the hood for zArch. I assume that somewhere there's a "copy space-A/address-B to space-C/address-D for length E" function. If the (real) page tables are available, it would be the work of a few instructions to let the hardware do the cross-address-space move. OTOH, I'm sure I'm not the first person to think of that...


> It turns out that specialized architecture features are almost never useful.
> Which makes for much less interesting processor architecture design ("you have
> to look like everybody else"), but much more useful actual processors.


It is a bit depressing - we now have a fairly portable OS, but now all the processors are the same. But less excitement at the lower levels of the stack lets us concentrate on getting actual useful work done at the top of the stack. But I do worry that all that consistency at ISA and OS level is hampering (disruptive types of) innovation there, which potentially is building up a problem for the future.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Cray to Develop ARM-based Isambard Supercomputer for UK Met Officegallier22017/01/20 07:48 AM
  Cray to Develop ARM-based Isambard Supercomputer for UK Met Officegallier22017/01/20 07:48 AM
    Ignore second post (NT)gallier22017/01/20 07:49 AM
  ARM-based supercomputersDavid Kanter2017/01/20 02:55 PM
    ARM-based supercomputersMaynard Handley2017/01/20 08:55 PM
      ARM-based supercomputers lolRobert David Graham2017/01/21 05:34 PM
        ARM-based supercomputers lolnone2017/01/21 06:44 PM
          ARM-based supercomputers lolMaynard Handley2017/01/21 07:36 PM
            ARM-based supercomputers lolMichael S2017/01/22 02:07 AM
            What did you learn?Innocent Bystander2017/01/22 08:26 AM
          ARM-based supercomputers lolanon2017/01/22 02:24 AM
            ARM-based supercomputers lolGabriele Svelto2017/01/23 12:53 AM
              ARM-based supercomputers lolanon2017/01/23 06:40 AM
            ARM-based supercomputers loljuanrga2017/01/29 05:01 AM
              ARM-based supercomputers lolanon2017/01/29 09:35 AM
        ARM-based supercomputers lolBrendan2017/01/21 07:36 PM
         Dumb belief? ROFL (NT)juanrga2017/01/22 02:59 AM
          Dumb belief? ROFLwumpus2017/01/22 11:31 AM
            Dumb belief? ROFLjuanrga2017/01/29 05:13 AM
        ARM-based supercomputers lolDavid Kanter2017/01/22 08:20 AM
    ARM-based supercomputersRichardC2017/01/22 07:54 AM
      ARM-based supercomputersAaron Spink2017/01/22 12:03 PM
        ARM-based supercomputersNiels Jørgen Kruse2017/01/22 12:16 PM
          ARM-based supercomputersAaron Spink2017/01/23 07:01 PM
            ARM-based supercomputersNiels Jørgen Kruse2017/01/23 10:41 PM
              ARM-based supercomputerstarlinian2017/01/24 08:58 AM
                ARM-based supercomputersMichael S2017/01/24 09:05 AM
                  ARM-based supercomputersAaron Spink2017/01/24 10:38 AM
        ARM-based supercomputersdmcq2017/01/22 01:03 PM
          A73 does not have L1 ECCDavid Kanter2017/01/22 09:50 PM
            A73 does not have L1 ECCdmcq2017/01/23 03:32 PM
              A73 does not have L1 ECCMichael S2017/01/23 03:50 PM
              A73 does not have L1 ECCdmcq2017/01/24 08:15 AM
                A73 does not have L1 ECCMichael S2017/01/24 08:47 AM
          ARM SOCs with ECC DRAMMichael S2017/01/23 03:17 AM
          ARM-based supercomputersAaron Spink2017/01/23 07:02 PM
            X-Gene 3 supports 1TB of memoryvvid2017/01/24 01:55 AM
              X-Gene 3 supports 1TB of memoryWilco2017/01/24 02:13 AM
                You seem to be confusing "SoC" with "CPU core" (NT)Anon2017/01/24 03:12 AM
                X-Gene 3 supports 1TB of memoryMichael S2017/01/24 05:26 AM
                  X-Gene 3 supports 1TB of memorydmcq2017/01/24 08:28 AM
                X-Gene 3 supports 1TB of memoryAaron Spink2017/01/24 06:46 AM
                  X-Gene 3 supports 1TB of memoryWilco2017/01/24 03:10 PM
                    X-Gene 3 supports 1TB of memoryDavid Kanter2017/01/24 07:31 PM
                X-Gene 3 supports 1TB of memorySimon Farnsworth2017/01/26 02:23 AM
              X-Gene 3 supports 1TB of memoryAaron Spink2017/01/24 06:44 AM
                X-Gene 3 supports 1TB of memoryAnon2017/01/24 01:54 PM
                  X-Gene 3 supports 1TB of memoryWilco2017/01/24 03:07 PM
                    X-Gene 3 supports 1TB of memoryDavid Kanter2017/01/24 07:34 PM
                      X-Gene 3 supports 1TB of memorynone2017/01/24 10:48 PM
                        X-Gene 3 supports 1TB of memoryMichael S2017/01/25 01:32 AM
                        X-Gene 3 supports 1TB of memoryDavid Hess2017/01/25 09:47 PM
                          X-Gene 3 supports 1TB of memoryMichael S2017/01/26 12:55 AM
                            Really that different?Daniel B2017/01/26 03:37 AM
                              Really that different?none2017/01/26 05:39 AM
                                Really that different?itsmydamnation2017/01/26 03:55 PM
                                  Really that different?none2017/01/26 11:14 PM
                      X-Gene 3 supports 1TB of memoryWilco2017/01/25 03:22 AM
                        Sharing between servers and phonesDavid Kanter2017/01/25 07:15 AM
                          Sharing between servers and phonesWilco2017/01/25 04:41 PM
                            Sharing between servers and phonesDavid Kanter2017/01/25 06:10 PM
                              Sharing between servers and phonesGabriele Svelto2017/01/26 03:15 AM
                                Sharing between servers and phonesDavid Kanter2017/01/26 07:33 AM
                                  Sharing between servers and phoneswumpus2017/01/26 03:25 PM
                                    Sharing between servers and phonesDavid Kanter2017/01/27 06:46 AM
                                      Any idea why Intel doesn't ship server chips with eDRAM?Mark Roulo2017/01/27 09:02 AM
                                        Any idea why Intel doesn't ship server chips with eDRAM?Per Hesselgren2017/01/28 06:49 AM
                                          Any idea why Intel doesn't ship server chips with eDRAM?Simon Farnsworth2017/01/28 07:06 AM
                                          Any idea why Intel doesn't ship server chips with eDRAM?Michael S2017/01/29 02:43 AM
                                            off die eDRAM?wumpus2017/01/29 06:57 AM
                                              off die eDRAM?anon2017/01/29 09:46 AM
                                      Sharing between servers and phonesrwessel2017/01/27 10:36 PM
                                      Sharing between servers and phonesLinus Torvalds2017/01/28 11:49 AM
                                        Sharing between servers and phonesrwessel2017/01/29 08:56 PM
                                          Sharing between servers and phonesLinus Torvalds2017/01/30 10:01 AM
                                            Sharing between servers and phonesrwessel2017/01/31 12:29 AM
                                              Sharing between servers and phonesIreland2017/01/31 07:55 AM
                                                Please keep your posts on topicDavid Kanter2017/01/31 11:32 AM
                                                  Technology that can survive in harsh conditionsIreland2017/01/31 12:51 PM
                                                    Stay on topic, you have been warnedDavid Kanter2017/01/31 06:56 PM
                                                On topic summary and my thoughts on it. Jouni Osmala2017/01/31 01:10 PM
                                                  On topic summary and my thoughts on it. Ireland2017/01/31 01:27 PM
                                              Sharing between servers and phonesLinus Torvalds2017/01/31 01:01 PM
                                                Sharing between servers and phonesLinus Torvalds2017/01/31 01:49 PM
                                                  Sharing between servers and phonesDoug S2017/01/31 02:12 PM
                                                Sharing between servers and phonesrwessel2017/01/31 05:54 PM
                                                  Sharing between servers and phonesLinus Torvalds2017/02/01 09:17 AM
                                                    Sharing between servers and phonesrwessel2017/02/02 03:40 PM
                                                  Sharing between servers and phonesjoncmu2017/02/01 01:36 PM
                                                    Sharing between servers and phonesChristian Borntraeger2017/02/02 03:46 AM
            ARM-based supercomputersRichardC2017/01/24 05:50 AM
              ARM-based supercomputersAaron Spink2017/01/24 07:20 AM
                ARM-based supercomputersdmcq2017/01/24 08:44 AM
                  ARM-based supercomputersnone2017/01/24 09:10 AM
                  ARM-based supercomputersjuanrga2017/01/29 05:33 AM
                    ARM-based supercomputershobel2017/01/30 02:35 AM
                ARM-based supercomputersRichardC2017/01/24 09:34 AM
                  ARM-based supercomputersdmcq2017/01/24 10:10 AM
                  ARM-based supercomputersAaron Spink2017/01/24 11:01 AM
                    ARM-based supercomputersRichardC2017/01/24 04:06 PM
                      ARM-based supercomputersIreland2017/01/24 05:16 PM
                      ARM-based supercomputersAaron Spink2017/01/24 07:43 PM
                    video renderingRichardC2017/01/24 05:08 PM
                      video renderingIreland2017/01/24 05:26 PM
                      video renderingAaron Spink2017/01/24 07:54 PM
                        video renderingRichardC2017/01/25 04:26 AM
                          display bandwidthRichardC2017/01/25 05:30 AM
                          video renderingIreland2017/01/25 08:11 AM
                            You can keep a coal furnace fed all the time. (NT)anon2017/01/25 03:27 PM
                              You can keep a coal furnace fed all the time.Ireland2017/01/25 03:36 PM
                                You can keep a coal furnace fed all the time.anon2017/01/27 03:29 AM
                                  You can keep a coal furnace fed all the time.Michael S2017/01/27 07:22 AM
                                    You can keep a coal furnace fed all the time.Ireland2017/01/27 10:59 AM
                                    You can keep a coal furnace fed all the time.anon2017/01/27 03:09 PM
                                      Information and Super Materials Ireland2017/01/28 11:13 AM
                            video renderingRichardC2017/01/26 11:39 AM
                              video renderingIreland2017/01/26 12:49 PM
                                video renderingIreland2017/01/26 12:58 PM
                                video renderingRichardC2017/01/26 03:24 PM
                                  video renderingIreland2017/01/26 05:09 PM
                          video renderingGabriele Svelto2017/01/25 08:16 AM
                            video renderingIreland2017/01/25 08:33 AM
                              The challenge at Pixar Ireland2017/01/25 08:40 AM
                                Pixar story & technologyRichard Cownie2017/01/25 11:36 AM
                                  Pixar story & technologyIreland2017/01/25 11:58 AM
                          video renderingAaron Spink2017/01/25 02:49 PM
                            video renderingIreland2017/01/25 03:26 PM
                            cloud network infrastructureRichardC2017/01/26 10:47 AM
                              cloud network infrastructureAaron Spink2017/01/26 07:37 PM
                ARM-based supercomputersGabriele Svelto2017/01/24 12:40 PM
                  ARM-based supercomputersAaron Spink2017/01/24 08:00 PM
        ARM-based supercomputersRichardC2017/01/22 09:45 PM
          ARM-based supercomputersGabriele Svelto2017/01/23 01:03 AM
            ARM-based supercomputersRichardC2017/01/23 06:57 AM
      ARM-based supercomputersGabriele Svelto2017/01/23 03:23 AM
    Mont blanc project and ARM HPC in generaljuanrga2017/01/29 04:42 AM
      Juan, why do you have to be such an arrogant twat to the person who provides this forum? (NT)Annoyed2017/01/30 02:32 AM
      Still idioticDavid Kanter2017/01/30 07:49 AM
        Still idioticMr. Camel2017/01/30 08:16 PM
        Still idioticjuanrga2017/02/02 08:11 AM
      Mont blanc project and ARM HPC in generalIreland2017/01/31 04:53 PM
        Four Things to Consider Ireland2017/01/31 05:11 PM
          Four Things to Consider tarlinian2017/01/31 06:38 PM
            Four Things to Consider Ireland2017/01/31 06:58 PM
              no evidence that it goes on a shipRichardC2017/02/01 06:05 AM
                no evidence that it goes on a shipIreland2017/02/02 01:57 PM
                  no evidence that it goes on a shipRichardC2017/02/03 06:04 AM
                    no evidence that it goes on a shipIreland2017/02/03 08:02 AM
              A better place to site a supercomputerAnon2017/02/01 06:57 AM
                A better place to site a supercomputerIreland2017/02/01 06:37 PM
                  A better Ireland..Anon2017/02/01 07:52 PM
                    A better Ireland..slacker2017/02/01 10:32 PM
                    A better Ireland..Ireland2017/02/02 04:06 AM
                    He is manic (NT)anonymo2017/02/02 12:21 PM
                      or a botanon2017/02/02 02:17 PM
                        or a botanonymou52017/02/02 03:46 PM
                        or a botnone2017/02/02 10:45 PM
                          or a botanon2017/02/03 09:30 AM
                        Better bot example: amanfrommars1Doug S2017/02/03 10:44 AM
            Make allowances, the man never sleeps..Anon2017/01/31 08:22 PM
        Mont blanc project and ARM HPC in generaletudiant2017/02/02 08:39 AM
          Mont blanc project and ARM HPC in generalIreland2017/02/02 11:12 AM
            Connection between two different modelsIreland2017/02/02 11:18 AM
              Connection between two different modelsetudiant2017/02/04 02:40 PM
                Connection between two different modelsRichardC2017/02/06 08:36 AM
                  Connection between two different modelsdmcq2017/02/06 10:07 AM
                  Connection between two different modelsIreland2017/02/06 11:17 AM
                    probably not the right forum ...RichardC2017/02/06 12:31 PM
                      probably not the right forum ...Ireland2017/02/06 01:53 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?