PAE sucks (Why didn't MS take advantage of PAE?)

By: Linus Torvalds (torvalds.delete@this.osdl.org), May 13, 2007 10:20 am
Room: Moderated Discussions
David W. Hess (dwhess@banishedsouls.org) on 5/13/07 wrote:
>
>I have tended to run into the physical memory space
>limitations instead of the single process memory
>limitations so I do not understand why Microsoft did not
>allow the use of PAE and more then 4GB of physical address
>space in their desktop operating systems.

PAE really really sucks.

The biggest single reason to go 64-bit is exactly
because of physical address space. Your virtual address
space needs to bea multiple of the physical one:
when you hit 1GB of RAM, 32-bit virtual memory is no longer
acceptable. You literally do need more virtual memory than
physical.

PAE turned that very simple fact on its head, and screwed
things up royally. Whoever came up with the idea was
totally incompetent, and had forgotten all the DOS HIGHMEM
pains. There's a damn good reason why we left the 286
behind, and started using 386's, instead of having HIGHMEM
crap with windows into a bigger physical space.

Repeat after me: virtual space needs to be bigger than
physical space. Not "as big". Not "smaller". It needs to
be bigger, by a factor of at least two, and that's
quite frankly pushing it, and you're much better off
having a factor of ten or more.

Anybody who doesn't get that is a moron. End of discussion.

Reasons for why you need a bigger virtual space:

- you need to map that physical memory somehow, and no,
tiny windows into the physical memory simply do not
cut it! If you cannot have normal pointers to the
physical space, it immediately means that you need to
jump through serious hoops to get there.

- you additionally need to be able to remap things
in alternate ways (ie user space) or make space for
non-memory issues (virtual page tables, IO, you name it)

Ergo, a factor-of-two is a requirement. PAE was a total
and utter disaster.

Yes, Linux supported it, and probably did so better than
anybody else. But "better than anybody else" still wasn't
very good. Because you couldn't use normal pointers to
point to arbitrary physical memory, all the memory that
couldn't be accessed directly (ie anythign that didn't fit
in the virtual address map, which also had the user
space memory in it) was basically limited to "special uses
only".

So you could allocate user pages in it, but you had huge
problems with things like internal kernel data structures,
which can be the bulk of your memory needs under some
(not that unusual) loads. Directory caches, inodes, etc
couldn't use it, and in general it meant that under Linux,
if you had more than 4GB of physical memory, you generally
ran into problems (since only 25% of memory was availble
for normal kernel stuff - the rest had to be addressed
through small holes in the tiny virtual address space).

I'm not at all surprised that Windows didn't push PAE
either. It was a total braindamage. I bet they supported
it in the server offerings just because they had to, and
I bet they did a much worse job of it than Linux did, and
the reason you can do that with servers is that the loads
are much easier, and you can expect maintainers to set
magic config entries to tweak it to make it appear
to work well for any particular load, when in reality it
is fragile as hell and works only with duct-tape and
prayers.

That kind of "duct-tape and prayers and lots of specialized
knowledge about the load" is simply not possible in
a desktop environment. Yeah, users have prayers, but they
lack the duct-tape and the knowledge to work around the
problems.

And dammit, in this age and date when almost everybody
has a gigabyte of RAM in any new machine, anybody who still
thinks that "not that many people need 64-bits" is simply
not aware of what he's speaking of.

Go back and play with HIGHMEM.SYS on a 286, and stop
blathering crap. When you've spent the last ten
years of your life working with HIGHMEM.SYS, then
you can come back and tell me that we still don't need
64 bits. Until that is the case, anybody who still doesn't
get why 64 bits is a requirement should just shut
up rather than make a total fool of himself.

So repeat after me: PAE didn't ever really fix anything.
It was a mistake. It was just a total failure, and the
result of hw engineers not understanding software.

So no, PAE does not mean that you can use more than
4GB of RAM. Even before PAE, the practical limit was around
1GB, and PAE didn't move that post a fraction of an inch!

Linus
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Rock/Tukwila rumorsmas2007/05/05 12:59 PM
  Rock/Tukwila rumorsDavid Kanter2007/05/05 02:33 PM
    Rock/Tukwila rumorsDean Kent2007/05/05 03:35 PM
      K8 vs Win64 timelineanonymous2007/05/05 06:19 PM
        Yes, I misremembered...Dean Kent2007/05/05 10:03 PM
    RockDaniel Biz√≥2007/05/06 02:34 AM
      RockDean Kent2007/05/06 07:11 AM
    Rock/Tukwila rumorsJoe2007/05/06 11:24 AM
      Rock/Tukwila rumorsDean Kent2007/05/06 11:49 AM
      Rock/Tukwila rumorsLinus Torvalds2007/05/06 12:09 PM
      Rock/Tukwila rumorsanon2007/05/07 01:32 AM
        Rock/Tukwila rumorsRakesh Malik2007/05/07 09:36 AM
          Rock/Tukwila rumorsMichael S2007/05/07 10:06 AM
          Rock/Tukwila rumorsanon2007/05/07 09:48 PM
            Rock/Tukwila rumorsRakesh Malik2007/05/08 06:45 AM
              Rock/Tukwila rumorsanon2007/05/08 05:30 PM
                Wow. (nt)Brannon2007/05/08 06:16 PM
                Rock/Tukwila rumorsrwessel2007/05/08 09:48 PM
                  Rock/Tukwila rumorsJS2007/05/08 10:07 PM
                    Rock/Tukwila rumorsJS2007/05/09 06:44 AM
                Rock/Tukwila rumorsRakesh Malik2007/05/09 05:35 AM
                  Much ado about xMichael S2007/05/09 09:39 AM
                    Call it x86-64Linus Torvalds2007/05/09 10:27 AM
                      (i)AMD64Michael S2007/05/09 12:16 PM
                        (i)AMD64Linus Torvalds2007/05/09 12:29 PM
                          (i)AMD64Groo2007/05/09 04:45 PM
                          TIFNAAanonymous2007/05/09 05:49 PM
                            Inspired by FYR Macedonia? (NT)Michael S2007/05/09 11:21 PM
                              More likely...rwessel2007/05/10 12:39 AM
                            TIFNAAGabriele Svelto2007/05/09 11:57 PM
                          (i)AMD64James2007/05/10 02:27 AM
                        i86Dean Kent2007/05/09 12:30 PM
                        (i)AMD64Max2007/05/09 01:28 PM
                          wide86? long86?hobold2007/05/10 05:05 AM
                            x87 perhaps, it is one more. :) (NT)Groo2007/05/10 05:50 AM
                              x86+Dean Kent2007/05/10 08:44 AM
                                Does it really matter?Doug Siebert2007/05/10 09:10 AM
                                  let's stay with x86-64 for now, pleaseMarcin Niewiadomski2007/05/10 11:50 AM
                                    let's stay with x86-64 for now, pleaseDean Kent2007/05/11 06:11 AM
                                      let's stay with x86-64 for now, pleaserwessel2007/05/11 02:46 PM
                                        let's stay with x86-64 for now, pleaseDean Kent2007/05/11 06:03 PM
                                          let's stay with x86-64 for now, pleaseMichael S2007/05/12 10:49 AM
                                            let's stay with x86-64 for now, pleaseDean Kent2007/05/12 01:05 PM
                                              let's stay with x86-64 for now, pleaseMichael S2007/05/12 01:25 PM
                                                let's stay with x86-64 for now, pleaseDean Kent2007/05/12 03:39 PM
                                                  let's stay with x86-64 for now, pleaseJasonB2007/05/13 07:43 AM
                                                    client consolidationMichael S2007/05/13 08:37 AM
                                                  let's stay with x86-64 for now, pleaseTzvetan Mikov2007/05/13 03:44 PM
                                                let's stay with x86-64 for now, pleaserwessel2007/05/14 02:42 PM
                                      What's your point?Doug Siebert2007/05/11 02:56 PM
                                        What's your point?Linus Torvalds2007/05/11 04:15 PM
                                          What's your point?Doug Siebert2007/05/13 03:11 PM
                                            What's your point?Dean Kent2007/05/13 07:04 PM
                                              What's your point?JasonB2007/05/14 02:06 AM
                                                What's your point?Dean Kent2007/05/14 07:20 AM
                                                  What's your point?JasonB2007/05/14 04:35 PM
                                                  What's your point?JasonB2007/05/14 07:35 PM
                                                    What's your point?Dean Kent2007/05/14 08:12 PM
                                        What's your point?Dean Kent2007/05/11 06:06 PM
                                        What's your point?Stephen H2007/05/13 01:55 AM
                                          Why didn't MS take advantage of PAE?David W. Hess2007/05/13 08:37 AM
                                            PAE sucks (Why didn't MS take advantage of PAE?)Linus Torvalds2007/05/13 10:20 AM
                                              PAE sucks (Why didn't MS take advantage of PAE?)Dean Kent2007/05/13 10:49 AM
                                              PAE sucks (Why didn't MS take advantage of PAE?)David W. Hess2007/05/13 12:37 PM
                                              > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 01:44 PM
                                                > 1 GB RAM on a 32-bit systemS. Rao2007/05/13 03:00 PM
                                                  > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 05:32 PM
                                                    > 1 GB RAM on a 32-bit systemS. Rao2007/05/14 12:19 AM
                                                > 1 GB RAM on a 32-bit systemLinus Torvalds2007/05/13 03:46 PM
                                                  > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 05:23 PM
                                                  > 1 GB RAM on a 32-bit systemJasonB2007/05/13 06:37 PM
                                                    Windows manages memory differentlyTzvetan Mikov2007/05/13 08:31 PM
                                                      Windows manages memory differentlyJasonB2007/05/14 01:50 AM
                                                        Windows manages memory differentlyTzvetan Mikov2007/05/14 08:56 AM
                                                          Windows manages memory differentlyrwessel2007/05/14 03:40 PM
                                                            Windows manages memory differentlyDavid W. Hess2007/05/14 04:07 PM
                                                              Windows manages memory differentlyrwessel2007/05/14 04:51 PM
                                                            Windows manages memory differentlyTzvetan Mikov2007/05/14 05:40 PM
                                                              Windows manages memory differentlyrwessel2007/05/14 06:09 PM
                                                      Windows manages memory differentlyHoward Chu2007/05/14 11:17 AM
                                                        Windows manages memory differentlyJukka Larja2007/05/14 11:30 AM
                                                        Windows manages memory differentlyTzvetan Mikov2007/05/14 01:54 PM
                                                          Windows manages memory differentlyHoward Chu2007/05/15 03:35 AM
                                                            Windows manages memory differentlyGroo2007/05/15 07:34 AM
                                                      Anyone know what OS X (10.4, Intel, desktop) does?Matt Sayler2007/05/15 06:23 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?Wes Felter2007/05/15 08:37 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?Anonymous2007/05/15 10:49 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?anon22007/05/15 07:13 PM
                                              PAE sucks (Why didn't MS take advantage of PAE?)Paul2007/05/13 03:40 PM
                                                PAE sucks (Why didn't MS take advantage of PAE?)Peter Arremann2007/05/13 05:38 PM
                                                  PAE sucks (Why didn't MS take advantage of PAE?)Henrik S2007/05/14 03:31 AM
                                              The fragility of your argumentslacker2007/05/13 03:56 PM
                                                The fragility of your argumentnick2007/05/13 05:42 PM
                                                The fragility of your argumentHoward Chu2007/05/14 02:52 AM
                                                  The fragility of your argumentDean Kent2007/05/14 09:19 AM
                                                The fragility of your argumentanon22007/05/14 08:26 AM
                                                  The fragility of your argumentTzvetan Mikov2007/05/14 09:01 AM
                                                  The fragility of your argumentDean Kent2007/05/14 09:16 AM
                                                    The fragility of your argumentLinus Torvalds2007/05/14 11:57 AM
                                                      The fragility of your argumentJasonB2007/05/14 04:48 PM
                                                        The fragility of your argumentDean Kent2007/05/14 07:36 PM
                                                          The fragility of your argumentRicardo B2007/05/16 02:40 AM
                                                            The fragility of your argumentDean Kent2007/05/16 03:32 AM
                                                              The fragility of your argumentRicardo B2007/05/16 06:41 AM
                                                                PSRicardo B2007/05/16 06:50 AM
                                                                The fragility of your argumentDean Kent2007/05/16 09:07 AM
                                                                  Modern web browsingS. Rao2007/05/16 09:16 AM
                                                                    Aha!Dean Kent2007/05/16 09:27 AM
                                                                      Aha!Dean Kent2007/05/16 09:32 AM
                                                                        Aha!S. Rao2007/05/16 10:34 AM
                                                                  The fragility of your argumentRicardo B2007/05/16 10:00 AM
                                                      The fragility of your argumentVincent Diepeveen2007/05/16 10:10 AM
                                                        The fragility of your argumentPaul2007/05/16 03:01 PM
                                                          The fragility of your argumentVincent Diepeveen2007/05/17 03:05 AM
                                                    The fragility of your argumentanon22007/05/15 01:35 AM
                                                    Splits vs page allocations?Matt Sayler2007/05/15 07:33 AM
                                          What's your point?Michael S2007/05/13 08:55 AM
                                            What's your point?anonymous2007/05/13 11:08 AM
                                              What's your point?Michael S2007/05/13 11:31 AM
                                      let's stay with x86-64 for now, pleaseJasonB2007/05/13 07:16 AM
                                x864 =) (NT)some12007/05/15 03:03 AM
    Rock/Tukwila rumorsIntelUser20002007/05/06 02:27 PM
  Rock/Tukwila rumorsm2007/05/13 08:05 AM
  Rock/Tukwila rumorsmas2007/05/15 09:40 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?