> 1 GB RAM on a 32-bit system

By: Linus Torvalds (torvalds.delete@this.osdl.org), May 13, 2007 3:46 pm
Room: Moderated Discussions
Tzvetan Mikov (tzvetanmi@yahoo.com) on 5/13/07 wrote:
>
>Can you please elaborate on the practical 1 GB limit a
>little further ?

Since the i386 has a 32-bit total limit on virtual
address space (the segmentation doesn't extend that, it
all works within the 4GB limit), and since user space needs
to be mapped in that area too, you effectively have the
requirement that

- user space virtual memory + kernel virtual memory = 4GB

Some other architectures effectively have separate virtual
memory spaces for user and kernel space, so they don't
necessarily need to be 4GB combined, they can be separately
4GB each. On x86, that isn't practical (you can emulate it
by switching page tables around a lot, and yes, it's been
done, but despite the fact that Intel makes TLB lookups
very fast, it's quite nasty for performance!)

And the thing is, user space really does want its own
virtual memory too. Under Linux, you can choose how to
split up the 4GB address space, but on the whole, you'd
actually prefer to give user space as much as possible,
since

- user space tends to use reasonably sparse virtual
address spaces (loadable libraries in one place, code,
data, stack in other areas etc)

- quite often, you have one "major" app, which wants to
use as much of the physical memory as possible, eg a
database, so you actually want user space to map as much
of the physical memory too!

So Linux defaults to giving user space 3GB, and just 1GB
for the kernel. That's basically the "user space is more
important in the end" version. I think WinXP does a 2:2GB
split.

So basically, the kernel really never has 4GB of virtual
memory. It has 1GB (possibly 2GB).

It also needs to map in PCI memory-mapped stuff etc, so in
practice, it's actually less than that. And then the kernel
will need some more virtual memory for its own use, eg
things like loading modules, and just for the window to do
physical addresses bigger than it can map directly.

So in practice, in Linux, when you had more than about
920MB (I forget the exact number), you ended up maxing out
what the kernel could map directly.

And once you max that out, think about what happens for
a moment...

Once you start having to access physical memory through
some kind of HIGHMEM.SYS window (which is what PAE is),
and cannot map it all, you can no longer keep normal
pointers to such memory around. Instead, you are basically
using a really ugly and strange segmented architecture,
where you keep some kind of indirect pointer, and every
time before you use it, you have to map it into the virtual
address space, access it, and then unmap it again.

So performance plummets, and the code actually gets really
nasty too, so you don't actually use the high memory for
any random data, you only use it for special stuff. As
an example, you'd use it for disk caching (that's what 90%
of all HIGHMEM.SYS usage was too - a lot of people just
set it all - or at least a big chunk of it - aside as a
harddisk cache, because so few programs could use it very
well for anything else).

Linux used it for more than disk caching, but the point
is, since it's special memory that you have to access
through a small window, or can just map into user space,
it's not generally usable any more.

And yes, there were serious problems. In theory, you
can have 64GB of RAM with Linux on a PAE x86 box. In
practice, it seldom worked very well past the 4GB mark,
so PAE itself was almost totally useless. The reason was
that once you had more than 4GB of memory, you usually
had filled out a large chunk of the easily accessible
memory with just all the data structures to keep track
of the rest of memory (that's exaggerated, but it's not
entirely off).

So you literally were in this situation where you had
tons of memory that could be used for some things,
but you ran out of memory for other basic bookkeeping
stuff.

So some people used 16GB of RAM, and by limiting their
working set to cases where the HIGHMEM stuff was fine,
and they didn't need a whole lot of other memory, they
made it work. But the code was (and is) grotty and pretty
disgusting, and performs much worse than just allowing
anything to use any of the RAM.

And the limit literally isn't the 4GB mark.

>So, in practical terms, is there benefit from using
>64-bit OS on a machine with, lets say, 2-3 GB RAM ?

Absolutely. No question what-so-ever. If you have 2GB
of RAM on a 64-bit architecture, you can access it all
easily in the kernel, and none of it is limited to just
some special use. And a single big process can also use
it effectively at the same time, so a database can
actually have it all mapped without having to play
windowing games in user space either.

Linus
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Rock/Tukwila rumorsmas2007/05/05 12:59 PM
  Rock/Tukwila rumorsDavid Kanter2007/05/05 02:33 PM
    Rock/Tukwila rumorsDean Kent2007/05/05 03:35 PM
      K8 vs Win64 timelineanonymous2007/05/05 06:19 PM
        Yes, I misremembered...Dean Kent2007/05/05 10:03 PM
    RockDaniel Biz√≥2007/05/06 02:34 AM
      RockDean Kent2007/05/06 07:11 AM
    Rock/Tukwila rumorsJoe2007/05/06 11:24 AM
      Rock/Tukwila rumorsDean Kent2007/05/06 11:49 AM
      Rock/Tukwila rumorsLinus Torvalds2007/05/06 12:09 PM
      Rock/Tukwila rumorsanon2007/05/07 01:32 AM
        Rock/Tukwila rumorsRakesh Malik2007/05/07 09:36 AM
          Rock/Tukwila rumorsMichael S2007/05/07 10:06 AM
          Rock/Tukwila rumorsanon2007/05/07 09:48 PM
            Rock/Tukwila rumorsRakesh Malik2007/05/08 06:45 AM
              Rock/Tukwila rumorsanon2007/05/08 05:30 PM
                Wow. (nt)Brannon2007/05/08 06:16 PM
                Rock/Tukwila rumorsrwessel2007/05/08 09:48 PM
                  Rock/Tukwila rumorsJS2007/05/08 10:07 PM
                    Rock/Tukwila rumorsJS2007/05/09 06:44 AM
                Rock/Tukwila rumorsRakesh Malik2007/05/09 05:35 AM
                  Much ado about xMichael S2007/05/09 09:39 AM
                    Call it x86-64Linus Torvalds2007/05/09 10:27 AM
                      (i)AMD64Michael S2007/05/09 12:16 PM
                        (i)AMD64Linus Torvalds2007/05/09 12:29 PM
                          (i)AMD64Groo2007/05/09 04:45 PM
                          TIFNAAanonymous2007/05/09 05:49 PM
                            Inspired by FYR Macedonia? (NT)Michael S2007/05/09 11:21 PM
                              More likely...rwessel2007/05/10 12:39 AM
                            TIFNAAGabriele Svelto2007/05/09 11:57 PM
                          (i)AMD64James2007/05/10 02:27 AM
                        i86Dean Kent2007/05/09 12:30 PM
                        (i)AMD64Max2007/05/09 01:28 PM
                          wide86? long86?hobold2007/05/10 05:05 AM
                            x87 perhaps, it is one more. :) (NT)Groo2007/05/10 05:50 AM
                              x86+Dean Kent2007/05/10 08:44 AM
                                Does it really matter?Doug Siebert2007/05/10 09:10 AM
                                  let's stay with x86-64 for now, pleaseMarcin Niewiadomski2007/05/10 11:50 AM
                                    let's stay with x86-64 for now, pleaseDean Kent2007/05/11 06:11 AM
                                      let's stay with x86-64 for now, pleaserwessel2007/05/11 02:46 PM
                                        let's stay with x86-64 for now, pleaseDean Kent2007/05/11 06:03 PM
                                          let's stay with x86-64 for now, pleaseMichael S2007/05/12 10:49 AM
                                            let's stay with x86-64 for now, pleaseDean Kent2007/05/12 01:05 PM
                                              let's stay with x86-64 for now, pleaseMichael S2007/05/12 01:25 PM
                                                let's stay with x86-64 for now, pleaseDean Kent2007/05/12 03:39 PM
                                                  let's stay with x86-64 for now, pleaseJasonB2007/05/13 07:43 AM
                                                    client consolidationMichael S2007/05/13 08:37 AM
                                                  let's stay with x86-64 for now, pleaseTzvetan Mikov2007/05/13 03:44 PM
                                                let's stay with x86-64 for now, pleaserwessel2007/05/14 02:42 PM
                                      What's your point?Doug Siebert2007/05/11 02:56 PM
                                        What's your point?Linus Torvalds2007/05/11 04:15 PM
                                          What's your point?Doug Siebert2007/05/13 03:11 PM
                                            What's your point?Dean Kent2007/05/13 07:04 PM
                                              What's your point?JasonB2007/05/14 02:06 AM
                                                What's your point?Dean Kent2007/05/14 07:20 AM
                                                  What's your point?JasonB2007/05/14 04:35 PM
                                                  What's your point?JasonB2007/05/14 07:35 PM
                                                    What's your point?Dean Kent2007/05/14 08:12 PM
                                        What's your point?Dean Kent2007/05/11 06:06 PM
                                        What's your point?Stephen H2007/05/13 01:55 AM
                                          Why didn't MS take advantage of PAE?David W. Hess2007/05/13 08:37 AM
                                            PAE sucks (Why didn't MS take advantage of PAE?)Linus Torvalds2007/05/13 10:20 AM
                                              PAE sucks (Why didn't MS take advantage of PAE?)Dean Kent2007/05/13 10:49 AM
                                              PAE sucks (Why didn't MS take advantage of PAE?)David W. Hess2007/05/13 12:37 PM
                                              > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 01:44 PM
                                                > 1 GB RAM on a 32-bit systemS. Rao2007/05/13 03:00 PM
                                                  > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 05:32 PM
                                                    > 1 GB RAM on a 32-bit systemS. Rao2007/05/14 12:19 AM
                                                > 1 GB RAM on a 32-bit systemLinus Torvalds2007/05/13 03:46 PM
                                                  > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 05:23 PM
                                                  > 1 GB RAM on a 32-bit systemJasonB2007/05/13 06:37 PM
                                                    Windows manages memory differentlyTzvetan Mikov2007/05/13 08:31 PM
                                                      Windows manages memory differentlyJasonB2007/05/14 01:50 AM
                                                        Windows manages memory differentlyTzvetan Mikov2007/05/14 08:56 AM
                                                          Windows manages memory differentlyrwessel2007/05/14 03:40 PM
                                                            Windows manages memory differentlyDavid W. Hess2007/05/14 04:07 PM
                                                              Windows manages memory differentlyrwessel2007/05/14 04:51 PM
                                                            Windows manages memory differentlyTzvetan Mikov2007/05/14 05:40 PM
                                                              Windows manages memory differentlyrwessel2007/05/14 06:09 PM
                                                      Windows manages memory differentlyHoward Chu2007/05/14 11:17 AM
                                                        Windows manages memory differentlyJukka Larja2007/05/14 11:30 AM
                                                        Windows manages memory differentlyTzvetan Mikov2007/05/14 01:54 PM
                                                          Windows manages memory differentlyHoward Chu2007/05/15 03:35 AM
                                                            Windows manages memory differentlyGroo2007/05/15 07:34 AM
                                                      Anyone know what OS X (10.4, Intel, desktop) does?Matt Sayler2007/05/15 06:23 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?Wes Felter2007/05/15 08:37 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?Anonymous2007/05/15 10:49 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?anon22007/05/15 07:13 PM
                                              PAE sucks (Why didn't MS take advantage of PAE?)Paul2007/05/13 03:40 PM
                                                PAE sucks (Why didn't MS take advantage of PAE?)Peter Arremann2007/05/13 05:38 PM
                                                  PAE sucks (Why didn't MS take advantage of PAE?)Henrik S2007/05/14 03:31 AM
                                              The fragility of your argumentslacker2007/05/13 03:56 PM
                                                The fragility of your argumentnick2007/05/13 05:42 PM
                                                The fragility of your argumentHoward Chu2007/05/14 02:52 AM
                                                  The fragility of your argumentDean Kent2007/05/14 09:19 AM
                                                The fragility of your argumentanon22007/05/14 08:26 AM
                                                  The fragility of your argumentTzvetan Mikov2007/05/14 09:01 AM
                                                  The fragility of your argumentDean Kent2007/05/14 09:16 AM
                                                    The fragility of your argumentLinus Torvalds2007/05/14 11:57 AM
                                                      The fragility of your argumentJasonB2007/05/14 04:48 PM
                                                        The fragility of your argumentDean Kent2007/05/14 07:36 PM
                                                          The fragility of your argumentRicardo B2007/05/16 02:40 AM
                                                            The fragility of your argumentDean Kent2007/05/16 03:32 AM
                                                              The fragility of your argumentRicardo B2007/05/16 06:41 AM
                                                                PSRicardo B2007/05/16 06:50 AM
                                                                The fragility of your argumentDean Kent2007/05/16 09:07 AM
                                                                  Modern web browsingS. Rao2007/05/16 09:16 AM
                                                                    Aha!Dean Kent2007/05/16 09:27 AM
                                                                      Aha!Dean Kent2007/05/16 09:32 AM
                                                                        Aha!S. Rao2007/05/16 10:34 AM
                                                                  The fragility of your argumentRicardo B2007/05/16 10:00 AM
                                                      The fragility of your argumentVincent Diepeveen2007/05/16 10:10 AM
                                                        The fragility of your argumentPaul2007/05/16 03:01 PM
                                                          The fragility of your argumentVincent Diepeveen2007/05/17 03:05 AM
                                                    The fragility of your argumentanon22007/05/15 01:35 AM
                                                    Splits vs page allocations?Matt Sayler2007/05/15 07:33 AM
                                          What's your point?Michael S2007/05/13 08:55 AM
                                            What's your point?anonymous2007/05/13 11:08 AM
                                              What's your point?Michael S2007/05/13 11:31 AM
                                      let's stay with x86-64 for now, pleaseJasonB2007/05/13 07:16 AM
                                x864 =) (NT)some12007/05/15 03:03 AM
    Rock/Tukwila rumorsIntelUser20002007/05/06 02:27 PM
  Rock/Tukwila rumorsm2007/05/13 08:05 AM
  Rock/Tukwila rumorsmas2007/05/15 09:40 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?