> 1 GB RAM on a 32-bit system

By: S. Rao (sonny.delete@this.burdell.org), May 14, 2007 12:19 am
Room: Moderated Discussions
Tzvetan Mikov (tzvetanmi@yahoo.com) on 5/13/07 wrote:
-
>Yes. I didn't realize this was the case for 1 GB. All popular press gives the impression
>that you only need 64bit after 4G, so I guess I just accepted it without much thinking.
>Even though I should know better. In my previous job I had hell of a time having
>to fit lots (hundreds of megabytes) of memory-mapped hardware into Windows kernel
>space. IIRC, W2K had very limited number of system PTEs reserved for that.
>
>Are you aware of any benchmarks that show the difference in performance between
>32-bit and 64-bit with < 4GB, that can be attributed to the overhead of the kernel switching HIGHMEM pages ?

Yep there are tons of them... and there are more
problems besides the kernel having to map >4gb pages.

You may or may not know this, but some background info
first for anyone else who might not:

In the 32bit x86 world using flat addressing, the address
space is typically partitioned into a kernel section and
a userspace section. This is done for performance reasons,
so the 32bit kernel can just directly address userspace
memory without having to do any extra mapping/unmapping
for (most) every single system call which does add
considerable overhead -- more on that later.

The typical split would be 1GB for kernel and 3GB for user
space (under Linux it was fairly easy to move that
around) I believe Windows used 2gb/2gb on most versions
except for Advanced Server which did the 3gb/1gb split
like Linux. So a single process is effectively limited
to what the OS lets them map either 2gb or 3gb in most
cases.. so that's the first problem.

So in the case of Windows, if you wanted
more than 2gb of flat address space, you had to run
Advanced Server, and I understand that some workstation
users had to do this on things like CAD apps.

The converse issue also exists for Linux, the kernel
is unable to map more than 1gb *total* for everything
it needs, and like user-space processes this space has
to be further sub-divided into bounded areas to have
a sane virtual memory management scheme. In Linux, there
are sections of address space for stack, heap, mmaps, etc.
I'm not certain if Windows is quite as strict, but I
wouldn't be surprised if it had similar restrictions.

So, for a given application that uses most of its memory
the same way, what started out as 4gb of address space is
usually cut down to 1gb or less because of the address
space partitioning. Yes, you could argue that applications
can work around these issues by doing their own memory
management... but this is just not practical in the general
case and it will involve a trade-off in either speed or
complexity to do this.

IMO, this is basic reasoning behind his argument that you
need several times the amount of address space as physical
memory. You can't have a sane VMM without it.

One example of the trade-off that had to be made was
in the case of the RedHat 32bit kernel which had support
for PAE. In order to sanely support the full 64GB of RAM
the kernel needed much more than 1GB of memory simply
because the amount of bookkeeping needed for 64GB of
physical memory was nearly 1GB by itself!

So what they did was have a kernel that didn't share any
(signification portion anyway) address space with userspace.
So kernel had 4gb, and userspace processes had 4gb, but
practically every system call system call required memory
maps to be made, which was very roughly said to be a 10-30%
hit on typical loads (jbb?).

I also ran into the situation where I had a few machines
with 8gb or 16gb of memory and I was doing filesystem work,
but it was largely unusable because the cost of
mapping/unmapping it for use as a general file cache was
so high that it just killed performance in the general
case -- so I ended up having to boot with < 4gb most of the
time to be able to isolate performance issues. That was
really irritating at the time, and I wasn't even the one
who paid for all that memory!


< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Rock/Tukwila rumorsmas2007/05/05 12:59 PM
  Rock/Tukwila rumorsDavid Kanter2007/05/05 02:33 PM
    Rock/Tukwila rumorsDean Kent2007/05/05 03:35 PM
      K8 vs Win64 timelineanonymous2007/05/05 06:19 PM
        Yes, I misremembered...Dean Kent2007/05/05 10:03 PM
    RockDaniel Bizó2007/05/06 02:34 AM
      RockDean Kent2007/05/06 07:11 AM
    Rock/Tukwila rumorsJoe2007/05/06 11:24 AM
      Rock/Tukwila rumorsDean Kent2007/05/06 11:49 AM
      Rock/Tukwila rumorsLinus Torvalds2007/05/06 12:09 PM
      Rock/Tukwila rumorsanon2007/05/07 01:32 AM
        Rock/Tukwila rumorsRakesh Malik2007/05/07 09:36 AM
          Rock/Tukwila rumorsMichael S2007/05/07 10:06 AM
          Rock/Tukwila rumorsanon2007/05/07 09:48 PM
            Rock/Tukwila rumorsRakesh Malik2007/05/08 06:45 AM
              Rock/Tukwila rumorsanon2007/05/08 05:30 PM
                Wow. (nt)Brannon2007/05/08 06:16 PM
                Rock/Tukwila rumorsrwessel2007/05/08 09:48 PM
                  Rock/Tukwila rumorsJS2007/05/08 10:07 PM
                    Rock/Tukwila rumorsJS2007/05/09 06:44 AM
                Rock/Tukwila rumorsRakesh Malik2007/05/09 05:35 AM
                  Much ado about xMichael S2007/05/09 09:39 AM
                    Call it x86-64Linus Torvalds2007/05/09 10:27 AM
                      (i)AMD64Michael S2007/05/09 12:16 PM
                        (i)AMD64Linus Torvalds2007/05/09 12:29 PM
                          (i)AMD64Groo2007/05/09 04:45 PM
                          TIFNAAanonymous2007/05/09 05:49 PM
                            Inspired by FYR Macedonia? (NT)Michael S2007/05/09 11:21 PM
                              More likely...rwessel2007/05/10 12:39 AM
                            TIFNAAGabriele Svelto2007/05/09 11:57 PM
                          (i)AMD64James2007/05/10 02:27 AM
                        i86Dean Kent2007/05/09 12:30 PM
                        (i)AMD64Max2007/05/09 01:28 PM
                          wide86? long86?hobold2007/05/10 05:05 AM
                            x87 perhaps, it is one more. :) (NT)Groo2007/05/10 05:50 AM
                              x86+Dean Kent2007/05/10 08:44 AM
                                Does it really matter?Doug Siebert2007/05/10 09:10 AM
                                  let's stay with x86-64 for now, pleaseMarcin Niewiadomski2007/05/10 11:50 AM
                                    let's stay with x86-64 for now, pleaseDean Kent2007/05/11 06:11 AM
                                      let's stay with x86-64 for now, pleaserwessel2007/05/11 02:46 PM
                                        let's stay with x86-64 for now, pleaseDean Kent2007/05/11 06:03 PM
                                          let's stay with x86-64 for now, pleaseMichael S2007/05/12 10:49 AM
                                            let's stay with x86-64 for now, pleaseDean Kent2007/05/12 01:05 PM
                                              let's stay with x86-64 for now, pleaseMichael S2007/05/12 01:25 PM
                                                let's stay with x86-64 for now, pleaseDean Kent2007/05/12 03:39 PM
                                                  let's stay with x86-64 for now, pleaseJasonB2007/05/13 07:43 AM
                                                    client consolidationMichael S2007/05/13 08:37 AM
                                                  let's stay with x86-64 for now, pleaseTzvetan Mikov2007/05/13 03:44 PM
                                                let's stay with x86-64 for now, pleaserwessel2007/05/14 02:42 PM
                                      What's your point?Doug Siebert2007/05/11 02:56 PM
                                        What's your point?Linus Torvalds2007/05/11 04:15 PM
                                          What's your point?Doug Siebert2007/05/13 03:11 PM
                                            What's your point?Dean Kent2007/05/13 07:04 PM
                                              What's your point?JasonB2007/05/14 02:06 AM
                                                What's your point?Dean Kent2007/05/14 07:20 AM
                                                  What's your point?JasonB2007/05/14 04:35 PM
                                                  What's your point?JasonB2007/05/14 07:35 PM
                                                    What's your point?Dean Kent2007/05/14 08:12 PM
                                        What's your point?Dean Kent2007/05/11 06:06 PM
                                        What's your point?Stephen H2007/05/13 01:55 AM
                                          Why didn't MS take advantage of PAE?David W. Hess2007/05/13 08:37 AM
                                            PAE sucks (Why didn't MS take advantage of PAE?)Linus Torvalds2007/05/13 10:20 AM
                                              PAE sucks (Why didn't MS take advantage of PAE?)Dean Kent2007/05/13 10:49 AM
                                              PAE sucks (Why didn't MS take advantage of PAE?)David W. Hess2007/05/13 12:37 PM
                                              > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 01:44 PM
                                                > 1 GB RAM on a 32-bit systemS. Rao2007/05/13 03:00 PM
                                                  > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 05:32 PM
                                                    > 1 GB RAM on a 32-bit systemS. Rao2007/05/14 12:19 AM
                                                > 1 GB RAM on a 32-bit systemLinus Torvalds2007/05/13 03:46 PM
                                                  > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 05:23 PM
                                                  > 1 GB RAM on a 32-bit systemJasonB2007/05/13 06:37 PM
                                                    Windows manages memory differentlyTzvetan Mikov2007/05/13 08:31 PM
                                                      Windows manages memory differentlyJasonB2007/05/14 01:50 AM
                                                        Windows manages memory differentlyTzvetan Mikov2007/05/14 08:56 AM
                                                          Windows manages memory differentlyrwessel2007/05/14 03:40 PM
                                                            Windows manages memory differentlyDavid W. Hess2007/05/14 04:07 PM
                                                              Windows manages memory differentlyrwessel2007/05/14 04:51 PM
                                                            Windows manages memory differentlyTzvetan Mikov2007/05/14 05:40 PM
                                                              Windows manages memory differentlyrwessel2007/05/14 06:09 PM
                                                      Windows manages memory differentlyHoward Chu2007/05/14 11:17 AM
                                                        Windows manages memory differentlyJukka Larja2007/05/14 11:30 AM
                                                        Windows manages memory differentlyTzvetan Mikov2007/05/14 01:54 PM
                                                          Windows manages memory differentlyHoward Chu2007/05/15 03:35 AM
                                                            Windows manages memory differentlyGroo2007/05/15 07:34 AM
                                                      Anyone know what OS X (10.4, Intel, desktop) does?Matt Sayler2007/05/15 06:23 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?Wes Felter2007/05/15 08:37 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?Anonymous2007/05/15 10:49 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?anon22007/05/15 07:13 PM
                                              PAE sucks (Why didn't MS take advantage of PAE?)Paul2007/05/13 03:40 PM
                                                PAE sucks (Why didn't MS take advantage of PAE?)Peter Arremann2007/05/13 05:38 PM
                                                  PAE sucks (Why didn't MS take advantage of PAE?)Henrik S2007/05/14 03:31 AM
                                              The fragility of your argumentslacker2007/05/13 03:56 PM
                                                The fragility of your argumentnick2007/05/13 05:42 PM
                                                The fragility of your argumentHoward Chu2007/05/14 02:52 AM
                                                  The fragility of your argumentDean Kent2007/05/14 09:19 AM
                                                The fragility of your argumentanon22007/05/14 08:26 AM
                                                  The fragility of your argumentTzvetan Mikov2007/05/14 09:01 AM
                                                  The fragility of your argumentDean Kent2007/05/14 09:16 AM
                                                    The fragility of your argumentLinus Torvalds2007/05/14 11:57 AM
                                                      The fragility of your argumentJasonB2007/05/14 04:48 PM
                                                        The fragility of your argumentDean Kent2007/05/14 07:36 PM
                                                          The fragility of your argumentRicardo B2007/05/16 02:40 AM
                                                            The fragility of your argumentDean Kent2007/05/16 03:32 AM
                                                              The fragility of your argumentRicardo B2007/05/16 06:41 AM
                                                                PSRicardo B2007/05/16 06:50 AM
                                                                The fragility of your argumentDean Kent2007/05/16 09:07 AM
                                                                  Modern web browsingS. Rao2007/05/16 09:16 AM
                                                                    Aha!Dean Kent2007/05/16 09:27 AM
                                                                      Aha!Dean Kent2007/05/16 09:32 AM
                                                                        Aha!S. Rao2007/05/16 10:34 AM
                                                                  The fragility of your argumentRicardo B2007/05/16 10:00 AM
                                                      The fragility of your argumentVincent Diepeveen2007/05/16 10:10 AM
                                                        The fragility of your argumentPaul2007/05/16 03:01 PM
                                                          The fragility of your argumentVincent Diepeveen2007/05/17 03:05 AM
                                                    The fragility of your argumentanon22007/05/15 01:35 AM
                                                    Splits vs page allocations?Matt Sayler2007/05/15 07:33 AM
                                          What's your point?Michael S2007/05/13 08:55 AM
                                            What's your point?anonymous2007/05/13 11:08 AM
                                              What's your point?Michael S2007/05/13 11:31 AM
                                      let's stay with x86-64 for now, pleaseJasonB2007/05/13 07:16 AM
                                x864 =) (NT)some12007/05/15 03:03 AM
    Rock/Tukwila rumorsIntelUser20002007/05/06 02:27 PM
  Rock/Tukwila rumorsm2007/05/13 08:05 AM
  Rock/Tukwila rumorsmas2007/05/15 09:40 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?