Windows manages memory differently

By: JasonB (no.delete@this.spam.com), May 14, 2007 12:50 am
Room: Moderated Discussions
Tzvetan Mikov (tzvetanmi@yahoo.com) on 5/13/07 wrote:
---------------------------
>Off the top of my head, conceptually, I can't really tell why Windows would have
>trouble with anything up to 2-3GB RAM.

Out of the box, the maximum virtual address space available to a process is 2GB. (The upper 2GB is reserved for the kernel.)

An application that is using much more than 1GB is going to start getting virtual address space fragmentation problems -- if you have a pointer that is pointing to an address at 1.5GB, you're currently using 1GB, and you try to allocate anything over 0.5GB, it will not be able to satisfy that request because the OS can't renumber your pointer. On a 2GB PC you might still have nearly 1GB of physical RAM free, and nearly 1GB of virtual address space free, but you can't allocate 0.5GB.

One of the key features of virtual memory is that you can remap physical addresses under the covers to get rid of memory fragmentation; what do you do when the virtual addresses are fragmented? You need virtual virtual addresses. :-)

A simple example where a nice, high-performance strategy comes unstuck when virtual addresses start to run out is the STL vector's growth behaviour. (It's a dynamically-extendable array, for those who don't know.) To satisfy the C++ Standard's requirement that appending to the end of the vector must be an amortised constant time operation, it grows by a constant percentage of its existing size each time it runs out of space. Under Visual C++ 6, developed at a time when virtual memory was plentiful, it would double each time; under VS2005 it grows by 50% each time. The latter runs out of memory less often at the expense of copying data more frequently. Doubling the size shouldn't actually cause physical memory to be wasted, because the pages will only be mapped when they are touched; but when virtual memory is scarce the append operation fails depressingly often, even when there is plenty of RAM available.

>So it is conceivable that the trouble you app is having with more than 1.5GB RAM
>under XP may not be so much a HIGHMEM issue, but a matter of XP not being optimized
>for such amount of RAM. The kernel is pretty old. So, it would be interesting to run the same test with 32-bit Vista.

It's not the kernel having the problem, it's the application not being able to use the full 2GB of physical (and virtual) address space that should be available to it, and one of the big causes of that is virtual address space fragmentation.

The point is that it starts to be an issue way before the theoretical 4GB limitation of 32 bit computers. Anything that attempts to use much more than half of a 2GB system's memory needs to take care to avoid having problems, and a memory footprint of "more than 1GB" is a lot less unusual for a modern desktop app than "4GB".
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Rock/Tukwila rumorsmas2007/05/05 11:59 AM
  Rock/Tukwila rumorsDavid Kanter2007/05/05 01:33 PM
    Rock/Tukwila rumorsDean Kent2007/05/05 02:35 PM
      K8 vs Win64 timelineanonymous2007/05/05 05:19 PM
        Yes, I misremembered...Dean Kent2007/05/05 09:03 PM
    RockDaniel Bizó2007/05/06 01:34 AM
      RockDean Kent2007/05/06 06:11 AM
    Rock/Tukwila rumorsJoe2007/05/06 10:24 AM
      Rock/Tukwila rumorsDean Kent2007/05/06 10:49 AM
      Rock/Tukwila rumorsLinus Torvalds2007/05/06 11:09 AM
      Rock/Tukwila rumorsanon2007/05/07 12:32 AM
        Rock/Tukwila rumorsRakesh Malik2007/05/07 08:36 AM
          Rock/Tukwila rumorsMichael S2007/05/07 09:06 AM
          Rock/Tukwila rumorsanon2007/05/07 08:48 PM
            Rock/Tukwila rumorsRakesh Malik2007/05/08 05:45 AM
              Rock/Tukwila rumorsanon2007/05/08 04:30 PM
                Wow. (nt)Brannon2007/05/08 05:16 PM
                Rock/Tukwila rumorsrwessel2007/05/08 08:48 PM
                  Rock/Tukwila rumorsJS2007/05/08 09:07 PM
                    Rock/Tukwila rumorsJS2007/05/09 05:44 AM
                Rock/Tukwila rumorsRakesh Malik2007/05/09 04:35 AM
                  Much ado about xMichael S2007/05/09 08:39 AM
                    Call it x86-64Linus Torvalds2007/05/09 09:27 AM
                      (i)AMD64Michael S2007/05/09 11:16 AM
                        (i)AMD64Linus Torvalds2007/05/09 11:29 AM
                          (i)AMD64Groo2007/05/09 03:45 PM
                          TIFNAAanonymous2007/05/09 04:49 PM
                            Inspired by FYR Macedonia? (NT)Michael S2007/05/09 10:21 PM
                              More likely...rwessel2007/05/09 11:39 PM
                            TIFNAAGabriele Svelto2007/05/09 10:57 PM
                          (i)AMD64James2007/05/10 01:27 AM
                        i86Dean Kent2007/05/09 11:30 AM
                        (i)AMD64Max2007/05/09 12:28 PM
                          wide86? long86?hobold2007/05/10 04:05 AM
                            x87 perhaps, it is one more. :) (NT)Groo2007/05/10 04:50 AM
                              x86+Dean Kent2007/05/10 07:44 AM
                                Does it really matter?Doug Siebert2007/05/10 08:10 AM
                                  let's stay with x86-64 for now, pleaseMarcin Niewiadomski2007/05/10 10:50 AM
                                    let's stay with x86-64 for now, pleaseDean Kent2007/05/11 05:11 AM
                                      let's stay with x86-64 for now, pleaserwessel2007/05/11 01:46 PM
                                        let's stay with x86-64 for now, pleaseDean Kent2007/05/11 05:03 PM
                                          let's stay with x86-64 for now, pleaseMichael S2007/05/12 09:49 AM
                                            let's stay with x86-64 for now, pleaseDean Kent2007/05/12 12:05 PM
                                              let's stay with x86-64 for now, pleaseMichael S2007/05/12 12:25 PM
                                                let's stay with x86-64 for now, pleaseDean Kent2007/05/12 02:39 PM
                                                  let's stay with x86-64 for now, pleaseJasonB2007/05/13 06:43 AM
                                                    client consolidationMichael S2007/05/13 07:37 AM
                                                  let's stay with x86-64 for now, pleaseTzvetan Mikov2007/05/13 02:44 PM
                                                let's stay with x86-64 for now, pleaserwessel2007/05/14 01:42 PM
                                      What's your point?Doug Siebert2007/05/11 01:56 PM
                                        What's your point?Linus Torvalds2007/05/11 03:15 PM
                                          What's your point?Doug Siebert2007/05/13 02:11 PM
                                            What's your point?Dean Kent2007/05/13 06:04 PM
                                              What's your point?JasonB2007/05/14 01:06 AM
                                                What's your point?Dean Kent2007/05/14 06:20 AM
                                                  What's your point?JasonB2007/05/14 03:35 PM
                                                  What's your point?JasonB2007/05/14 06:35 PM
                                                    What's your point?Dean Kent2007/05/14 07:12 PM
                                        What's your point?Dean Kent2007/05/11 05:06 PM
                                        What's your point?Stephen H2007/05/13 12:55 AM
                                          Why didn't MS take advantage of PAE?David W. Hess2007/05/13 07:37 AM
                                            PAE sucks (Why didn't MS take advantage of PAE?)Linus Torvalds2007/05/13 09:20 AM
                                              PAE sucks (Why didn't MS take advantage of PAE?)Dean Kent2007/05/13 09:49 AM
                                              PAE sucks (Why didn't MS take advantage of PAE?)David W. Hess2007/05/13 11:37 AM
                                              > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 12:44 PM
                                                > 1 GB RAM on a 32-bit systemS. Rao2007/05/13 02:00 PM
                                                  > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 04:32 PM
                                                    > 1 GB RAM on a 32-bit systemS. Rao2007/05/13 11:19 PM
                                                > 1 GB RAM on a 32-bit systemLinus Torvalds2007/05/13 02:46 PM
                                                  > 1 GB RAM on a 32-bit systemTzvetan Mikov2007/05/13 04:23 PM
                                                  > 1 GB RAM on a 32-bit systemJasonB2007/05/13 05:37 PM
                                                    Windows manages memory differentlyTzvetan Mikov2007/05/13 07:31 PM
                                                      Windows manages memory differentlyJasonB2007/05/14 12:50 AM
                                                        Windows manages memory differentlyTzvetan Mikov2007/05/14 07:56 AM
                                                          Windows manages memory differentlyrwessel2007/05/14 02:40 PM
                                                            Windows manages memory differentlyDavid W. Hess2007/05/14 03:07 PM
                                                              Windows manages memory differentlyrwessel2007/05/14 03:51 PM
                                                            Windows manages memory differentlyTzvetan Mikov2007/05/14 04:40 PM
                                                              Windows manages memory differentlyrwessel2007/05/14 05:09 PM
                                                      Windows manages memory differentlyHoward Chu2007/05/14 10:17 AM
                                                        Windows manages memory differentlyJukka Larja2007/05/14 10:30 AM
                                                        Windows manages memory differentlyTzvetan Mikov2007/05/14 12:54 PM
                                                          Windows manages memory differentlyHoward Chu2007/05/15 02:35 AM
                                                            Windows manages memory differentlyGroo2007/05/15 06:34 AM
                                                      Anyone know what OS X (10.4, Intel, desktop) does?Matt Sayler2007/05/15 05:23 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?Wes Felter2007/05/15 07:37 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?Anonymous2007/05/15 09:49 AM
                                                        Anyone know what OS X (10.4, Intel, desktop) does?anon22007/05/15 06:13 PM
                                              PAE sucks (Why didn't MS take advantage of PAE?)Paul2007/05/13 02:40 PM
                                                PAE sucks (Why didn't MS take advantage of PAE?)Peter Arremann2007/05/13 04:38 PM
                                                  PAE sucks (Why didn't MS take advantage of PAE?)Henrik S2007/05/14 02:31 AM
                                              The fragility of your argumentslacker2007/05/13 02:56 PM
                                                The fragility of your argumentnick2007/05/13 04:42 PM
                                                The fragility of your argumentHoward Chu2007/05/14 01:52 AM
                                                  The fragility of your argumentDean Kent2007/05/14 08:19 AM
                                                The fragility of your argumentanon22007/05/14 07:26 AM
                                                  The fragility of your argumentTzvetan Mikov2007/05/14 08:01 AM
                                                  The fragility of your argumentDean Kent2007/05/14 08:16 AM
                                                    The fragility of your argumentLinus Torvalds2007/05/14 10:57 AM
                                                      The fragility of your argumentJasonB2007/05/14 03:48 PM
                                                        The fragility of your argumentDean Kent2007/05/14 06:36 PM
                                                          The fragility of your argumentRicardo B2007/05/16 01:40 AM
                                                            The fragility of your argumentDean Kent2007/05/16 02:32 AM
                                                              The fragility of your argumentRicardo B2007/05/16 05:41 AM
                                                                PSRicardo B2007/05/16 05:50 AM
                                                                The fragility of your argumentDean Kent2007/05/16 08:07 AM
                                                                  Modern web browsingS. Rao2007/05/16 08:16 AM
                                                                    Aha!Dean Kent2007/05/16 08:27 AM
                                                                      Aha!Dean Kent2007/05/16 08:32 AM
                                                                        Aha!S. Rao2007/05/16 09:34 AM
                                                                  The fragility of your argumentRicardo B2007/05/16 09:00 AM
                                                      The fragility of your argumentVincent Diepeveen2007/05/16 09:10 AM
                                                        The fragility of your argumentPaul2007/05/16 02:01 PM
                                                          The fragility of your argumentVincent Diepeveen2007/05/17 02:05 AM
                                                    The fragility of your argumentanon22007/05/15 12:35 AM
                                                    Splits vs page allocations?Matt Sayler2007/05/15 06:33 AM
                                          What's your point?Michael S2007/05/13 07:55 AM
                                            What's your point?anonymous2007/05/13 10:08 AM
                                              What's your point?Michael S2007/05/13 10:31 AM
                                      let's stay with x86-64 for now, pleaseJasonB2007/05/13 06:16 AM
                                x864 =) (NT)some12007/05/15 02:03 AM
    Rock/Tukwila rumorsIntelUser20002007/05/06 01:27 PM
  Rock/Tukwila rumorsm2007/05/13 07:05 AM
  Rock/Tukwila rumorsmas2007/05/15 08:40 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?