By: anon2 (example.delete@this.example.com), May 15, 2007 12:35 am
Room: Moderated Discussions
Dean Kent (dkent@realworldtech.com) on 5/14/07 wrote:
---------------------------
>anon2 (example@example.net) on 5/14/07 wrote:
>---------------------------
>>No it is NOT. I have no idea where you got this impression. The requirements are (as Linus already explained):
>>
>>Virtual address space >= user address space + kernel address space
>>Kernel address space >= physical memory size + memory-mapped I/O size + kernel virtual mappings
>
>It seems odd to me that virtually *every* address must be mapped, whether actually used or not.
I don't understand what you are saying is odd here. The kernel uses all of physical memory, at all times. For cacheing if it can't find a better use. Doing less would be pointlessly throwing performance away.
There are kinds of cacheing where the kernel rarely or never needs to directly access the memory, which allows it to use memory not permanently mapped into the kernel's address space, but there's extra overhead if it does have to access it directly, and such memory can't be used for all purposes. This is Linux's HIGHMEM kludge.
>>Assuming you want to allow a user virtual address space
>>on the order of physical memory size, virtual memory size should be at least 2x physical memory.
>
>Even if *none* of your apps uses that much memory?
You need room in the total virtual address space for the kernel's needs (all of physical memory, if the kernel is doing its job right), plus the maximum potential user address space. Some processors have additional virtual address bits in the form of "address space IDs" which allow the 2^32 limit to be stretched, but the x86-32 whose PAE we are talking about does not.
Of each user address space, some fraction is mapped. Some fraction of the mapped space is backed by physical (resident) memory pages. Physical memory needs to be large enough to hold the latter. Virtual memory needs to be large enough to hold the former, which is always larger.
>Shouldn't an address space
>only be as large as it needs to be?
The basic size of an address space is fixed by the processor architecture. On x86-32, it is 2^32 bytes. (This may be further limited by the processor silicon, but no existing x86-32 imposes such a limit.)
Software may decide what fraction of that address space can be accessed, but its basic size is carved in silicon; it cannot be changed dynamically.
> Shouldn't virtual address pages be allocated only when actually needed?
They mostly are. One exception is the user/kernel split, which has to be in the same place in every process, so moving it dynamically is thoroughly impractical.
Instead, the virtual address space is statically divided into a "user region" which is different for every process and a "kernel region" which is shared.
>>The resident set size has NOTHING do do with it. The user (virtual) address space
>>contains all mapped addresses, both resident and non-resident. You need space for all of that.
>
>You need space for what you use, it seems to me. If you are allocating pages for
>unused memory, that seems to be a waste of effort to me.
I don't understand what "space" you are talking about. There are two kinds of space: virtual and physical. You need virtual address space for all virtual (potentially usable) addresses. You need physical memory for the subset of virtual addresses that you actually use.
When you memory-map a resource, you consume virtual address space equal to the size of that resource, but (modulo kernel bookkeeping) no physical memory. When you access that resource, you additionally need physical memory to hold the portion of the resource that you access.
---------------------------
>anon2 (example@example.net) on 5/14/07 wrote:
>---------------------------
>>No it is NOT. I have no idea where you got this impression. The requirements are (as Linus already explained):
>>
>>Virtual address space >= user address space + kernel address space
>>Kernel address space >= physical memory size + memory-mapped I/O size + kernel virtual mappings
>
>It seems odd to me that virtually *every* address must be mapped, whether actually used or not.
I don't understand what you are saying is odd here. The kernel uses all of physical memory, at all times. For cacheing if it can't find a better use. Doing less would be pointlessly throwing performance away.
There are kinds of cacheing where the kernel rarely or never needs to directly access the memory, which allows it to use memory not permanently mapped into the kernel's address space, but there's extra overhead if it does have to access it directly, and such memory can't be used for all purposes. This is Linux's HIGHMEM kludge.
>>Assuming you want to allow a user virtual address space
>>on the order of physical memory size, virtual memory size should be at least 2x physical memory.
>
>Even if *none* of your apps uses that much memory?
You need room in the total virtual address space for the kernel's needs (all of physical memory, if the kernel is doing its job right), plus the maximum potential user address space. Some processors have additional virtual address bits in the form of "address space IDs" which allow the 2^32 limit to be stretched, but the x86-32 whose PAE we are talking about does not.
Of each user address space, some fraction is mapped. Some fraction of the mapped space is backed by physical (resident) memory pages. Physical memory needs to be large enough to hold the latter. Virtual memory needs to be large enough to hold the former, which is always larger.
>Shouldn't an address space
>only be as large as it needs to be?
The basic size of an address space is fixed by the processor architecture. On x86-32, it is 2^32 bytes. (This may be further limited by the processor silicon, but no existing x86-32 imposes such a limit.)
Software may decide what fraction of that address space can be accessed, but its basic size is carved in silicon; it cannot be changed dynamically.
> Shouldn't virtual address pages be allocated only when actually needed?
They mostly are. One exception is the user/kernel split, which has to be in the same place in every process, so moving it dynamically is thoroughly impractical.
Instead, the virtual address space is statically divided into a "user region" which is different for every process and a "kernel region" which is shared.
>>The resident set size has NOTHING do do with it. The user (virtual) address space
>>contains all mapped addresses, both resident and non-resident. You need space for all of that.
>
>You need space for what you use, it seems to me. If you are allocating pages for
>unused memory, that seems to be a waste of effort to me.
I don't understand what "space" you are talking about. There are two kinds of space: virtual and physical. You need virtual address space for all virtual (potentially usable) addresses. You need physical memory for the subset of virtual addresses that you actually use.
When you memory-map a resource, you consume virtual address space equal to the size of that resource, but (modulo kernel bookkeeping) no physical memory. When you access that resource, you additionally need physical memory to hold the portion of the resource that you access.
Topic | Posted By | Date |
---|---|---|
Rock/Tukwila rumors | mas | 2007/05/05 11:59 AM |
Rock/Tukwila rumors | David Kanter | 2007/05/05 01:33 PM |
Rock/Tukwila rumors | Dean Kent | 2007/05/05 02:35 PM |
K8 vs Win64 timeline | anonymous | 2007/05/05 05:19 PM |
Yes, I misremembered... | Dean Kent | 2007/05/05 09:03 PM |
Rock | Daniel Bizó | 2007/05/06 01:34 AM |
Rock | Dean Kent | 2007/05/06 06:11 AM |
Rock/Tukwila rumors | Joe | 2007/05/06 10:24 AM |
Rock/Tukwila rumors | Dean Kent | 2007/05/06 10:49 AM |
Rock/Tukwila rumors | Linus Torvalds | 2007/05/06 11:09 AM |
Rock/Tukwila rumors | anon | 2007/05/07 12:32 AM |
Rock/Tukwila rumors | Rakesh Malik | 2007/05/07 08:36 AM |
Rock/Tukwila rumors | Michael S | 2007/05/07 09:06 AM |
Rock/Tukwila rumors | anon | 2007/05/07 08:48 PM |
Rock/Tukwila rumors | Rakesh Malik | 2007/05/08 05:45 AM |
Rock/Tukwila rumors | anon | 2007/05/08 04:30 PM |
Wow. (nt) | Brannon | 2007/05/08 05:16 PM |
Rock/Tukwila rumors | rwessel | 2007/05/08 08:48 PM |
Rock/Tukwila rumors | JS | 2007/05/08 09:07 PM |
Rock/Tukwila rumors | JS | 2007/05/09 05:44 AM |
Rock/Tukwila rumors | Rakesh Malik | 2007/05/09 04:35 AM |
Much ado about x | Michael S | 2007/05/09 08:39 AM |
Call it x86-64 | Linus Torvalds | 2007/05/09 09:27 AM |
(i)AMD64 | Michael S | 2007/05/09 11:16 AM |
(i)AMD64 | Linus Torvalds | 2007/05/09 11:29 AM |
(i)AMD64 | Groo | 2007/05/09 03:45 PM |
TIFNAA | anonymous | 2007/05/09 04:49 PM |
Inspired by FYR Macedonia? (NT) | Michael S | 2007/05/09 10:21 PM |
More likely... | rwessel | 2007/05/09 11:39 PM |
TIFNAA | Gabriele Svelto | 2007/05/09 10:57 PM |
(i)AMD64 | James | 2007/05/10 01:27 AM |
i86 | Dean Kent | 2007/05/09 11:30 AM |
(i)AMD64 | Max | 2007/05/09 12:28 PM |
wide86? long86? | hobold | 2007/05/10 04:05 AM |
x87 perhaps, it is one more. :) (NT) | Groo | 2007/05/10 04:50 AM |
x86+ | Dean Kent | 2007/05/10 07:44 AM |
Does it really matter? | Doug Siebert | 2007/05/10 08:10 AM |
let's stay with x86-64 for now, please | Marcin Niewiadomski | 2007/05/10 10:50 AM |
let's stay with x86-64 for now, please | Dean Kent | 2007/05/11 05:11 AM |
let's stay with x86-64 for now, please | rwessel | 2007/05/11 01:46 PM |
let's stay with x86-64 for now, please | Dean Kent | 2007/05/11 05:03 PM |
let's stay with x86-64 for now, please | Michael S | 2007/05/12 09:49 AM |
let's stay with x86-64 for now, please | Dean Kent | 2007/05/12 12:05 PM |
let's stay with x86-64 for now, please | Michael S | 2007/05/12 12:25 PM |
let's stay with x86-64 for now, please | Dean Kent | 2007/05/12 02:39 PM |
let's stay with x86-64 for now, please | JasonB | 2007/05/13 06:43 AM |
client consolidation | Michael S | 2007/05/13 07:37 AM |
let's stay with x86-64 for now, please | Tzvetan Mikov | 2007/05/13 02:44 PM |
let's stay with x86-64 for now, please | rwessel | 2007/05/14 01:42 PM |
What's your point? | Doug Siebert | 2007/05/11 01:56 PM |
What's your point? | Linus Torvalds | 2007/05/11 03:15 PM |
What's your point? | Doug Siebert | 2007/05/13 02:11 PM |
What's your point? | Dean Kent | 2007/05/13 06:04 PM |
What's your point? | JasonB | 2007/05/14 01:06 AM |
What's your point? | Dean Kent | 2007/05/14 06:20 AM |
What's your point? | JasonB | 2007/05/14 03:35 PM |
What's your point? | JasonB | 2007/05/14 06:35 PM |
What's your point? | Dean Kent | 2007/05/14 07:12 PM |
What's your point? | Dean Kent | 2007/05/11 05:06 PM |
What's your point? | Stephen H | 2007/05/13 12:55 AM |
Why didn't MS take advantage of PAE? | David W. Hess | 2007/05/13 07:37 AM |
PAE sucks (Why didn't MS take advantage of PAE?) | Linus Torvalds | 2007/05/13 09:20 AM |
PAE sucks (Why didn't MS take advantage of PAE?) | Dean Kent | 2007/05/13 09:49 AM |
PAE sucks (Why didn't MS take advantage of PAE?) | David W. Hess | 2007/05/13 11:37 AM |
> 1 GB RAM on a 32-bit system | Tzvetan Mikov | 2007/05/13 12:44 PM |
> 1 GB RAM on a 32-bit system | S. Rao | 2007/05/13 02:00 PM |
> 1 GB RAM on a 32-bit system | Tzvetan Mikov | 2007/05/13 04:32 PM |
> 1 GB RAM on a 32-bit system | S. Rao | 2007/05/13 11:19 PM |
> 1 GB RAM on a 32-bit system | Linus Torvalds | 2007/05/13 02:46 PM |
> 1 GB RAM on a 32-bit system | Tzvetan Mikov | 2007/05/13 04:23 PM |
> 1 GB RAM on a 32-bit system | JasonB | 2007/05/13 05:37 PM |
Windows manages memory differently | Tzvetan Mikov | 2007/05/13 07:31 PM |
Windows manages memory differently | JasonB | 2007/05/14 12:50 AM |
Windows manages memory differently | Tzvetan Mikov | 2007/05/14 07:56 AM |
Windows manages memory differently | rwessel | 2007/05/14 02:40 PM |
Windows manages memory differently | David W. Hess | 2007/05/14 03:07 PM |
Windows manages memory differently | rwessel | 2007/05/14 03:51 PM |
Windows manages memory differently | Tzvetan Mikov | 2007/05/14 04:40 PM |
Windows manages memory differently | rwessel | 2007/05/14 05:09 PM |
Windows manages memory differently | Howard Chu | 2007/05/14 10:17 AM |
Windows manages memory differently | Jukka Larja | 2007/05/14 10:30 AM |
Windows manages memory differently | Tzvetan Mikov | 2007/05/14 12:54 PM |
Windows manages memory differently | Howard Chu | 2007/05/15 02:35 AM |
Windows manages memory differently | Groo | 2007/05/15 06:34 AM |
Anyone know what OS X (10.4, Intel, desktop) does? | Matt Sayler | 2007/05/15 05:23 AM |
Anyone know what OS X (10.4, Intel, desktop) does? | Wes Felter | 2007/05/15 07:37 AM |
Anyone know what OS X (10.4, Intel, desktop) does? | Anonymous | 2007/05/15 09:49 AM |
Anyone know what OS X (10.4, Intel, desktop) does? | anon2 | 2007/05/15 06:13 PM |
PAE sucks (Why didn't MS take advantage of PAE?) | Paul | 2007/05/13 02:40 PM |
PAE sucks (Why didn't MS take advantage of PAE?) | Peter Arremann | 2007/05/13 04:38 PM |
PAE sucks (Why didn't MS take advantage of PAE?) | Henrik S | 2007/05/14 02:31 AM |
The fragility of your argument | slacker | 2007/05/13 02:56 PM |
The fragility of your argument | nick | 2007/05/13 04:42 PM |
The fragility of your argument | Howard Chu | 2007/05/14 01:52 AM |
The fragility of your argument | Dean Kent | 2007/05/14 08:19 AM |
The fragility of your argument | anon2 | 2007/05/14 07:26 AM |
The fragility of your argument | Tzvetan Mikov | 2007/05/14 08:01 AM |
The fragility of your argument | Dean Kent | 2007/05/14 08:16 AM |
The fragility of your argument | Linus Torvalds | 2007/05/14 10:57 AM |
The fragility of your argument | JasonB | 2007/05/14 03:48 PM |
The fragility of your argument | Dean Kent | 2007/05/14 06:36 PM |
The fragility of your argument | Ricardo B | 2007/05/16 01:40 AM |
The fragility of your argument | Dean Kent | 2007/05/16 02:32 AM |
The fragility of your argument | Ricardo B | 2007/05/16 05:41 AM |
PS | Ricardo B | 2007/05/16 05:50 AM |
The fragility of your argument | Dean Kent | 2007/05/16 08:07 AM |
Modern web browsing | S. Rao | 2007/05/16 08:16 AM |
Aha! | Dean Kent | 2007/05/16 08:27 AM |
Aha! | Dean Kent | 2007/05/16 08:32 AM |
Aha! | S. Rao | 2007/05/16 09:34 AM |
The fragility of your argument | Ricardo B | 2007/05/16 09:00 AM |
The fragility of your argument | Vincent Diepeveen | 2007/05/16 09:10 AM |
The fragility of your argument | Paul | 2007/05/16 02:01 PM |
The fragility of your argument | Vincent Diepeveen | 2007/05/17 02:05 AM |
The fragility of your argument | anon2 | 2007/05/15 12:35 AM |
Splits vs page allocations? | Matt Sayler | 2007/05/15 06:33 AM |
What's your point? | Michael S | 2007/05/13 07:55 AM |
What's your point? | anonymous | 2007/05/13 10:08 AM |
What's your point? | Michael S | 2007/05/13 10:31 AM |
let's stay with x86-64 for now, please | JasonB | 2007/05/13 06:16 AM |
x864 =) (NT) | some1 | 2007/05/15 02:03 AM |
Rock/Tukwila rumors | IntelUser2000 | 2007/05/06 01:27 PM |
Rock/Tukwila rumors | m | 2007/05/13 07:05 AM |
Rock/Tukwila rumors | mas | 2007/05/15 08:40 AM |