By: Travis Downs (travis.downs.delete@this.gmail.com), June 5, 2019 4:48 pm
Room: Moderated Discussions
Maybe it's a bit OT for this forum, but hey I figure it can't be worse than Ireland spam, right?
On Linux, given a mapped-but-probably-not-populated anonymous region of memory, is there any way to ask the kernel to populate it?
The usual approach is just to write to it, but on x86 you take a fault for every 4k page - and thanks to our friends Spectre and Meltdown this hurts a lot more than it used to. I know I'm about to write to all this memory, so I'd like to just populate the entire region right now.
If I were allocating the memory myself with mmap, the solution is just to pass MAP_POPULATE to mmap. However, there are scenarios were I'm not allocating the memory, can I can't control the caller, but I'd like to populate all the pages after the initial allocation.
One dirty hack is back-to-back mlock()/munlock() on the region, but this does way more work than is necessary (all the munlock() is wasted, and anything mlock() does other than filling the PTEs is wasted as well). Moreover, it isn't work for non-privledged processes unless you restrict the granularity to RLIMIT_MEMLOCK, and on memory constrained systems you might blow yourself up.
One interesting observation is that in this respect anonymous pages are slower than file-backed pages: file-backed pages on any kernel in the last 5-10 years use "fault-around" which reads in additional nearby pages (16 by default on x86) when a page faults. This behavior doesn't apply to anonymous pages. So in some cases it's faster just to map a bunch of existing files on tmpfs than it is to allocate anonymous regions because (a) they benefit from faultaround and (b) they won't be zeroed. The (a) part is mostly BS because it implies you control the allocation call, in which case you could just use MAP_POPULATE.
On Linux, given a mapped-but-probably-not-populated anonymous region of memory, is there any way to ask the kernel to populate it?
The usual approach is just to write to it, but on x86 you take a fault for every 4k page - and thanks to our friends Spectre and Meltdown this hurts a lot more than it used to. I know I'm about to write to all this memory, so I'd like to just populate the entire region right now.
If I were allocating the memory myself with mmap, the solution is just to pass MAP_POPULATE to mmap. However, there are scenarios were I'm not allocating the memory, can I can't control the caller, but I'd like to populate all the pages after the initial allocation.
One dirty hack is back-to-back mlock()/munlock() on the region, but this does way more work than is necessary (all the munlock() is wasted, and anything mlock() does other than filling the PTEs is wasted as well). Moreover, it isn't work for non-privledged processes unless you restrict the granularity to RLIMIT_MEMLOCK, and on memory constrained systems you might blow yourself up.
One interesting observation is that in this respect anonymous pages are slower than file-backed pages: file-backed pages on any kernel in the last 5-10 years use "fault-around" which reads in additional nearby pages (16 by default on x86) when a page faults. This behavior doesn't apply to anonymous pages. So in some cases it's faster just to map a bunch of existing files on tmpfs than it is to allocate anonymous regions because (a) they benefit from faultaround and (b) they won't be zeroed. The (a) part is mostly BS because it implies you control the allocation call, in which case you could just use MAP_POPULATE.
Topic | Posted By | Date |
---|---|---|
Pre-populating anonymous pages | Travis Downs | 2019/06/05 04:48 PM |
Pre-populating anonymous pages | Jeff S. | 2019/06/05 08:03 PM |
Pre-populating anonymous pages | Travis Downs | 2019/06/06 07:11 AM |
Pre-populating anonymous pages | Jeff S. | 2019/06/06 08:40 AM |
Pre-populating anonymous pages | Travis Downs | 2019/06/06 08:59 AM |
Pre-populating anonymous pages | Jeff S. | 2019/06/06 09:19 AM |
Pre-populating anonymous pages | Foo_ | 2019/06/06 12:30 AM |
Pre-populating anonymous pages | Travis Downs | 2019/06/06 06:59 AM |
Pre-populating anonymous pages | Foo_ | 2019/06/06 07:56 AM |
Pre-populating anonymous pages | Travis Downs | 2019/06/06 09:02 AM |
Pre-populating anonymous pages | Linus Torvalds | 2019/06/06 11:01 AM |
Pre-populating anonymous pages | Travis Downs | 2019/06/07 02:16 PM |
Pre-populating anonymous pages | Brendan | 2019/06/08 02:55 AM |
Pre-populating anonymous pages | Travis Downs | 2019/06/08 08:18 AM |
Pre-populating anonymous pages | Linus Torvalds | 2019/06/08 11:43 AM |
Pre-populating anonymous pages | Brendan | 2019/06/09 03:29 AM |
Pre-populating anonymous pages | Linus Torvalds | 2019/06/10 11:20 AM |
Pre-populating anonymous pages | Travis Downs | 2019/06/17 09:18 AM |
Pre-populating anonymous pages | Linus Torvalds | 2019/06/18 04:28 PM |