Pre-populating anonymous pages

By: Travis Downs (travis.downs.delete@this.gmail.com), June 6, 2019 7:59 am
Room: Moderated Discussions
Jeff S. (fake.delete@this.fake.com) on June 6, 2019 8:40 am wrote:
> Travis Downs (travis.downs.delete@this.gmail.com) on June 6, 2019 7:11 am wrote:
> > Jeff S. (fakity.delete@this.fake.com) on June 5, 2019 8:03 pm wrote:
>
> > > Just MAP_FIXED|MAP_POPULATE over page rages not mincore()? You're at least safe from mlock() ulimits then.
> > Yeah, this crossed my mind, but it seems a bit risky?
>
> In mm/mmap.c, do_brk_flags() and its various wrappers don't doing anything special with the
> VMA that makes it special from any other; all the extra state about the program break really
> is just held in a handful of mm_struct members only examined in brk-related functions.
>
> That said, relying on undocumented behavior is usually completely nuts, so if I really needed to go this far
> out of my way to avoid normal prefaults, I would probably double-check the page map for weird flags/inodes/devs
> as well as not doing remaps under the page break on systems I didn't completely control and understand.
>
> > This will replace the existing mapping. If the new mmap has the same characteristics as the
> > old one, I guess that is harmless, but I can't really be sure of how the pages were mapped
> > in the first place and I feel like it could break when the allocator changes. Maybe the pages
> > weren't even allocated with mmap in the first place, but rather [s]brk(2) or something like
> > that (IIRC most allocators still try to use brk for at least some allocations).
>
> I believe the remap should be brk()-safe for now, but a future change in the kernel's
> special handling/accounting for brk() is a more than plausible fear.

Thanks for looking into brk(), but not even my primary concern. My primary concern is that the original allocator mapped the area with some flags (just check out the number of MAP_ flags to mmap) that I am not aware of, and mmam(MAP_FIXED) will replace those flags and I don't know what the impact of that is. Now going through the flags, it doesn't seem like there are too many that are likely to be set and cause a problem if overwritten, but...


> The uselessness of madvise(..., MADV_WILLNEED) for anonymous pages struct me more as an
> oddity than even a real annoyance, but I'd never worried about manual prefaulting sidechannel
> issues either. Maybe a proposal to amend or augment the interface is realistic now?

Yeah maybe. The existing behavior doesn't strike me as that odd though - it seems like all that stuff was totally oriented towards file-backed pages. I don't think it does a synchronous read either, it just kicks off the async readahead or something?
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Pre-populating anonymous pagesTravis Downs2019/06/05 03:48 PM
  Pre-populating anonymous pagesJeff S.2019/06/05 07:03 PM
    Pre-populating anonymous pagesTravis Downs2019/06/06 06:11 AM
      Pre-populating anonymous pagesJeff S.2019/06/06 07:40 AM
        Pre-populating anonymous pagesTravis Downs2019/06/06 07:59 AM
          Pre-populating anonymous pagesJeff S.2019/06/06 08:19 AM
  Pre-populating anonymous pagesFoo_2019/06/05 11:30 PM
    Pre-populating anonymous pagesTravis Downs2019/06/06 05:59 AM
      Pre-populating anonymous pagesFoo_2019/06/06 06:56 AM
        Pre-populating anonymous pagesTravis Downs2019/06/06 08:02 AM
  Pre-populating anonymous pagesLinus Torvalds2019/06/06 10:01 AM
    Pre-populating anonymous pagesTravis Downs2019/06/07 01:16 PM
      Pre-populating anonymous pagesBrendan2019/06/08 01:55 AM
        Pre-populating anonymous pagesTravis Downs2019/06/08 07:18 AM
        Pre-populating anonymous pagesLinus Torvalds2019/06/08 10:43 AM
          Pre-populating anonymous pagesBrendan2019/06/09 02:29 AM
            Pre-populating anonymous pagesLinus Torvalds2019/06/10 10:20 AM
          Pre-populating anonymous pagesTravis Downs2019/06/17 08:18 AM
            Pre-populating anonymous pagesLinus Torvalds2019/06/18 03:28 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell tangerine? ūüćä