By: Ungo (a.delete@this.b.c.d.e), May 16, 2013 3:54 pm
Room: Moderated Discussions
Maynard Handley (name99.delete@this.name99.org) on May 15, 2013 3:33 pm wrote:
> I don't know my way around the Darwin source tree, but both a manual exploration and a Google
> site specific search did not reveal anything helpful. There IS a malloc.c (not in the xnu tree,
> but at http://www.opensource.apple.com/source/Libc/Libc-498/gen/malloc.c), however it is basically
> a thin adaptor layer between the malloc API and an internal zone-based API. The zone calls live
> in objc/zone.c, and a google search did not turn up a public version of that file.
They're now up to Libc-825.26; 498 is rather old.
Also you're looking in the wrong place for the zone implementation. Maybe you were keying off the inclusion of objc/zone.h in Libc-498/gen/malloc.c? They've removed that include some time between then and now. I suspect it was never a zone implementation, it seems likely to be the Obj-C interface to malloc_zone_register() etc.
What we're really looking for is the default zone implementation, the one you get from userspace when calling plain malloc() without registering your own zone. _malloc_initialize() in malloc.c creates the default zone with this call:
zone = create_scalable_zone(0, malloc_debug_flags);
Libc-825.26 has two files which implement the "scalable_zone" allocator: scalable_malloc.c and magazine_malloc.c. They appear to be different versions of the same code, and magazine_malloc.c appears to be the current one. After a brief scan through the code, all paths appear to allocate pages with mmap().
Which is why I suggested looking in XNU. Libc malloc() code (or application-supplied code which uses the zone plugin hooks to add one or more malloc() implementations of its own) ultimately can't do anything but ask the kernel for blocks of virtual addresses to subdivide. So the information you need is whether XNU itself supports large pages, and if so, when it chooses to use them.
Some digging suggests that it does, but it's limited (e.g. it looks like large pages are always wired). Download the full 10.8.3 xnu tarball and search it for "superpage" (case insensitive).
> I don't know my way around the Darwin source tree, but both a manual exploration and a Google
> site specific search did not reveal anything helpful. There IS a malloc.c (not in the xnu tree,
> but at http://www.opensource.apple.com/source/Libc/Libc-498/gen/malloc.c), however it is basically
> a thin adaptor layer between the malloc API and an internal zone-based API. The zone calls live
> in objc/zone.c, and a google search did not turn up a public version of that file.
They're now up to Libc-825.26; 498 is rather old.
Also you're looking in the wrong place for the zone implementation. Maybe you were keying off the inclusion of objc/zone.h in Libc-498/gen/malloc.c? They've removed that include some time between then and now. I suspect it was never a zone implementation, it seems likely to be the Obj-C interface to malloc_zone_register() etc.
What we're really looking for is the default zone implementation, the one you get from userspace when calling plain malloc() without registering your own zone. _malloc_initialize() in malloc.c creates the default zone with this call:
zone = create_scalable_zone(0, malloc_debug_flags);
Libc-825.26 has two files which implement the "scalable_zone" allocator: scalable_malloc.c and magazine_malloc.c. They appear to be different versions of the same code, and magazine_malloc.c appears to be the current one. After a brief scan through the code, all paths appear to allocate pages with mmap().
Which is why I suggested looking in XNU. Libc malloc() code (or application-supplied code which uses the zone plugin hooks to add one or more malloc() implementations of its own) ultimately can't do anything but ask the kernel for blocks of virtual addresses to subdivide. So the information you need is whether XNU itself supports large pages, and if so, when it chooses to use them.
Some digging suggests that it does, but it's limited (e.g. it looks like large pages are always wired). Download the full 10.8.3 xnu tarball and search it for "superpage" (case insensitive).