Hi,

Gabriele Svelto (gabriele.svelto.delete@this.gmail.com) on January 22, 2020 12:53 am wrote:
> Brendan (btrotter.delete@this.gmail.com) on January 21, 2020 5:07 pm wrote:
> > Unfortunately, no, that's not obvious if I think about it. E.g. for a simple "mark and sweep", I can't
> > see why you can't have a "backlink" (plus "mark flag") built
> > into each allocated object's header ("pre-allocated"
> > as part of allocating memory for the object), so that you can traverse in one direction by searching the
> > object for references to other objects (that you aren't
> > already marking because they don't have their backlink
> > field set already) and can traverse in the opposite direction by following the current object's backlink;
> > without recursion, and without any allocations made during garbage collection.
> >
> > Further, (assuming the ability to use some kind of "atomic
> > compare and swap") I can't see why it would be hard
> > (using the same "pre-allocated backlink" approach) to have multiple threads/CPUs doing GC at the same time.
>
> That's not "pre-allocation", that's dynamically bloating all your objects severely increasing overall memory
> consumption. Also modern GCs are not only parallel but also concurrent and let allocations happen while
> they're running (at least to a certain extent). Threads are already manipulating the object header atomically
> so I don't think it's possible to manipulate another field without actual synchronization.

The extra ~8 byte field (per object) was probably already wasted (padding to achieve 16-byte alignment) and will probably save you from consuming more in stack space for recursion.

My previous comment (the part you deleted) covered concurrency.

Separate fields are separate (it's not a correctness issue); but yes, if fields share a cache line there may be some performance loss caused by the cache line bouncing between CPUs wanting exclusive access.

> > > 5 processes and a total of 580 MiB, for one open tab? In that case, depending on what the page was
> > > (not forgetting that some pages are like this: https://bellard.org/jslinux/vm.html?url=buildroot-x86.cfg
> > > ), I would say it's "potentially excessive". However; I still remember (e.g.) running Netscape on a
> > > machine with 64 MiB of RAM (back when 500 MiB of disk space was rare) without a single problem, so
> > > maybe my idea of "acceptable" is lagging too far behind what other people consider "normal".
>
> That's 60MiB per process including space committed by the graphics drivers plus 380MiB of file/executable
> mappings. The double-buffered surface for a 1920x1080 window will take 16MiB on its own. We do better than
> that on Linux but it's quite hard to implement a standard-compliant browser using less than that.
>
> > Of course I hope we all agree that "no swap at all" (for any machine that doesn't overcommit) is silly.
>
> I do agree with that.
>
> > My idea of infallible is "guaranteed to always succeed", not "maybe
> > failing but handling the failure in a generic way for you".
>
> Nothing can be guaranteed to always succeed.
>
> > I didn't describe "a way", I described multiple potential methods of
> > handling OOM (where which is best depends on the specific case).
> >
> > I won't waste my time finding more examples of each possible way of handling OOM just so you can say
> > "Oh, it only uses method A and not method B" again; especially when my main point is that overcommit
> > has led to software not doing these things; and especially after you spent days trying to convince
> > me that FireFox does do (some of) these things (and is an example that you're asking for).
>
> The only method you pointed out to is "explicitly handle malloc() returning NULL".
> I haven't seen a codebase doing that that's not abort()'ing by default.
>
> > Can you point to any actual example of a codebase that does not use any of the multiple
> > potential methods I've described, that is "not shit" (not improvable) and is not
> > intended for a "no multi-tasking, all computers identical" environment?
>
> I don't understand the question, but since you haven't brought
> any examples to prove your point I don't think it matters.
>
> > What, exactly, do you think is funny about (e.g.) preventing rogue processes from
> > tricking the OS into killing system services/daemons or the entire GUI?
>
> If you exhaust commit-space on Windows, memory allocations in system
> processes will start to fail too and make the system unusable.

That looks like a system service (StorageUsage.dll) leaking memory and causing its own OOM; not a normal process causing problems for a "not buggy" system service.

> System-wide OOM is bad on every machine. But here we weren't talking about system-wide
> OOM, we were talking about user-space applications crashing while there's still
> plenty of memory available because Windows doesn't overcommit memory.
>
> Stop changing the topic.

Fine.

The right way to think about the way memory management works in Windows (unless swap is disabled by an idiot) is to think of physical memory as nothing more than a cache of the page file; where "max. virtual memory size = page file size". Physical memory size and physical memory availability are irrelevant - they have no effect on OOM whatsoever.

For Linux; it's mostly the opposite - e.g. "max. virtual memory = physical + swap". This difference between Windows and Linux is causing your delusions about the importance of physical memory availability on Windows.

When user-space applications get OOM in response to allocation failures (and either handle them or crash, depending on how good/bad they are and what the developer felt like doing) Windows is correctly doing the right thing (ensuring it can meet its promises by ensuring memory it committed to providing can be provided, while "accidentally" avoiding the insanity of "OOM killer + broken work-arounds to hide symptoms" that are the consequences of a kernel's failure to meet its promises). This may happen when the irrelevant amount of free physical memory is irrelevant, but that is not important because user-space software is only supposed to care about virtual memory (swap space) while the kernel takes care of "deliberately abstracted" lower level details (whatever kernel felt like caching in physical memory, which may even include things like data from pre-fetched files that haven't been requested (yet) and aren't part of any process' virtual memory).

- Brendan