Hi,

Gabriele Svelto (gabriele.svelto.delete@this.gmail.com) on January 15, 2020 5:56 am wrote:
> Brendan (btrotter.delete@this.gmail.com) on January 15, 2020 5:20 am wrote:
> > No, you're assuming overcommit exists and its leading you to false assumptions.
> >
> > If an allocation fails and there is overcommit; yes, you're probably screwed regardless.
> >
> > If an allocation fails and there is no overcommit; then it's not a problem at all -
> > you're able to do anything you like to handle that (without needing any notifications)
> > without worrying about "OOM killer" preventing you from doing anything sane.
>
> No, you assume that you can't detect the OOM killer killing you. You can, you just need a handler
> for SIGBUS. The only scenario in which you can't detect the OOM killer is IIRC when the kernel
> needs memory and then it will take it no matter what. I don't know how the Windows kernel deals
> with that but I doubt it will fall over and panic instead of killing a user process.

Oh, that sounds easy. When my process receives a new TCP/IP connection request it creates a thread for it (memory for thread stack, etc), then allocates thread local data I use to keep track of the connection and some buffers, and because all that "succeeded" I send a "You connected, everything is fine, here's RSA public key info for login details" packet back to the client. Then, because of what I allocated, a completely different process that has nothing to do with me gets a SIGBUS signal and (because apparently every process anyone will ever write will immediately know about my thread's memory usage) somehow will inform my process (without using any extra memory) that the kernel is an idiotic piece of shit. Then my process travels back in time so that it can tell the client "Sorry, not enough resources for a new connection at this time" so that the end user (who is using a different computer) can see a nice dialog box telling them why they won't be able to log in.

> > No, it's easy to pre-allocate whatever you need to handle errors like this (e.g. during process
> > startup) to ensure you don't need to allocate when you can't. Sometimes that's not even necessary
> > - in my case (server with one thread per connection) I can output to log without allocating
> > and then terminate the thread/connection (which also causes its "thread local data" to be
> > freed and helps other threads/connections avoid future allocation failures).
>
> OK, your assumption of "handling OOM situations correctly" is:

• You can recover from
  > the failure of every allocation
• You will have error handling for every allocation

>

That's not possible unless your handling is "OMG we're out of memory, spit something on the
> log and abort". That is not what I call handling an OOM condition. We have plenty of fallible
> allocation points in Firefox where we can actually fully recover from an allocation failure
> and go on, but it's not possible to do that for every single allocation, especially if you
> don't control all your dependencies (and that's practically impossible nowadays).

You control which dependencies you depend on; and if you choose to depend on a hideous design failure (e.g. that has functions that need to allocate memory but don't tell you they failed and force you to assume something succeeded when it didn't) then that is your fault for making bad choices.

> That also assumes that all allocation points (including indirect ones) have proper checking. Because if they don't
> you'll have NULL pointers spreading through your data possibly causing a random crash at a later point where you
> can do absolutely nothing about it. You can't just assume that software has perfect error detection.

Um, what?

If there's a library function that always returns "success" (but the docs don't say that success is guaranteed in all future versions of the library) and another programmer calls it without checking the return value (that is always "success"), then I tell the other programmer they're incompetent and refuse to work with that programmer ever again (because a future version of that library might return something else).

Do you honestly think I worry about "without proper checking you'll have NULL pointers spreading everywhere"?

> > Of course if you do have overcommit, then things like modifying data or writing to a log
> > file could cause "commited to but not actually allocated" pages to be allocated (and trigger
> > "OOM killer"); so trying to handle OOM conditions does become a "tip-toe through the minefield"
> > nightmare scenario that you've falsely assumed applies to "without overcommit".
>
> I'm not sure what you're talking about. The OOM killer will send you a signal which you can handle in a signal
> handler and you can even resume from it.

Have you ever tried to use signals before? You get a signal while your in the middle of who-knows-what; then what? Cause a deadlock while trying to acquire a lock you need to free memory you were using to cache something? Inform a thread that it needs to terminate to save other threads and wait for it to also receive SIGBUS and cause more problems?

> The process doesn't disappear into thin air unless the kernel is running
> out of memory and then it's obviously better to kill a user process than the entire machine. As I pointed out
> the above applies to Windows as well. Also, you can control the OOM killer from user-space.

LOL. It's not possible for a sane person to read the first 2 paragraphs of that page without loud alarm bells screaming "WTF? WTF? WTF?" going off in their head. The entire history of "OOM killer" on Linux (all the complaints, all the "admin tunables/knobs", all the extra interfaces slapped on top) is like a long running series of "The Three Stooges Attempt To Work Around Symptoms of a Batishit Insane Design Flaw".

> > https://stackoverflow.com/questions/45605806/out-of-memory-allocating-bytes
> >
> > Note: "failed to allocate" was correctly handled by logging/reporting the error (instead
> > of letting a random and potentially unrelated process get terminated by "OOM killer").
>
> LOL. GCC is just quitting at that point possibly leaving behind a half-written
> object file. I don't see how that's better than dying at a random point.
>
> GCC is also not a really good example because it handles SIGBUS on Linux
> and when it does it tries to quit more gracefully (though not by much).

Yes; GCC was just the easiest example of "bare minimum handling of allocation failures".

> > You're being disingenuous by assuming a system without overcommit
> > will be set up the same as a system without overcommit.
> >
> > If you have one system that overcommits, and another system that doesn't overcommit but is given
> > adequate swap space to ensure that it can handle the exact same load as the first system; then
> > the only difference will be that the first system has "OOM killer" and may terminate unrelated
> > processes (which is incredibly idiotic) and the second system will be able to handle OOM in
> > whatever manner the developers wanted to (which can be significantly less idiotic).
> >
> > The important thing is how OOM is handled, not how many cents
> > worth of extra swap space are needed to avoid idiocy.
>
> You're handwaving stuff away without presenting data to back your assumptions. I have crash data
> - from million of Firefox users - that shows over two thirds of OOM crashes on Windows happening
> on machines with plenty of physical memory available. That means that Windows lack of overcommit
> causes a very large amount of crashes that would simply not happen on a system with overcommit.

Heh, no. What you have is proof that a well known resource hog is actually a resource hog. Look at the crash reports - they're all saying "90%+ of system memory used, available page file size reduced to zero". Tell me, how do you think Windows is supposed to hibernate properly when you've gobbled 100% of RAM and 100% of disk? The only problem I see here is that Windows isn't using per-process quotas to stop your memory leak sooner.

As far as I can tell (I'm "not great" at C++ and only looked at small scraps of a large project) the reason they're crashes (and not "reductions in amount of data cached for later" or "dialog boxes telling user one of their tabs needs to be closed" or anything else that could be considered acceptable) is that FireFox's screws up "new()" so that (despite the "noexcept(false)") it fails to throw an exception when there's OOM (preventing the caller, who has the knowledge needed to figure out the best way to handle OOM and/or can send the error up the call stack to whatever is most suited to handling the OOM, from being involved) and just ends up at an incredibly stupid "We might proceed to a stage 2 in which an attempt is made to reclaim memory" comment in unfinished code that will never attempt to reclaim memory (because actually trying to reclaim memory would be horrifically complex when you don't have any of the caller's knowledge about the current context - e.g. which mutexes are currently held, etc).

- Brendan