Hi,

Jukka Larja (roskakori2006.delete@this.gmail.com) on January 20, 2020 4:54 am wrote:
> Brendan (btrotter.delete@this.gmail.com) on January 19, 2020 11:25 am wrote:
> > Jukka Larja (roskakori2006.delete@this.gmail.com) on January 18, 2020 9:13 pm wrote:
> > > Brendan (btrotter.delete@this.gmail.com) on January 18, 2020 11:40 am wrote:
> > >
> > > > (which could include dropping cached data, and/or waiting until another thread finishes and
> > > > releases resources,
> > > > and/or refusing to accept additional work to ensure already accepted work succeeds,
> > >
> > > I don't really see how one is supposed to do these things without trying
> > > to allocate any new memory, except in trivial toy examples.
> >
> > Why is this hard to figure out? Don't forget that you can pre-allocate (e.g. at program
> > startup/initialization, or at compile time) to avoid allocations during error handling.
>
> If everyone starts preallocating, system just runs out faster. If the preallocation is just to print
> some error or to show a popup, the amount doesn't need to be huge, but you still have to be able
> to somehow write to log file or show that popup without any part of that requiring any allocation.
> I have no idea if you can do either in Windows, even less so in language independent way.
>
> Our code generally speaking tries to handle errors using just stack allocations, but if something produces
> really long error message (about 4000 bytes), pushing that out will require stack allocation (that's even
> in case that message was generated within stack). That's just our code though. I have no idea how many
> allocations the Windows dialog does or if file writes generate some allocations outside our code.

Yes, I meant "preallocate what you need to handle errors" (not "preallocate everything you'll ever need").

Handling errors with stack allocations makes perfect sense (until the OS supports overcommit and you end up getting OOM when accessing more stack, and some fool tries to be helpful by suggesting you try to write a signal handler for SIGBUS that "works" when there's no memory for signal handler's stack).

Most GUIs I've used allow you to create an invisible dialog box, so (most?) resources are already allocated and you only need to set a flag to make it visible (for Windows, maybe https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-showwindow ).

> The big issue here, however, is how are you supposed to free some cache when an allocation
> fails in random place in code? You can't just drop it without some synchronization and then
> you are likely just going to deadlock. And that's just a simple cache. I don't think we have
> any of those. The places where we could free extra buffers are much more complex.

Either you can free some cache, or you can return "Failed, free some cache and retry" status back to the caller.

> > > > and/or reporting failure
> > > > in a way that is consistent with the way all other problems are reported, and/or anything else); and a
> > > > software developer should not be prevented from doing whatever they think is sane because they're forced
> > > > to accept something that is almost never sane (either OOM killer, or SIGBUS that is unusable for most
> > > > cases, or constant monitoring that can't solve the problem due to race conditions).
> > >
> > > How exactly is getting an OOM on memory allocation so much better than getting it at some other random
> > > time via signal?
> >
> > Do you honestly expect me to believe that you don't think you can handle OOM (e.g. without
> > allocations, etc) when it happens at the most convenient place you choose, but do think you
> > can handle OOM when it happens at the least convenient place that you don't choose?
>
> I'm saying it doesn't matter much, because it will be inconvenient place anyway. Convenient
> thing to do could be e.g. to tell the caches and similar places to reduce memory consumption
> as soon as possible. If I'm allowed to make extra preallocations "just in case", they can be
> (depending on which case we have here. OOM on allocation or OOM signal) used or freed and then
> we just hope that's enough until the caches and other places start freeing memory too.
>
> > > In typical code, most allocations probably aren't done by applications own code, but some
> > > library. For example, put some std::string or std::vector to stack and they will allocate from heap too.
> > > If there's a system wide OOM going on, how can you even expect to be able to write a log message?
> >
> > Stack is pre-allocated (and without overcommit the OS guarantees that the pre-allocated stack
> > is safe to use and you won't trigger OOM by using it), when you create an simple object (e.g.
> > std:string, std:vector) you allocate memory (not the library that contains the objects class,
> > constructor, etc) and you will (unless you intentionally broke C++) either get a reference that
> > is NULL (that you must check yourself) or a "std:bad_alloc" exception (that you must catch).
>
> And then you have bad_alloc exception handling code practically everywhere.
> That's just crazy talk for an issue that practically never happens.

Why "everywhere"? The entire point of exceptions is that you can catch them wherever you want. For a very simple game, maybe that means you have a "try { do_game_tick(); tick++; } catch (const std::bad_alloc&) { free_cached_stuff(); }" in one place in the game's main loop.

> Of course, we don't use exceptions on most platforms, but since that may not be
> that useful these days, I can agree that it may be mostly irrelevant edge case.

Without exceptions, you'd end up with everything returning status back (e.g. things like "if( foo() == FAILED) { return FAILED; }" everywhere), which is the problem that exceptions were created to avoid.

> > In theory there might be a library that allocates memory itself without giving you any hope of detecting
> > that when the allocation/s fail. I have never seen this and I doubt it actually exists in practice. Anybody
> > that is too incompetent to write a usable library will not write a library that anyone will use.
>
> On one of the console platforms you just crash when certain libraries fail to allocate
> memory (from preallocated and pretty limited general heap). You'll get an error message,
> so you know to increase the preallocation, but there's nothing to do runtime.
>
> That's by design, but I don't see why many more codes wouldn't
> be written in the same way. It certainly isn't hard.

That's game consoles though (every computer has same amount of RAM, no resource is shared with other software, end user's data has no practical value).

> > > > Yes; but even for games, "what makes sense for the game" can depend on the game. For example, maybe for
> > > > one game it makes sense to do an auto-save before changing to a different level/zone, then load everything
> > > > and pre-allocate everything for the next level/zone (while managing allocation failures), then not have
> > > > a reason to care while the player remains in that level/zone
> > > > because you know everything was already allocated
> > > > successfully; and maybe for a different game that approach
> > > > can't make sense because there are no levels/zones
> > > > (or there's multiple players or ...) and something very different is far more suitable.
> > >
> > > Yeah, sure. If things happen to be just right, then it
> > > makes sense to do X. The game will need extra buffers
> > > to load everything the level needs from disk. They will be needed right until everything is loaded, at
> > > which point the memory consumption will be the highest (add some random variation due to multithreading
> > > and the fact that the player may decide to put the console to sleep in the middle of this).
> > >
> > > You are also completely ignoring that memory fragmentation
> > > gets really bad really fast, if you are constantly
> > > running near the limit. If your normal mode of operation is doing something in response to allocation
> > > failures, you're screwed anyway. All that crazy amount of error handling for nothing (note that this is
> > > no longer about system wide OOM, but strictly what's happening inside a game on a console).
> >
> > If you're constantly running near the limit; then the user is constantly being
> > told "Sorry, you can't do that because their isn't enough memory", given that:
>
> I'm about 99.999% sure we can't tell user that. Did you miss the part that this wasn't
> about system wide OOM, but about running in fixed size memory on console?
>
> > a1) if the software happens to support a "deallocate unnecessary data to increase the chance
> > of success" approach, then it will reach a point where there is nothing left for the software
> > to deallocate; causing the software to fall back to some other approach.
> >
> > a2) if the software happens to support a "reduce quality of work to increase the chance
> > of success" approach (e.g. maybe switching to lower resolution textures, etc), then
> > it will reach a point where there the quality of work has reached its minimum and
> > can't be reduced; causing the software to fall back to some other approach.
>
> With oracle telling us that the end is near, these are practical approaches (not on consoles
> in my opinion, but on general purpose system). Unfortunately we don't have oracle.

Would you like me to write an oracle that "predicts" that a previous call to malloc() returned NULL, that a previous call to "mmap()" returned -1 (with errno set to ENOMEM), or that a previous attempt to create a new object resulted in "std::bad_alloc"?

It seems to me that for all cases the oracle can be a macro that does literally nothing (or maybe just a line of white space inside a "try to allocate and free cache if you need to" wrapper).

> > d) the only other approach (that I can think of) is "cancel that one piece of work and
> > inform the user, so all other pieces of work can continue" (possibly as a fall-back).
>
> This too, is hard when OOM has already happened. With oracle we could avoid spawning new work.
>
> -JLarja

- Brendan