Brendan (btrotter.delete@this.gmail.com) on January 22, 2020 2:56 pm wrote:
> Jukka Larja (roskakori2006.delete@this.gmail.com) on January 22, 2020 6:30 am wrote:
> > Brendan (btrotter.delete@this.gmail.com) on January 21, 2020 5:04 pm wrote:
> > > Jukka Larja (roskakori2006.delete@this.gmail.com) on January 21, 2020 6:01 am wrote:
> > > >
> > > > This is pointless. You are talking about some toy programs where freeing (some mysterious) cache can
> > > > be done at any point and no one will care.
> > >
> > > No, the same approach can/will work for anything that has any kind of cache, regardless of the complexity.
> > >
> > > > Any such memory in our game engine is such that it can't
> > > > be freed before the operation using it is done.
> > >
> > > If none of the memory can be freed because it's being used then your game engine has no cache/s.
> >
> > What's the point of cache if no-one can touch it, except to free it?
>
> The point of a cache is to improve performance by storing data that isn't currently needed, so that if the
> data is needed in future it can be fetched faster. A cache has 3 basic operations - evicting something from
> the cache (e.g. because the data became stale), adding something to the cache (which may cause other data
> to be evicted to make room), and retrieving something from the cache (because it became needed).
>
> My point was that if the data is currently being used then it is not a cache by definition.
>
> More specifically, I was "too subtly" suggesting that you chose the wrong approach for your situation (because
> you don't know what "cache" means) and are complaining that you made the wrong choice for your situation.

Ah, semantics. By that definition, sure, we don't have caches anywhere. We do have things that hold repetably used data, but when accessed don't make a copy of that data, but instead hand out a reference, thus causing the whole structure to turn into something other than cache. We use them instead of cache, because they are faster and waste less memory, thus helping us avoid OOMs.

> > > > That may be OK, if that operation itself faces OOM
> > > > and can abort, but that's unlikely. Much more likely the OOM will happen in some other thread. And
> > > > when you get one, you'll probably get several more soon. That's why the oracle is needed.
> > >
> > > You don't need an oracle to do this. E.g. you can inform other threads that they need
> > > to reduce the amount of memory they're currently using (and wait until the other threads
> > > inform you that they have); without any attempt to predict the future.
> >
> > How?
>
> However you want. I mostly use messages/events (ie. would use "sendMessage()" to tell
> other threads I need them to free memory and "getMessage()" to wait for them to tell me
> they have). Other people may use anything else (pipes, sockets, shared memory, ...).

We would need to check the messages before every allocation.

> > That's extremely stupid engineering. Seems like you have
> > infinite programmer resources, but lacking hardware.
>
> For multi-tasking systems (many different programs competing for resources) on normal PCs (no clue what
> hardware the software might be running on); you can not assume that the resources you felt using will
> actually be available, so assuming that the resources might not be available is necessary to consider
> yourself competent. As I pointed out previously, games on game consoles are not like this.

Yeah, on PC we print "Required: 4 GB, recommended: 8 GB" on the box. Not that hard. We can probably manage with 2, but it's good to consider that most users have some other stuff running too.

We could manage with less, with longer loading times, less predictable performance or more code complexity, but like with most software these days, chasing "perfect" doesn't make sense.

> > > Note: For previous work (operating systems) I don't even assume hardware works
> > > (e.g. I've probably spent 6 months just researching/prototyping "software/kernel
> > > based RAM fault detection and correction" for systems that don't have ECC).
> >
> > You do understand how utterly irrelevant that is for current discussion, on so many levels?
>
> I wrote that in the hope that you'd get a better understanding of what I expect (in terms of resilience)
> from my software, and in the hope that you'd be better able to understand why I care about crappy
> software that crashes due to OOM and loses important data because the developer is incompetent.

But why pick OOM? I don't even remember when was the last time I saw something crashing due to OOM. Instead I had at least one crash and several cases of incorrect results from various software just today. I'd rather the creators of those software didn't spend their time on some niche OOM crash protection, which may not even help (if the OOM was mostly caused by some other software or user error, including running too many parallel compiles on his workstation).

> Of course, as I pointed out previously, games on game consoles are not like this.

As I pointed out earlier, if a game on a console crashes in platform owner's tests, it pretty much automatically fails the submission. In the bad old days (Xbox 360 and PS3), resubmission after two failures cost tens of thousands of dollars, so console games don't usually crash much (especially games by small developers, who can't bribe the platform owner with expected hit status).

What's easier for games is that they aren't bothered (much) by other software, always have certain amount of resources available and are, when it comes to memory management, probably couple of orders of magnitude simpler than e.g. Firefox.

> > Sounds like it would be much better to spend some money on more
> > memory than implementing crazy OOM survival strategies.
>
> How do you think that would work? I set up a web site where people can download my software, and every time
> someone clicks on a "download" link I issue a voucher to cover the cost of a motherboard replacement (in
> case their existing motherboard can't support more RAM) plus an infinite amount of RAM to go with it (because
> there's no limit to the amount of work they can expect my software to do and no limit to how much other software
> they might also be running); and then I still have to worry about allocations failing because it became possible
> for my software to reach the "max. of about 128 TiB per process" virtual address space size limit on most
> 80x86 PCs? Then what? I split "server" into many separate processes distributed across many separate computers
> so that I have to worry about handling "Out Of Computers" errors instead?

I'd just put a "requires about X of memory to run + Y for every concurrent user (+ Z for blahblah). It probably doesn't need to be accurate. If it requires such significant amounts of cache, the area between "runs just about well enough" and "OOMs" is prabably pretty big.

If you are doing something requiring over 128 TBs of memory, then whatever you are doing isn't relevant for pretty much all the software on Earth.[1]

If you are willing to work for free, without slowing down other work on the software, I'm sure your clients are happy for your OOM shield.

If you expect people to pay you thousands of dollars for less memory consumption instead of paying couple hundred for more RAM, that's just bad engineering (I don't know what you expect. You talk like it's trival to add, and perhaps it is for you. Maybe you don't feel the need to charge any extra, since it's so trivial. Maybe you are a supercoder. They do exists, I know of one. Unfortunately rest of the about 30 I know about, including me, aren't, so they can't just add OOM handlers without additional cost).

[1] I have a server code that caches maybe a gigabyte of data (probably tens of megabytes is enough, if I want to save memory) and doesn't consume significant amounts of memory in addition to that. Unlike the non-caches I've talked about before, this is actually something that could be pretty easily dropped, though there's still the data copy phase that could cause a deadlock due to OOM inside and outside of said code (bloating the code with exception handlers would be mostly straight forward, though probably some places would be accidentally missed). It is unlikely that it will run on a computer with less than 64 GB RAM and 64-128 GB page file.

Do you think it makes sense to do OOM protection for this software?

-JLarja