Hi,

Jukka Larja (roskakori2006.delete@this.gmail.com) on January 22, 2020 6:30 am wrote:
> Brendan (btrotter.delete@this.gmail.com) on January 21, 2020 5:04 pm wrote:
> > Jukka Larja (roskakori2006.delete@this.gmail.com) on January 21, 2020 6:01 am wrote:
> > > Brendan (btrotter.delete@this.gmail.com) on January 20, 2020 11:43 am wrote:
> > > > Jukka Larja (roskakori2006.delete@this.gmail.com) on January 20, 2020 4:54 am wrote:
> > > >
> > > > > With oracle telling us that the end is near, these are practical approaches (not on consoles
> > > > > in my opinion, but on general purpose system). Unfortunately we don't have oracle.
> > > >
> > > > Would you like me to write an oracle that "predicts" that a previous call to malloc()
> > > > returned NULL, that a previous call to "mmap()" returned -1 (with errno set to ENOMEM),
> > > > or that a previous attempt to create a new object resulted in "std::bad_alloc"?
> > > >
> > > > It seems to me that for all cases the oracle can be a macro that does literally nothing (or maybe
> > > > just a line of white space inside a "try to allocate and free cache if you need to" wrapper).
> > >
> > > This is pointless. You are talking about some toy programs where freeing (some mysterious) cache can
> > > be done at any point and no one will care.
> >
> > No, the same approach can/will work for anything that has any kind of cache, regardless of the complexity.
> >
> > > Any such memory in our game engine is such that it can't
> > > be freed before the operation using it is done.
> >
> > If none of the memory can be freed because it's being used then your game engine has no cache/s.
>
> What's the point of cache if no-one can touch it, except to free it?

The point of a cache is to improve performance by storing data that isn't currently needed, so that if the data is needed in future it can be fetched faster. A cache has 3 basic operations - evicting something from the cache (e.g. because the data became stale), adding something to the cache (which may cause other data to be evicted to make room), and retrieving something from the cache (because it became needed).

My point was that if the data is currently being used then it is not a cache by definition.

More specifically, I was "too subtly" suggesting that you chose the wrong approach for your situation (because you don't know what "cache" means) and are complaining that you made the wrong choice for your situation.

> > > That may be OK, if that operation itself faces OOM
> > > and can abort, but that's unlikely. Much more likely the OOM will happen in some other thread. And
> > > when you get one, you'll probably get several more soon. That's why the oracle is needed.
> >
> > You don't need an oracle to do this. E.g. you can inform other threads that they need
> > to reduce the amount of memory they're currently using (and wait until the other threads
> > inform you that they have); without any attempt to predict the future.
>
> How?

However you want. I mostly use messages/events (ie. would use "sendMessage()" to tell other threads I need them to free memory and "getMessage()" to wait for them to tell me they have). Other people may use anything else (pipes, sockets, shared memory, ...).

> > > And you are running your toy programs on some mysterious system that seems to be constantly running out
> > > of memory. I mean, why else would you spend so much time trying to handle OOMs without crashing?
> >
> > I (try to) handle every possible problem that might happen, regardless of how likely/unlikely it is;
>
> That's extremely stupid engineering. Seems like you have
> infinite programmer resources, but lacking hardware.

For multi-tasking systems (many different programs competing for resources) on normal PCs (no clue what hardware the software might be running on); you can not assume that the resources you felt using will actually be available, so assuming that the resources might not be available is necessary to consider yourself competent. As I pointed out previously, games on game consoles are not like this.

> > Note: For previous work (operating systems) I don't even assume hardware works
> > (e.g. I've probably spent 6 months just researching/prototyping "software/kernel
> > based RAM fault detection and correction" for systems that don't have ECC).
>
> You do understand how utterly irrelevant that is for current discussion, on so many levels?

I wrote that in the hope that you'd get a better understanding of what I expect (in terms of resilience) from my software, and in the hope that you'd be better able to understand why I care about crappy software that crashes due to OOM and loses important data because the developer is incompetent. Of course, as I pointed out previously, games on game consoles are not like this.

> > The software I'm currently working on is half "collaborate real-time editor"
> > (see https://en.wikipedia.org/wiki/Collaborative_real-time_editor
> > ). Basically; users log in to the server, select a project/s they want to work on, then edit the project/s
> > (while many other users may also be editing the same project/s).
> > If the server crashes all users get annoyed
> > (then probably try to reconnect before server comes back
> > online, then realize they have no idea how long server
> > will be down for and probably take a coffee break); then (after the server comes back online) all users log
> > back in and have to try to figure out how much of their work was lost from which file/s, then users have to
> > redo the work that was lost. If you assume (as a crude estimate for design goal purposes) there will be 100
> > users and the disruption costs each user an average of 20 minutes (e.g. auto-save every modified file every
> > 5 minutes); then it'd add up to about 33 hours of lost productivity,
> > and you might be able to say it'll costs
> > about $1000 each time the server crashes (despite auto-saves).
>
> Sounds like it would be much better to spend some money on more
> memory than implementing crazy OOM survival strategies.

How do you think that would work? I set up a web site where people can download my software, and every time someone clicks on a "download" link I issue a voucher to cover the cost of a motherboard replacement (in case their existing motherboard can't support more RAM) plus an infinite amount of RAM to go with it (because there's no limit to the amount of work they can expect my software to do and no limit to how much other software they might also be running); and then I still have to worry about allocations failing because it became possible for my software to reach the "max. of about 128 TiB per process" virtual address space size limit on most 80x86 PCs? Then what? I split "server" into many separate processes distributed across many separate computers so that I have to worry about handling "Out Of Computers" errors instead?

I think I'd rather be "crazy" and do almost nothing to reuse the code to discard cached data that I have to have anyway (in case the other software changes files on disk) and reuse the code to handle errors (and send "Sorry, can't do that because ..." back to clients) that I have to have anyway (in case of "permission denied" or "someone deleted it while you were editing" or "thread crashed" or any of the many other errors that have nothing to do with OOM).

- Brendan