By: Carson (carson.delete@this.example.edu), March 5, 2021 3:31 am
Room: Moderated Discussions
Etienne Lorrain (etienne_lorrain.delete@this.yahoo.fr) on March 4, 2021 6:58 am wrote:
> You cannot "only initialising a page when it was allocated" because the BIOS do not allocate
> memory, it just tells you how much memory is available, and the concept of page is not known
> (no virtual memory at that point). Moreover the page size is decided by the OS, processor support
> different sizes. Moreover memory over 1 Megabyte was special, either EMS or XMS or HMA.
> And you could not modify DOS, proprietary software with no source.
> The real missing thing was (and still partly is) a proper DMA, able to send more than 64 Kbytes
> and able to access more than 1 Megabyte / 16 Mbytes with chipset recognition and NDA docs.
These all seem like non-issues.
For OSes which cannot handle uninitialized ECC, have a forward-compatibility flag somewhere in the boot loader which means "I can handle uninitialized ECC". If it's not present, BIOS clears memory before transferring to the boot loader. A chaining boot loader (LILO or whatever) is required to do the same check on OSes is chain-loads. (By calling back into the BIOS which has all the necessary code, so the chain loader doesn't suffer much code bloat.)
For loaders which do support uninitialized ECC, a simple BIOS data structure (like the 0xE820 memory map) describes the initialized parts, and there are BIOS calls to extend the initialized parts.
The loader asks the BIOS to initialize enough to hold whatever it's chain-loading, and that loader does the same for its own bss and initial stack.
Then the OS's early boot probes the hardware to find out if it's capable of talking to the ECC hardware without BIOS support. If not, before the BIOS is fully disabled, fall back to asking it to initialize everything.
With all of those fallbacks implemented, the hopefully common case is that the OS does know how to drive the ECC hardware and it initializes its own memory allocator with all unallocated memory flagged "ECC bad". The first time that memory is allocated, it is initialized (which may be as simple as CLZERO).
Since the OS is doing all of this, it understands the page size and all the necessary rules. The important point is that the initialization is done lazily, after applications have started running.
A more sophisticated OS might extend the idea of "potentially uninitialized page" beyond the memory allocator proper, and things like disk DMA which are going to overwrite the whole page could elide the initialization. Since this overlaps heavily with the zeroing required for security, it's not actually a major project.
This all seems like a pretty straightforward SMOP.
> You cannot "only initialising a page when it was allocated" because the BIOS do not allocate
> memory, it just tells you how much memory is available, and the concept of page is not known
> (no virtual memory at that point). Moreover the page size is decided by the OS, processor support
> different sizes. Moreover memory over 1 Megabyte was special, either EMS or XMS or HMA.
> And you could not modify DOS, proprietary software with no source.
> The real missing thing was (and still partly is) a proper DMA, able to send more than 64 Kbytes
> and able to access more than 1 Megabyte / 16 Mbytes with chipset recognition and NDA docs.
These all seem like non-issues.
For OSes which cannot handle uninitialized ECC, have a forward-compatibility flag somewhere in the boot loader which means "I can handle uninitialized ECC". If it's not present, BIOS clears memory before transferring to the boot loader. A chaining boot loader (LILO or whatever) is required to do the same check on OSes is chain-loads. (By calling back into the BIOS which has all the necessary code, so the chain loader doesn't suffer much code bloat.)
For loaders which do support uninitialized ECC, a simple BIOS data structure (like the 0xE820 memory map) describes the initialized parts, and there are BIOS calls to extend the initialized parts.
The loader asks the BIOS to initialize enough to hold whatever it's chain-loading, and that loader does the same for its own bss and initial stack.
Then the OS's early boot probes the hardware to find out if it's capable of talking to the ECC hardware without BIOS support. If not, before the BIOS is fully disabled, fall back to asking it to initialize everything.
With all of those fallbacks implemented, the hopefully common case is that the OS does know how to drive the ECC hardware and it initializes its own memory allocator with all unallocated memory flagged "ECC bad". The first time that memory is allocated, it is initialized (which may be as simple as CLZERO).
Since the OS is doing all of this, it understands the page size and all the necessary rules. The important point is that the initialization is done lazily, after applications have started running.
A more sophisticated OS might extend the idea of "potentially uninitialized page" beyond the memory allocator proper, and things like disk DMA which are going to overwrite the whole page could elide the initialization. Since this overlaps heavily with the zeroing required for security, it's not actually a major project.
This all seems like a pretty straightforward SMOP.
Topic | Posted By | Date |
---|---|---|
CPU & Memory bit flips | Ganon | 2021/03/03 10:05 AM |
Also "Silent Data Corruption" | Adrian | 2021/03/03 11:42 AM |
Thanks for the reference | Ganon | 2021/03/03 12:47 PM |
Implications for linux page cache | anon | 2021/03/03 12:54 PM |
Implications for linux page cache | Linus Torvalds | 2021/03/03 02:54 PM |
memory errors | blaine | 2021/03/03 03:53 PM |
memory errors | anon2 | 2021/03/03 06:30 PM |
memory errors | dmcq | 2021/03/04 06:16 AM |
memory errors | Etienne Lorrain | 2021/03/04 07:26 AM |
memory errors | dmcq | 2021/03/04 07:40 AM |
memory errors | Etienne Lorrain | 2021/03/04 07:58 AM |
memory errors | dmcq | 2021/03/04 08:12 AM |
memory errors | Carson | 2021/03/05 03:31 AM |
memory errors | Etienne Lorrain | 2021/03/05 07:23 AM |
memory errors | rwessel | 2021/03/05 08:48 AM |
memory errors | dmcq | 2021/03/05 01:01 PM |
memory errors | rwessel | 2021/03/05 01:23 PM |
memory errors | dmcq | 2021/03/05 01:51 PM |
memory errors | Brendan | 2021/03/06 12:38 AM |
memory errors | Carson | 2021/03/06 02:35 AM |
memory errors | Carson | 2021/03/06 07:24 AM |
memory errors | David Hess | 2021/03/04 02:44 PM |
memory errors | rwessel | 2021/03/04 06:14 PM |
memory errors | Linus Torvalds | 2021/03/04 09:21 PM |
memory errors | anon2 | 2021/03/04 10:46 PM |
memory errors | Carson | 2021/03/05 03:43 AM |
memory errors | anon2 | 2021/03/05 08:55 AM |
memory errors | gallier2 | 2021/03/05 03:22 AM |
memory errors | dmcq | 2021/03/05 01:59 PM |
memory errors | David Hess | 2021/03/06 05:27 AM |
memory errors | Carson | 2021/03/06 07:44 AM |
memory errors | Gabriele Svelto | 2021/03/06 11:11 AM |
memory errors | David Hess | 2021/03/06 11:28 AM |
memory errors | Michael S | 2021/03/06 03:45 PM |
memory errors | Doug S | 2021/03/04 11:48 AM |
memory errors | Michael S | 2021/03/04 12:36 PM |
memory errors | Jörn Engel | 2021/03/04 04:32 PM |
memory errors | Linus Torvalds | 2021/03/04 09:47 PM |
memory errors | Etienne Lorrain | 2021/03/05 02:09 AM |
memory errors | Michael S | 2021/03/05 05:06 AM |
memory errors | Linus Torvalds | 2021/03/05 12:59 PM |
memory errors | rwessel | 2021/03/05 01:32 PM |
memory errors | rwessel | 2021/03/05 01:37 PM |
memory errors | zArchJon | 2021/03/06 09:39 PM |
memory errors | Gabriele Svelto | 2021/03/06 01:58 PM |
memory errors | Jörn Engel | 2021/03/05 11:12 AM |
Amiga recoverable RAM disk? | Carson | 2021/03/05 04:03 AM |
Thanks - TIL a cool Amiga feature (nt) (NT) | John | 2021/03/05 01:51 PM |
Another cool Amiga feature, datatypes | Charles | 2021/03/06 01:01 AM |
Another cool Amiga feature, datatypes | Jukka Larja | 2021/03/06 02:23 AM |
Another cool Amiga feature, datatypes | Anon | 2021/03/06 01:40 PM |
Another cool Amiga feature, filesystems | Marcus | 2021/03/07 01:28 AM |
CPU & Memory bit flips | zArchJon | 2021/03/04 07:39 AM |
CPU & Memory bit flips | dmcq | 2021/03/04 07:59 AM |
CPU & Memory bit flips | rwessel | 2021/03/04 01:27 PM |
speak of the devil | Robert Williams | 2021/03/05 08:53 AM |
speak of the devil | dmcq | 2021/03/05 12:26 PM |
speak of the devil | Robert Williams | 2021/03/05 04:15 PM |