By: Etienne Lorrain (etienne_lorrain.delete@this.yahoo.fr), March 5, 2021 7:23 am
Room: Moderated Discussions
Carson (carson.delete@this.example.edu) on March 5, 2021 2:31 am wrote:
> Etienne Lorrain (etienne_lorrain.delete@this.yahoo.fr) on March 4, 2021 6:58 am wrote:
> > You cannot "only initialising a page when it was allocated" because the BIOS do not allocate
> > memory, it just tells you how much memory is available, and the concept of page is not known
> > (no virtual memory at that point). Moreover the page size is decided by the OS, processor support
> > different sizes. Moreover memory over 1 Megabyte was special, either EMS or XMS or HMA.
> > And you could not modify DOS, proprietary software with no source.
> > The real missing thing was (and still partly is) a proper DMA, able to send more than 64 Kbytes
> > and able to access more than 1 Megabyte / 16 Mbytes with chipset recognition and NDA docs.
>
> These all seem like non-issues.
>
> For OSes which cannot handle uninitialized ECC, have a forward-compatibility flag somewhere
> in the boot loader which means "I can handle uninitialized ECC". If it's not present, BIOS
> clears memory before transferring to the boot loader. A chaining boot loader (LILO or whatever)
> is required to do the same check on OSes is chain-loads. (By calling back into the BIOS which
> has all the necessary code, so the chain loader doesn't suffer much code bloat.)
>
> For loaders which do support uninitialized ECC, a simple BIOS data structure (like the 0xE820 memory
> map) describes the initialized parts, and there are BIOS calls to extend the initialized parts.
>
You make me feel old, that is bad on Friday -:)
BIOS was developed at a time where Linux was a sin, you would be made immediately redundant if you installed it on a PC at work (because there was no anti-virus on Linux).
Because DOS did not manage ECC at all, and Windows 3.1/95 started from DOS, the BIOS had to initialise all the memory it knew about.
DOS did not know about EMS or XMS memory, nothing more than 1 Mbyte.
EMS could be provided by an ISA card (i.e. something like PCI), and video/network card BIOS would be on the ISA/PCI card itself, their BIOS did not manage ECC either.
> The loader asks the BIOS to initialize enough to hold whatever it's chain-loading,
> and that loader does the same for its own bss and initial stack.
>
> Then the OS's early boot probes the hardware to find out if it's capable of talking to the ECC hardware without
> BIOS support. If not, before the BIOS is fully disabled, fall back to asking it to initialize everything.
>
> With all of those fallbacks implemented, the hopefully common case is that the OS does know how to drive
> the ECC hardware and it initializes its own memory allocator with all unallocated memory flagged "ECC bad".
> The first time that memory is allocated, it is initialized (which may be as simple as CLZERO).
>
> Since the OS is doing all of this, it understands the page size and all the necessary rules. The important
> point is that the initialization is done lazily, after applications have started running.
>
> A more sophisticated OS might extend the idea of "potentially uninitialized page" beyond the memory allocator
> proper, and things like disk DMA which are going to overwrite the whole page could elide the initialization.
> Since this overlaps heavily with the zeroing required for security, it's not actually a major project.
>
> This all seems like a pretty straightforward SMOP.
Nowadays things have changed, the EFI system is a lot more complicated, its whole behaviour/specification is under NDA (Non Disclosure Agreement), and secure boot will stop you implementing anything which is not approved by the manufacturer - or not supported by Window 10.
So basically none of your sophisticated idea can be implemented, if ECC has to be supported it has to be done by the EFI BIOS (Windows10 do not want to hear about ECC), and you get what Linus was complaining about: a machine-check exception with no way to know even the processor which triggered the ECC error or the address which caused the problem.
That is what people call progress: a nice background image telling you how intelligent you are by having bought this top of the range brand of PC, during the whole Windows boot time.
> Etienne Lorrain (etienne_lorrain.delete@this.yahoo.fr) on March 4, 2021 6:58 am wrote:
> > You cannot "only initialising a page when it was allocated" because the BIOS do not allocate
> > memory, it just tells you how much memory is available, and the concept of page is not known
> > (no virtual memory at that point). Moreover the page size is decided by the OS, processor support
> > different sizes. Moreover memory over 1 Megabyte was special, either EMS or XMS or HMA.
> > And you could not modify DOS, proprietary software with no source.
> > The real missing thing was (and still partly is) a proper DMA, able to send more than 64 Kbytes
> > and able to access more than 1 Megabyte / 16 Mbytes with chipset recognition and NDA docs.
>
> These all seem like non-issues.
>
> For OSes which cannot handle uninitialized ECC, have a forward-compatibility flag somewhere
> in the boot loader which means "I can handle uninitialized ECC". If it's not present, BIOS
> clears memory before transferring to the boot loader. A chaining boot loader (LILO or whatever)
> is required to do the same check on OSes is chain-loads. (By calling back into the BIOS which
> has all the necessary code, so the chain loader doesn't suffer much code bloat.)
>
> For loaders which do support uninitialized ECC, a simple BIOS data structure (like the 0xE820 memory
> map) describes the initialized parts, and there are BIOS calls to extend the initialized parts.
>
You make me feel old, that is bad on Friday -:)
BIOS was developed at a time where Linux was a sin, you would be made immediately redundant if you installed it on a PC at work (because there was no anti-virus on Linux).
Because DOS did not manage ECC at all, and Windows 3.1/95 started from DOS, the BIOS had to initialise all the memory it knew about.
DOS did not know about EMS or XMS memory, nothing more than 1 Mbyte.
EMS could be provided by an ISA card (i.e. something like PCI), and video/network card BIOS would be on the ISA/PCI card itself, their BIOS did not manage ECC either.
> The loader asks the BIOS to initialize enough to hold whatever it's chain-loading,
> and that loader does the same for its own bss and initial stack.
>
> Then the OS's early boot probes the hardware to find out if it's capable of talking to the ECC hardware without
> BIOS support. If not, before the BIOS is fully disabled, fall back to asking it to initialize everything.
>
> With all of those fallbacks implemented, the hopefully common case is that the OS does know how to drive
> the ECC hardware and it initializes its own memory allocator with all unallocated memory flagged "ECC bad".
> The first time that memory is allocated, it is initialized (which may be as simple as CLZERO).
>
> Since the OS is doing all of this, it understands the page size and all the necessary rules. The important
> point is that the initialization is done lazily, after applications have started running.
>
> A more sophisticated OS might extend the idea of "potentially uninitialized page" beyond the memory allocator
> proper, and things like disk DMA which are going to overwrite the whole page could elide the initialization.
> Since this overlaps heavily with the zeroing required for security, it's not actually a major project.
>
> This all seems like a pretty straightforward SMOP.
Nowadays things have changed, the EFI system is a lot more complicated, its whole behaviour/specification is under NDA (Non Disclosure Agreement), and secure boot will stop you implementing anything which is not approved by the manufacturer - or not supported by Window 10.
So basically none of your sophisticated idea can be implemented, if ECC has to be supported it has to be done by the EFI BIOS (Windows10 do not want to hear about ECC), and you get what Linus was complaining about: a machine-check exception with no way to know even the processor which triggered the ECC error or the address which caused the problem.
That is what people call progress: a nice background image telling you how intelligent you are by having bought this top of the range brand of PC, during the whole Windows boot time.
Topic | Posted By | Date |
---|---|---|
CPU & Memory bit flips | Ganon | 2021/03/03 10:05 AM |
Also "Silent Data Corruption" | Adrian | 2021/03/03 11:42 AM |
Thanks for the reference | Ganon | 2021/03/03 12:47 PM |
Implications for linux page cache | anon | 2021/03/03 12:54 PM |
Implications for linux page cache | Linus Torvalds | 2021/03/03 02:54 PM |
memory errors | blaine | 2021/03/03 03:53 PM |
memory errors | anon2 | 2021/03/03 06:30 PM |
memory errors | dmcq | 2021/03/04 06:16 AM |
memory errors | Etienne Lorrain | 2021/03/04 07:26 AM |
memory errors | dmcq | 2021/03/04 07:40 AM |
memory errors | Etienne Lorrain | 2021/03/04 07:58 AM |
memory errors | dmcq | 2021/03/04 08:12 AM |
memory errors | Carson | 2021/03/05 03:31 AM |
memory errors | Etienne Lorrain | 2021/03/05 07:23 AM |
memory errors | rwessel | 2021/03/05 08:48 AM |
memory errors | dmcq | 2021/03/05 01:01 PM |
memory errors | rwessel | 2021/03/05 01:23 PM |
memory errors | dmcq | 2021/03/05 01:51 PM |
memory errors | Brendan | 2021/03/06 12:38 AM |
memory errors | Carson | 2021/03/06 02:35 AM |
memory errors | Carson | 2021/03/06 07:24 AM |
memory errors | David Hess | 2021/03/04 02:44 PM |
memory errors | rwessel | 2021/03/04 06:14 PM |
memory errors | Linus Torvalds | 2021/03/04 09:21 PM |
memory errors | anon2 | 2021/03/04 10:46 PM |
memory errors | Carson | 2021/03/05 03:43 AM |
memory errors | anon2 | 2021/03/05 08:55 AM |
memory errors | gallier2 | 2021/03/05 03:22 AM |
memory errors | dmcq | 2021/03/05 01:59 PM |
memory errors | David Hess | 2021/03/06 05:27 AM |
memory errors | Carson | 2021/03/06 07:44 AM |
memory errors | Gabriele Svelto | 2021/03/06 11:11 AM |
memory errors | David Hess | 2021/03/06 11:28 AM |
memory errors | Michael S | 2021/03/06 03:45 PM |
memory errors | Doug S | 2021/03/04 11:48 AM |
memory errors | Michael S | 2021/03/04 12:36 PM |
memory errors | Jörn Engel | 2021/03/04 04:32 PM |
memory errors | Linus Torvalds | 2021/03/04 09:47 PM |
memory errors | Etienne Lorrain | 2021/03/05 02:09 AM |
memory errors | Michael S | 2021/03/05 05:06 AM |
memory errors | Linus Torvalds | 2021/03/05 12:59 PM |
memory errors | rwessel | 2021/03/05 01:32 PM |
memory errors | rwessel | 2021/03/05 01:37 PM |
memory errors | zArchJon | 2021/03/06 09:39 PM |
memory errors | Gabriele Svelto | 2021/03/06 01:58 PM |
memory errors | Jörn Engel | 2021/03/05 11:12 AM |
Amiga recoverable RAM disk? | Carson | 2021/03/05 04:03 AM |
Thanks - TIL a cool Amiga feature (nt) (NT) | John | 2021/03/05 01:51 PM |
Another cool Amiga feature, datatypes | Charles | 2021/03/06 01:01 AM |
Another cool Amiga feature, datatypes | Jukka Larja | 2021/03/06 02:23 AM |
Another cool Amiga feature, datatypes | Anon | 2021/03/06 01:40 PM |
Another cool Amiga feature, filesystems | Marcus | 2021/03/07 01:28 AM |
CPU & Memory bit flips | zArchJon | 2021/03/04 07:39 AM |
CPU & Memory bit flips | dmcq | 2021/03/04 07:59 AM |
CPU & Memory bit flips | rwessel | 2021/03/04 01:27 PM |
speak of the devil | Robert Williams | 2021/03/05 08:53 AM |
speak of the devil | dmcq | 2021/03/05 12:26 PM |
speak of the devil | Robert Williams | 2021/03/05 04:15 PM |