rwessel (robertwessel.delete@this.yahoo.com) on May 31, 2013 10:59 am wrote:
> Sebastian Soeiro (sebastian_2896.delete@this.hotmail.com) on May 31, 2013 8:01 am wrote:
> > Klimax (danklima.delete@this.gmail.com) on May 31, 2013 7:15 am wrote:
> > > Simple example: Due to bug (like unchecked end of array) it will write past allocated range. (Or just
> > > randomly all over memory) And in process it will destroy whatever it can including kernel structure.
> > > For crazy reading about crazy programs I recommend http://ptgmedia.pearsoncmg.com/images/9780321440303/samplechapter/Chen_bonus_ch02.pdf
> > > Bonus chapter for book "The Old New Thing" by Raymond Chen.
> > >
> > > As for contiguous memory - arrays. (Enables simple and fast way to access
> > > any element while knowing only starting address and size of structure)
> >
> > Thank you for your reply;
> >
> > Now I understand how virtual addresses can help arrays through contiguous addresses. I
> > suppose where the physical addresses is the DTLB's problem; but the ALU math is probably
> > vastly simplified by having contiguous virtual memory to calculate like (array+2) instead
> > of (YMM15*14/2^3), and there are probably other benefits that I'm not even seeing.
> >
> > However, one question. Say an array has an error and just continues writing more and more and more
> > elements to the array, filling memory. How would having contiguous virtual addresses prevent this
> > from happening? I can see how it'd make it much easier to "clear" the memory of these errored entries,
> > but is there a mechanism in place to detect if this error array is happening to stop it?
>
>
> By itself, virtual address translation won't stop it, but eventually the bad application
> will attempt to write to an address that is not valid in that process's (virtual) address
> space, and the CPU will generate an exception, which the OS will catch, and then (usually)
> terminate the offending process. When the offending process is terminated, all of its resources,
> including memory allocations, are released by the OS, which tracks such things.
>
> Note that on most processors virtual addresses and physical address are very similar, occupying a flat
> series of consecutive addresses. For example, on most 32 bit processors (supporting VM), addresses, both
> virtual and physical, range from 0 to 4,294,967,295 (4GiB). Those address spaces are commonly divided
> into 1,048,576 consecutive pages of 4096 bytes (4KiB). Many processors can turn off address translation,
> and the any running code would then just access a physical address. Address translations basically sets
> up a table to translate those million virtual page numbers for a process to physical page numbers. With
> address translation on, a process wanting (say) 12KB of memory might get that assigned to virtual pages
> 100, 101 and 102, and thus would access those as 12K consecutive addresses starting at (virtual) address
> 409,600. Those three virtual pages might be mapped to physical pages 1000, 9876, and 123467, at the OS's
> whim. Absent a specific request to share them, those three physical pages would *not* be mapped into any
> other process's virtual address space, and so would only be accessible to the one process. And while actual
> virtual memory is less relevant these days, the OS might need more physical pages for one process and it
> might page one of those virtual pages to disk, freeing up the associated physical page. When the process
> tries to access that paged-out page, the OS will catch the associated exception, read the needed physical
> page back into memory, fix up the translation, and let the application continue.
>
> In some situations, guard pages are generated adjacent to certain areas of (virtual) memory,
> which allows running off the end of arrays to be caught fairly quickly. It's a limited solution
> at best, since pages are too large to waste on small objects in most cases.
>
> But having a separate virtual address space prevents the errant process for trashing the
> memory assigned to *other* processes. So while the buggy word processor dies a horrible
> death, taking your document with it, the spreadsheet you were editing at the same time continues
> running as if nothing had happened. Of course some processes cooperate and communicate with
> each other, and will often react badly to one of the partners suddenly dying.
>

Ah, that all makes sense! So the OS can detect when things go rogue and kills off buggy processes! That's quite logical!

Everything you said makes sense, and I definitely am learning quite a bit from it. But I have one outstanding question;

You said that the OS usually uses 4KB pages, right? Well, how come Haswell has support for 4MB and 1GB pages? What benefit do these extra page sizes grant if the OS can only assign 4KB pages?

Thank you for your very informative reply!