Tzvetan Mikov (tmikov@gmail.com) on 5/9/06 wrote:
---------------------------
>Bill Todd (billtodd@metrocast.net) on 5/9/06 wrote:
>---------------------------
>>>Although you are factually correct, I disagree with your interpretation. The resemblance
>>>you are refering to is not in important aspects of the design - it is like saying
>>>that a car resembles a train because they both have windows.
>>
>>Hmmm - sounds a bit like blustering to me, and from someone who didn't understand
>>the difference between IRPs and messages that may not be surprising.
>
>Oh, come on, nice way to elevate the discussion :-)

(Well, that if nothing else does deserve further comment, so here goes:)

It is possible that I 'elevated' it unnecessarily: that's my natural inclination here, since on average I find those RWT contributors most likely to jump in and criticize something to be arrogant and often less than well-informed and/or less than objective. If that's not true in your case, I apologize.

...

Many things which I thought were unique NT
>design decisions turned out to be just cosmetics and implementation details. The
>OS personalities are one of them, and so is LPC.

That is, of course, purely a matter of opinion. In principle, the division between the kernel and the OS personalities could have occupied any point along a spectrum from extremely minimal micro-kernel on the one hand to even more minimal non-kernel support than currently exists. In practice, Microsoft seems to have elected to place most common features in the kernel and most disparate features in the applicable personality (which could again occupy any point on that spectrum according to the degree and depth of feature disparity among the personalities - but it turns otu that they have a lot more in common than not).

I tend to consider the micro-kernel approach itself to be largely 'cosmetics and implementation details' rather than particularly significant (save for the cases in which that approach severely compromises performance) - because so much of what they evict to protected user-mode services is sufficiently critical to system operation that the system can't reasonably continue (at least for very long) without it anyway (again, random ancillary devices and their drivers possibly excepted). In plain words, I've tended to view micro-kernels as something of an academic fad which never caught on very well because the problems it solved were less significant than the problems it created (considering that information-hiding and modularity can be achieved in monolithic kernels by suitably object-oriented approaches that don't enforce address-space separation and its resulting overheads - and indeed the NT kernel/executive is mostly implemented in this manner).

Save, perhaps, for systems dating back to the 16-bit era, where separating kernel services into protected user-mode processes gave the OS a great deal more aggregate virtual address space to play with. DEC RSX-11 systems from the 1970s used such a protected service to handle much of their file system code (but still streamlined basic read and write operations directly through the kernel for performance), and VMS inherited this mechanism until the advent of clustering (which found it to be something of an obstacle and replaced it with a distributed, per-process mechanism).

But even modern OSs based on Mach (such as DEC's Tru64 Unix, IIRC and the wiki mentions OS/X as well) moved a lot of code (the file system, for example) back into the kernel for performance reasons. Ironically, NT might have benefited from placing its file system cache in a separate process, given that it's direct-mapped into kernel virtual memory and significantly constrained as a consequence even in a 32-bit environment (a direct benefit of 64-bit Windows even to people who will never run a 64-bit application program) - but access to cached data is, of course, performance-sensitive, so that was not an option.

A time may be approaching when interconnect speeds allow multiple clustered systems to cooperate effectively in executing the (separate and modular) service processes of a single classic micro-kernel, though I don't think that even Infiniband comes close to that yet for services requiring significant performance (e.g., once again, the file system: while some current interconnects are likely fast enough for any operation requiring an actual disk access, the impact on access to cache-resident data would be significant). Even then the question would remain, is that ability really all that worthwhile, or is it more effective to coordinate each individual node as a separate OS instance (especially given a desire to isolate the impacts of any individual node's failure)?

>
>You did not address my analogy with Wine. Does running an application under Wine
>suddenly turn Linux into a hybrid kernel ?

No, of course. The difference between Wine/X and the NT implementation is that both Wine and X are effectively third-party features added to the OS in ways intended to minimize the impact on the existing kernel, whereas the user-mode NT services were designed into the OS as integral components from Day 1.

...

>>>In my understanding, the personalities (Win32,OS/2,POSIX) are mostly user-mode
>>>libraries mapping the calls to the kernel API possibly with the support of some
>>>daemons to maintain state.
>>
>>Your understanding is incorrect. Support for the original graphics interface,
>>for example, was completely contained in the Win32 user-mode environment subsystem
>>(which was even called by the POSIX and OS/2 subsystems to perform any grahpics
>>activity on their behalf, adding yet another level of user-mode indirection). Only
>>in NT 4 and later did graphics support move largely into the kernel and (mostly)
>>get called directly there via linked libraries rather than via a subsystem process.
>
>Please, show me how my understanding is incorrect.

You'll note that I cited GDI as an *example* above - and, indeed, it is (or at least in early NT was) probably the largest one. But aspects of the security and network mechanisms also ran as protected user-mode subsystems in the first NT release (whether they still do I don't know), as well as the balance of the 'personality' subsystems themselves (Win32 handled thread creation/destruction and parts of the 16-bit virtual DOS machine support, for example).

All these mechanisms were implemented by message-passing between the application and the applicable user-mode subsystem (which in some cases invoked kernel facilities to help with the operation), just as in a classic micro-kernel.

>So, you are basically saying "GDI is a kernel component, which was moved to user
>space, which is what a micro-kernel would do", while I claim that GDI is an user-mode
>component which was artifically injected into the kernel.

Are you seriously suggesting that a classic monolithic kernel developed from the ground up (unlike the case with Unix/Linux) to support a graphical enviroment would *not* include GDI support? If so, that would be just about the *only* system-wide component that did not reside in such a monolithic kernel.

I realize it is a matter
>of interpretation and I think I understand your point - the original GDI and OS
>personalities implementation did resemble techniques used in a microkernel.

Exactly - and having revisited the wiki I'll grant you that its first sentence, while literally correct, may *suggest* a closer connection between classic micro-kernels and systems like NT than I consider appropriate (given that on the spectrum from classic micro-kernel to classic monolithic kernel NT falls considerably closer to the latter than to the former).

But I'm a very literal kind of guy, and I read it literally.

Also
>GDI did have to go to some deliberate lengths to enable running the display driver
>in user mode - like separating the video miniport driver, etc.

Inside Win2K states that each process thread had a matching thread in the Win32 user-mode subsystem waiting to handle its graphics requests - yuck!

...

>Obviously you hold the more conservative view that a monolithic literally kernel
>provides user API syscalls.

Not at all: I consider the kind of separation which system DLLs provide between the application API and the 'real' kernel interface to be quite desirable but in no way affecting whether the kernel is considered monolithic or not.

Once the servicing of *any* application OS request (and again I'll point out that X is *not* an OS service per se but only a - very significant - add-on, unlike the Windows GDI) is vectored to a protected, separate, non-kernel service process, however, I don't consider the kernel to be strictly monolithic any more.

...

Here is another Linux example: does FUSE (http://fuse.sourceforge.net/)
>turn Linux into a hybrid kernel ? It doesn't, because Linux wasn't designed with
>the explicit goal of executing kernel components into separate address spaces and surviving crashes in them.

Exactly. Whereas NT *was* explicitly designed with the goal of separating its POSIX and OS/2 environment subsystems into separate address spaces and surviving crashes in them (again, I'm not sure whether the same was true of the security and network user-mode services, and as implemented it was clearly not true of the Win32 subsystem - though it could have been made so).

- bill