By: Linus Torvalds (torvalds.delete@this.linux-foundation.org), March 16, 2021 7:25 pm
Room: Moderated Discussions
anon2 (anon.delete@this.anon.com) on March 16, 2021 5:53 pm wrote:
> I see. So at least it tries to get away from the RPC proxying as much as possible.

This whole thread has been inane.

Absolutely nobody sane disputes that the IPC overhead is a big and very real deal. Even the people who seem to be arguing for microkernels seem to be admitting it.

It's something to generally be avoided, but sometimes you can't (or you don't care about the downsides, and have a load where it works for you). So if your sandboxing capabilities can't do it any other way, you fall back to some kind of IPC model.

Which just brings us back to the original issue: designing your whole kernel around something that people know is bad is a bad idea.

That sounds pretty simple, doesn't it?

And it really is that simple.

It's not that some RPC-=based model would always a bad idea. Sometimes it's very much called for. Sometimes it's the only way to do it, and sometimes it's a really convenient model to take. But IPC isn't simple - particularly not some reasonably high-performance one - it adds a lot of complexity, and it often adds a lot of overhead unless your hardware is explicitly designed for it.

Sometimes the overhead doesn't much matter, and it can be really nice to (for example) develop random USB communication protocol drivers in user space, and overhead be damned. Or do filesystem prototyping, or things like that. Running gdb on the driver and single-stepping through it is all kinds of powerful, and logging to regular files is nice and easy.

Or, as Gabriele pointed out, doing it in a browser when there aren't any better alternatives, and you just need the separation.

So IPC isn't bad. It's most definitely "one tool among many" in the toolchest, and sometimes it's the right tool.

But the key part of that previous sentence is very much that "sometimes".

Designing your system as if RPC is the be-all and end-all of all you do, and what the whole system design is all about - that's just incredibly stupid. Give it up, people. It was always a bad idea, and when you still try to make excuses for it decades later by trying to make some high-level arguments for "modularity" and "security" or "stability" that turned out to never actually be true, and you still - after all those decades - try to brush all the practical problems under the rug by just ignoring them, you are not intellectually very honest.

Because yes, the practical problems are huge and in many cases basically insurmountable. Your memory manager really does want to interact with your filesystems pretty much directly, and they want to share data structures so that they can do simple things like "I still use this page" (ie memory management uses it for mapping, filesystem uses it for reading or whatever) without sending a effing message to each other, for chrissake.

Does latency for some random USB driver matter? No. You can often do simple device drivers using RPC mechanisms. In fact, sometimes the whole physical device itself is built around RPC, and has its own CPU that runs a completely different instruction set, and that you basically use RPC to communicate with it (ie think all those wireless modems that have their own microcontroller CPU entirely just to do all the basics, and you then talk to that over some - often very specialized - RPC protocol using doorbells and a special shared memory area).

But latency can be a big deal in other situations - think a network driver or a block driver where we're talking below microsecond latencies on modern hardware. You'll find that high-performance filesystems that do disk IO basically want to have the option to poll for the result (similar issues go for networking packet reception). The filesystem wants to be very intimately involved with setting up the command queues for the disk driver, and it needs to know exactly what the disk controller IO capabilities and current state is (outstanding requests etc - you might want to do different things if the IO is being saturated vs not, for example).

So now your scheduler is involved in this all too - because that whole decision of "do I poll or not" can then in turn depend on things like "Oh, there are other things pending, so let's not poll after all". So now you have your memory management, your filesystem, your drivers, your network stack, and your scheduler, all basically wanting to communicate state to each other.

Trying to separate the filesystem and the disk driver and the memory management and the scheduler into different address spaces without sharing data structures is insane.

All these things have tons of subtle interconnects that are not about "I'm sending a request to you". They are very much about co-operating with each other, and you want to have a shared request queue between the different pieces, you want to have visibility into (some of) the scheduler state, you want to have all these things where you just look at what is going on (and perhaps use an atomic op to change said state).

The filesystem code might also easily want to know whether the initiating user process might have a signal pending - maybe it can still cancel the whole thing if it can tell that the originating process is being killed.

So the whole belief that "user process sends a message to a filesystem process, and waits for the reply" is simply not true. Or rather, it's only true at such a high level that if you only see that part, you've lost sight of the underlying reality.

It was always insane to think that these pieces were unrelated and should be separate things, and only communicate with each other over some very limited and idealized channel.

Why is this even a discussion any more? Microkernels failed. Give them up. You want a monolithic kernel. End of story.

Yes, that monolithic kernel can then do RPC for the cases where that then makes sense.

Because again: sometimes it makes sense to use RPC. Sometimes you have to do it. If you're doing a network filesystem, you pretty much by definition aren't doing everything inside the same address space, and you'll want to have a nice and well-defined communication protocol between nodes.

But the fact that you sometimes want to do it, in absolutely no way argues that you should always do it.

Is it really so hard to understand the difference between sometimes and always?

But that really was the fundamental argument for microkernels. Some of them were expressly designed with the notion that "cluster computers are the future", and some of the arguments go back to the whole "SMP and cache coherence is hard, and thus not valid" mode of thinking.

If you really live in that kind of world, where SMP and cache coherence failed and will never succeed, and machines instead have thousands of nodes that all communicate fundamentally using message passing, then microkernels might make sense.

I admit it, the Transputer was actually all kinds of cool. I never got to program one, but it always intrigued me.

But I really think that outside of the mythical world where the Transputer took over, microkernels are just crazy talk.

But this thread somehow devolved into then discussing whether IPC is crazy talk. No, obviously not. But IPC being a valid and sane thing to do does not equate to microkernels being a valid and sane thing to do. See?

< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
x86 - why unite when you can fragment?anonymou52021/03/12 06:16 PM
  x86 - why unite when you can fragment?Linus Torvalds2021/03/13 01:18 PM
    x86 - why unite when you can fragment?Jon Masters2021/03/13 07:25 PM
      x86 - why unite when you can fragment?Jon Masters2021/03/13 07:44 PM
        x86 - why unite when you can fragment?Yuhong Bao2021/03/13 08:49 PM
        x86 - why unite when you can fragment?tt2021/03/20 09:30 AM
    x86 - why unite when you can fragment?Andrey2021/03/14 04:15 PM
      x86 - why unite when you can fragment?Linus Torvalds2021/03/14 04:58 PM
        x86 - why unite when you can fragment?anonymou52021/03/14 05:31 PM
          x86 - why unite when you can fragment?anon22021/03/14 08:07 PM
        Microkernel?Anon2021/03/14 11:49 PM
          Microkernel?none2021/03/15 12:37 AM
            Microkernel?Anon2021/03/15 01:56 AM
          Microkernel?anon22021/03/15 01:58 AM
            Microkernel?Simon Farnsworth2021/03/15 03:12 AM
              Microkernel?anon22021/03/15 04:53 AM
                Microkernel?Simon Farnsworth2021/03/15 06:56 AM
                  Microkernel?iz2021/03/15 08:10 AM
                    Microkernel?Anon2021/03/15 09:05 AM
                      Microkernel?iz2021/03/16 01:25 AM
                        Microkernel?Andrey2021/03/16 02:54 AM
                          Microkernel?iz2021/03/16 08:36 AM
                            Microkernel?Andrey2021/03/16 10:06 AM
                              Microkernel?anonymou52021/03/16 11:44 AM
                              Microkernel?iz2021/03/21 02:58 AM
                                Microkernel?Andrey2021/03/21 09:34 AM
                  Microkernel?anon22021/03/15 08:31 AM
                    Microkernel?Simon Farnsworth2021/03/16 04:42 AM
            Microkernel?Gabriele Svelto2021/03/15 03:21 AM
              Microkernel?anon22021/03/15 04:56 AM
                Microkernel?Gabriele Svelto2021/03/15 10:41 AM
                  Microkernel?anon22021/03/15 08:00 PM
                    Microkernel?Gabriele Svelto2021/03/16 07:23 AM
                      Microkernel?anon22021/03/16 05:13 PM
                        Microkernel?anon22021/03/16 05:16 PM
                    Microkernel?Gian-Carlo Pascutto2021/03/16 01:40 PM
                      Microkernel?anon22021/03/16 05:53 PM
                        Microkernel?Linus Torvalds2021/03/16 07:25 PM
                          Microkernel?Doug S2021/03/17 09:30 AM
                            Microkernel?Linus Torvalds2021/03/17 10:30 AM
                              Microkernel?Brendan2021/03/17 10:56 PM
                                Microkernel?Michael S2021/03/18 03:47 AM
                                  Microkernel?Brendan2021/03/18 09:07 AM
                              Microkernel?Jose2021/03/18 09:35 AM
                            Microkernel?zArchJon2021/03/18 05:42 PM
                          TransputerRichardC2021/03/17 09:47 AM
                          Microkernel?dmcq2021/03/17 11:15 AM
                            Microkernel?Linus Torvalds2021/03/17 11:59 AM
                              Microkernel?dmcq2021/03/17 12:38 PM
                              Microkernel?Adrian2021/03/17 01:00 PM
                              Microkernel?Ana R. Riano2021/03/18 04:33 AM
                              Microkernel?2021/04/30 04:52 PM
                          Microkernel?NvaxPlus2021/03/17 11:48 AM
                            Microkernel?Michael S2021/03/18 03:32 AM
                              Microkernel?Adrian2021/03/18 04:12 AM
                                Microkernel?dmcq2021/03/18 06:30 AM
                                  Microkernel?dmcq2021/03/18 06:55 AM
                                  Microkernel?Adrian2021/03/18 08:35 AM
                                    Microkernel?---2021/03/18 09:49 AM
                                    Microkernel?dmcq2021/03/18 10:59 AM
                                      Microkernel?dmcq2021/03/18 04:09 PM
                              Microkernel?---2021/03/18 09:27 AM
                          Microkernel?Kalle A. Sandström2021/03/20 06:34 AM
                            Microkernel?---2021/03/20 08:35 AM
                            Microkernel?anon22021/03/21 05:29 PM
            Microkernel?dmcq2021/03/15 04:06 AM
              Microkernel?anon22021/03/15 04:59 AM
                Microkernel?dmcq2021/03/15 11:51 AM
                  Microkernel?anon22021/03/15 08:31 PM
                    Microkernel?dmcq2021/03/16 09:17 AM
                      Microkernel?Jukka Larja2021/03/16 11:22 AM
                        Microkernel?dmcq2021/03/16 04:06 PM
                          Microkernel?Jukka Larja2021/03/17 03:42 AM
                            Microkernel?dmcq2021/03/17 07:00 AM
                      Microkernel?anon22021/03/16 05:26 PM
                    Microkernel?---2021/03/16 10:07 AM
            Microkernel?-.-2021/03/15 08:15 PM
              Microkernel?anon22021/03/15 09:18 PM
                Microkernel?Foo_2021/03/16 03:37 AM
                  Read the thread (NT)anon22021/03/16 05:27 PM
                    Already did (NT)Foo_2021/03/17 02:55 AM
                      Already didanon22021/03/17 03:46 AM
                        Already didEtienne Lorrain2021/03/18 02:31 AM
                Microkernel?-.-2021/03/17 05:04 AM
                  Microkernel?Gabriele Svelto2021/03/17 08:53 AM
                    Microkernel?-.-2021/03/17 02:43 PM
              Microkernel?dmcq2021/03/16 08:40 AM
        x86 - why unite when you can fragment?Konrad Schwarz2021/03/17 10:19 AM
    x86 - why unite when you can fragment?anonon2021/03/15 07:37 AM
Reply to this Topic
Body: No Text
How do you spell avocado?