Hybrid kernel, not NT

By: nick (anon.delete@this.anon.com), May 14, 2006 4:57 pm
Room: Moderated Discussions
Brendan (btrotter@gmail.com) on 5/13/06 wrote:
---------------------------
>Hi,
>
>nick (anon@anon.com) on 5/13/06 wrote:

>>No, it would suck. You're about 10 years behind: lock
>>contention doesn't hurt scalability any more, cacheline
>>contention does.
>
>Am I?
>
>For my last prototype, each thread has a "thread control block" containing it's
>message queue lock (one queue per thread). While each lock wasn't alone in it's
>own 128 byte area (Intel's recommendation) the data around

That's because it is the size of their largest dcache line.

it was deliberately arranged
>so that only rarely accessed things are in the same area as the lock. The chance
>of 2 or more CPUs trying to read/write to any of the items in the same cache line
>as the lock (including the lock itself) is very small.

Doesn't matter, if they have to both take the lock ->
cacheline bouncing; if one CPU writes a messages which is
read by another -> cacheline bouncing.

>>The microkernel handles scheduler, interrupts, IPC, and
>>queueing messages to servers? And it has a big kernel lock?
>>Then scalability sucks, full stop.
>
>If the OS is designed for UP and SMP with 4 or less CPUs, and if the slowest kernel
>function is relatively fast, then it may be worth sacrificing a tiny amount of performance for simplified locking.

Even at 4 CPUs, it is not a tiny amount of performance!
(unless you're talking about 4 cores sharing cache, which
you're not).

On some workloads (eg. pagecache or scheduler intensive,
because each of those subsystems had a single lock), Linux
2.4 would barely get 2x the throughput of a single CPU on
a 4 CPU system (and 2.4 had hundreds/thousands of locks).
I could probably dig up some links if you're interested.

>
>Given that (IIRC) Linux itself used a "big kernel lock" and nothing else for version
>2.0, and that this lock is still used (although for a lot less than it originally
>was), and that the slowest kernel function within Linux 2.0 would've been much longer,
>I don't think you're in a position to argue too much.

Err, I don't follow your logic at all. Linux 2.0 has a
global lock, scalability sucked. What is my position, and
why isn't it one to argue too much? How much am I allowed
to argue?

I think your position of arguing unproven/unimplemented/
untested ideas is much more tenuous than mine.

>
>It's just one of many compromises - scalability vs. developer time in this case.
>

Exactly. Don't try to pretend it would be scalable though.

>>>For control information you've got a choice - access the control data across nodes,
>>>or access the device driver code and it's local state across nodes. Unless everything
>>>(control data, code and state) is on the same node it can't be "perfect" regardless
>>>of what you do (for all kernel designs).
>>
>>That isn't a choice, that is a restriction!
>
>It's a hardware restriction that can't be avoided, where software developers can
>choose a way to attempt to minimize the "inter-node" access penalties. I will assume
>that for Linux, you end up with penalties when accessing device driver code and
>the device driver's local state (and no penalties when accessing the control data).
>

In Linux, the control data does not need to go across
node because it is basically kept in the stack of the
running process. Instruction text is a different matter,
but AFAIK, that can be replicated on the big NUMA machines
in Linux.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Hybrid (micro)kernelsTzvetan Mikov2006/05/08 03:41 PM
  Hybrid (micro)kernelsS. Rao2006/05/08 05:14 PM
  Hybrid (micro)kernelsBill Todd2006/05/08 05:16 PM
    Hybrid (micro)kernelsTzvetan Mikov2006/05/08 06:21 PM
      Hybrid (micro)kernelsnick2006/05/08 06:50 PM
      Hybrid (micro)kernelsBill Todd2006/05/09 12:26 AM
        There aren't enough words...Rob Thorpe2006/05/09 01:39 AM
          There aren't enough words...Tzvetan Mikov2006/05/09 02:10 PM
            There aren't enough words...Rob Thorpe2006/05/14 11:25 PM
        Hybrid (micro)kernelsTzvetan Mikov2006/05/09 10:17 AM
          Hybrid (micro)kernelsBill Todd2006/05/09 03:05 PM
  Hybrid (micro)kernelsrwessel2006/05/08 10:23 PM
    Hybrid kernel, not NTRichard Urich2006/05/09 05:03 AM
      Hybrid kernel, not NT_Arthur2006/05/09 06:06 AM
        Hybrid kernel, not NTRob Thorpe2006/05/09 06:40 AM
          Hybrid kernel, not NT_Arthur2006/05/09 07:30 AM
            Hybrid kernel, not NTRob Thorpe2006/05/09 08:07 AM
              Hybrid kernel, not NT_Arthur2006/05/09 08:36 AM
                Linux vs MacOSX peformance, debunked_Arthur2006/05/18 06:30 AM
                  Linux vs MacOSX peformance, debunkedRob Thorpe2006/05/18 07:19 AM
                    Linux vs MacOSX peformance, debunkedAnonymous2006/05/18 11:31 AM
        Hybrid kernel, not NTLinus Torvalds2006/05/09 07:16 AM
          Hybrid kernel, not NTAndi Kleen2006/05/09 01:32 PM
            Hybrid kernel, not NTmyself2006/05/09 02:24 PM
              Hybrid kernel, not NTmyself2006/05/09 02:41 PM
              Hybrid kernel, not NTBrendan2006/05/09 04:26 PM
                Hybrid kernel, not NTLinus Torvalds2006/05/09 07:06 PM
                  Hybrid kernel, not NTBrendan2006/05/13 12:35 AM
                    Hybrid kernel, not NTnick2006/05/13 03:40 AM
                      Hybrid kernel, not NTBrendan2006/05/13 08:48 AM
                        Hybrid kernel, not NTnick2006/05/13 06:41 PM
                          Hybrid kernel, not NTBrendan2006/05/13 08:51 PM
                            Hybrid kernel, not NTnick2006/05/14 04:57 PM
                              Hybrid kernel, not NTBrendan2006/05/14 09:40 PM
                                Hybrid kernel, not NTnick2006/05/14 10:46 PM
                                  Hybrid kernel, not NTBrendan2006/05/15 03:00 AM
                                    Hybrid kernel, not NTrwessel2006/05/15 06:21 AM
                                      Hybrid kernel, not NTBrendan2006/05/15 07:55 AM
                                        Hybrid kernel, not NTLinus Torvalds2006/05/15 08:49 AM
                                          Hybrid kernel, not NTnick2006/05/15 03:41 PM
                                          Hybrid kernel, not NTtony roth2008/01/31 01:20 PM
                                    Hybrid kernel, not NTnick2006/05/15 05:33 PM
                                      Hybrid kernel, not NTBrendan2006/05/16 12:39 AM
                                        Hybrid kernel, not NTnick2006/05/16 01:53 AM
                                          Hybrid kernel, not NTBrendan2006/05/16 04:37 AM
                  Hybrid kernel, not NTAnonymous2008/05/01 09:31 PM
                    Following the structure of the treeMichael S2008/05/02 03:19 AM
                      Following the structure of the treeDean Kent2008/05/02 04:31 AM
                        Following the structure of the treeMichael S2008/05/02 05:02 AM
                        Following the structure of the treeDavid W. Hess2008/05/02 05:48 AM
                          Following the structure of the treeDean Kent2008/05/02 08:14 AM
                            Following the structure of the treeDavid W. Hess2008/05/02 09:05 AM
                              LOL!Dean Kent2008/05/02 09:33 AM
                              Following the structure of the treeanonymous2008/05/02 02:04 PM
                                Following the structure of the treeDean Kent2008/05/02 06:52 PM
                                Following the structure of the treeFoo_2008/05/03 01:01 AM
                                  Following the structure of the treeDavid W. Hess2008/05/03 05:54 AM
                                    Following the structure of the treeDean Kent2008/05/03 09:06 AM
                                      Following the structure of the treeFoo_2008/05/04 12:06 AM
                                        Following the structure of the treeMichael S2008/05/04 12:22 AM
            Hybrid kernel, not NTLinus Torvalds2006/05/09 04:19 PM
              Microkernel Vs Monolithic KernelKernel_Protector2006/05/09 08:41 PM
                Microkernel Vs Monolithic KernelDavid Kanter2006/05/09 09:30 PM
                  Sigh, Stand back, its slashdotting time. (NT)Anonymous2006/05/09 09:44 PM
                  Microkernel Vs Monolithic Kernelblah2006/05/12 07:58 PM
                  Microkernel Vs Monolithic KernelRob Thorpe2006/05/15 12:41 AM
          Hybrid kernel, not NTAnalGuy2006/05/16 02:10 AM
            Theory versus practiceDavid Kanter2006/05/16 11:55 AM
              Distributed algorithmsRob Thorpe2006/05/16 11:53 PM
              Theory versus practiceHoward Chu2006/05/17 01:54 AM
                Theory versus practiceJS2006/05/17 03:29 AM
          Play online poker, blackjack !!! Gamezonex2007/08/16 12:49 PM
  Hybrid (micro)kernelsphilt2006/05/14 08:15 PM
    Hybrid (micro)kernelsLinus Torvalds2006/05/15 07:20 AM
      Hybrid (micro)kernelsLinus Torvalds2006/05/15 10:56 AM
        Hybrid (micro)kernelsRob Thorpe2006/05/16 12:22 AM
          Hybrid (micro)kernelsrwessel2006/05/16 10:23 AM
            Hybrid (micro)kernelsRob Thorpe2006/05/16 11:43 PM
              Hybrid (micro)kernelsrwessel2006/05/17 12:33 AM
                Hybrid (micro)kernelsRob Thorpe2006/05/19 06:51 AM
                  Hybrid (micro)kernelsrwessel2006/05/19 11:27 AM
      Hybrid (micro)kernelstechIperson2006/05/15 12:25 PM
      Hybrid (micro)kernelsmas2006/05/15 04:17 PM
        Hybrid (micro)kernelsLinus Torvalds2006/05/15 04:39 PM
          Hybrid (micro)kernelsColonel Kernel2006/05/15 08:17 PM
            Hybrid (micro)kernelsWink Saville2006/05/15 09:31 PM
              Hybrid (micro)kernelsLinus Torvalds2006/05/16 09:08 AM
                Hybrid (micro)kernelsWink Saville2006/05/16 08:55 PM
          Hybrid (micro)kernelsrwessel2006/05/16 10:31 AM
            Hybrid (micro)kernelsLinus Torvalds2006/05/16 11:00 AM
        Hybrid (micro)kernelsBrendan2006/05/16 12:36 AM
        Hybrid (micro)kernelsPaul Elliott2006/09/03 07:44 AM
          Hybrid (micro)kernelsRob Thorpe2006/09/04 08:25 AM
      Hybrid (micro)kernelsphilt2006/05/15 11:55 PM
        Hybrid (micro)kernelspgerassi2007/08/16 06:41 PM
  Another questionable entry on Wikipedia?Chung Leong2006/05/18 09:33 AM
  Hybrid (micro)kernelsisrael2006/05/20 03:25 AM
    Hybrid (micro)kernelsRob Thorpe2006/05/22 07:35 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?