Travis Downs (travis.downs.delete@this.gmail.com) on January 15, 2020 10:35 am wrote:
>
> As far as I can tell, if someone were to actually use the mcs
> locks in mcs_spinlock.h, you'd declare your lock object like:
>
>
> struct mcs_spinlock *lock;
>
>
> I.e., a pointer, and not like:
>
>
> struct mcs_spinlock lock;
>

Yes.

Or rather, you'd probably hide that pointer inside the spinlock abstraction, and do something like

typedef struct mcs_spinlock *spinlock_t;
#define SPIN_UNLOCKED NULL

and people would just use "spinlock_t x = SPIN_UNLOCKED" to use it.

That said, what we used to have in the kernel was just a separate "mcs_spinock" (in fact, originally merged as "mspin_node"), and the initial implementation had a function like this:

void mspin_lock(struct mspin_node **lock, struct mspin_node *node)

that would take the lock by inserting that "mspin_node" into the lock. And that original use case actually just had

void *spin_mlock; /* Spinner MCS lock */

as the lock itself, and then the actual user had

#define MLOCK(mutex) ((struct mspin_node **)&((mutex)->spin_mlock))
...
struct mspin_node node;
mspin_lock(MLOCK(lock), &node);

and if that looks messy and crazy, then let me just explain the context: the Linux kernel use of this MCS lock wasn't for spinlocks, it was used purely for the internal implementation of the locking inside the kernel "mutex" sleeping lock.

Spinlocks are actually somewhat hard to get contention on in the kernel, because spinlocks are used to protect tight small non-blocking things. But the mutex case saw much more contention, because mutexes are used for bigger things - so thus the (initial) work on making those mutexes use data structures that scale better. In particular, avoid the nasty thundering herd problem on the cacheline as the lock is released.

So that mcs lock wasn't really ever exposed to anybody else. It was an internal detail in a bigger locking scheme.

But you can see that initial implementation back in between v3.14 and v3.15, where it was used as a more traditional pure mcs lock and have that "just a pointer" implementation (but by the time v3.15 was done, it was already slightly more abstracted than the above messy thing).

The current use is kind of an evolution of that original case, and the mutex usage case has evolved too (it's now doing

struct optimistic_spin_queue osq; /* Spinner MCS lock */

instead of the original - and more traditional - "pointer to mcs lock".

So the kernel doesn't really have much sign of the original usage any more, because the mutex case evolved away from the pointer, and the queued spinlock case could never use the direct pointer approach in the first place and was a modified version of it. But you can see something that looks more like a traditional MCS lock during that early development.

Linus