Linus Torvalds (torvalds.delete@this.linux-foundation.org) on January 4, 2020 1:24 pm wrote:
>
> Now, if you didn't get the lock, you might decide that "hey, lots of locks are held only
> for very short times, and I'll try to wait for a very bounded time".

Btw, I do not really suggest you do this unless you have serious numbers that show that it helps on a real-world load (and not just some "lock/unlock" benchmark).

Even a very limited "let's busy-loop" can have serious disadvantages. Doing that "CPU yield" thing ("rep nop" on x86, "yield" instruction on arm64) can trigger the hypervisor to do things in virtualized environments, as can things like doing a cycle counter read to limit the time you wait etc. It might work fine on your machine, but it can cause unexpected behavior elsewhere.

And if you only do a hard busy loop reading the lock value, you might be wasting time, energy and cause unnecessary cache contention. If the loop is limited to a small value, it might have very dubious value, and if you make it longer, the downsides can grow a lot. And the value tuning might be as much about your load as it is about the particular machine you are running on, etc, so you might again be tuning only for your development machine, not any real life situation.

So to a first approximation, start off with that fast-case "trylock/tryunlock" model where you handle the case of absolutely no contention specially yourself. And then fall back to a sleeping real model like futexes immediately after that unless you really know a lot about your workload.

In fact, feel free to skip even that fast case. Yes, it often makes a lot of sense to do a special and simple fast case: in a lot of real-world locking situations you'll hardly ever see any contention at all, and you might be writing a library that is often used in a completely single-threaded manner.

So handling the trivial non-contended locking case with a special fast-case is quite often a very good idea.

But "quite often" doesn't mean "always". Locking might simply either be so rare that it's pointless to have a special fast case in the first place if the fallback isn't actively horrible. Or the lock might be designed to be a resource limiter and the contention case is the expected case because you have a thousand threads that might want to run, but you use the lock as a way to limit them to a much smaller number (ie the GNU make "jobserver" case).

So my suggestion is to basically always start off with just pthread_mutex_lock() or other generic non-spinning locking (std::mutex, whatever you have in your environment).

And if that turns out to have performance problems and you know your behavior, go out and google for somebody that already solved it and wrote a paper about it, and has numbers for a lot of different scenarios. I see that El Presidente already pointed to a report by Waiman Long elsewhere in this thread. Waiman knows what he's doing. He's one of the Linux kernel locking people.

And only after that, and only if you can't for the life of you find something that works for you, and you know that you have very very particular locking behavior (that may be the reason you can't find something that works for you), do you start going down the "let me try to write my own", and start off with the special fast case optimized for your known load, and fall back to something like futexes. But realize that it's a big job, and it will be painful.

And not in a million years will any of those choices hopefully ever include "sched_yield()", although sometimes I might worry about standard libraries (particularly if you're targeting a new architecture or new language environment, so that the standard locking implementation may basically be some stub function - it generally will take many years for this kind of infrastructure to mature).

Linus