Linus Torvalds (torvalds.delete@this.linux-foundation.org) on January 5, 2020 3:30 pm wrote:
> Jörn Engel (joern.delete@this.purestorage.com) on January 5, 2020 1:45 pm wrote:
> > One thing I noticed while watching Weiman's presentation is an off-hand remark that FUTEX_LOCK_PI
> > has worse performance than FUTEX_WAIT. Do you happen to know why that is the case?
> > In particular, is that a fundamental limitation or just a bug we can fix?
>
> I think it's just that FUTEX_WAIT is a fundamentally simpler operation, and
> when it works for your load, there's just much less setup and complexity.
>
> FUTEX_LOCK_PI is rather fancier, and is a rather more complete locking
> model, but that also implies a fair amount more housekeeping.

At least in the common case where you don't have realtime threads, FUTEX_LOCK_PI should be pretty cheap. A quick check to decide that there is no priority to gift, plus a check on unlock whether priority was gifted. Since both checks are always negative, the overhead should only be a handful of cycles.

When there are realtime threads and different priorities there obviously is more work to be done. But skipping that work leads to priority inversion and rather nasty latencies, if not deadlocks.

> My personal opinion is that it makes no sense at all to have cleanup on thread death - if it held a lock
> while dying, you can't rely on the data structures that that lock protected anyway, so trying to "clean
> up" is just silly. But not everybody agrees with me, and some people want that EOWNERDEAD kind of error
> handling capability, which pretty much requires a large extra set of complexity (and a performance hit).

I would mostly agree. Printing a backtrace has some value, but beyond that the process should just die and restart as fast as possible.

> This is just one example why a custom locking library may be a better choice for you than
> some generic locking model: there are a lot of corner case issues wrt lock handling that
> may or may not be things you care about, but that can have a big impact on performance.

You may enjoy this one:
static inline int trylock_unsafe(struct lock *l)
{
int ret = l->lock;
if (!l->lock) {
l->lock = 1;
}
return ret;
}

It doesn't use any atomics, making it much faster. It is also totally unsafe for most users. The one exception is a thread-cache. Without a lock the same thread can recurse into the thread cache in a signal handler, corrupting data structures. But if we only need protection from code in the same thread, we can get by without atomics.