Montaray Jack (none.delete@this.none.org) on January 3, 2020 12:14 pm wrote:
> Beastian (no.email.delete@this.aol.com) on January 3, 2020 11:46 am wrote:
> > I'm usually on the other side of these primitives when I write code as a consumer of them,
> > but it's very interesting to read about the nuances related to their implementations:
> >
> > https://probablydance.com/2019/12/30/measuring-mutexes-spinlocks-and-how-bad-the-linux-scheduler-really-is/
> >
> >

The problem was that there was a thread that spent several milliseconds trying to
> > acquire a spinlock at a time when no other thread was holding the spinlock.

> >
> >
>
> Paul Davis (of JACK audio) has a bunch of useful comments at the bottom of the blog

oiaohm comments are even more insightful.

One of the most damaging effects of running a multi-threaded application on a multi-processor machine is re-peated transfer of memory cache lines from one proces-sor to the other (a.k.a. cache line ping-pong).

Using spinlock provides no protection from this under Linux. Futex or Mutex does under Linux. In theory you could lock your spinlock using process to a single core and avoid this but you have just lost the advantage of the multi core system.

Yes cache line ping-pong problem risk with spinlocks increase as system load goes up because the scheduler does not have the information it needs. This is made worse if system has multi NUMA zones.

Also you have the Priority Inversion problem. Again spinlocks scheduler does not know what is going on. So cannot use Priority Inheritance to fix this.

Yes there is a reason why well code Linux futex is doing a syscall on unlock that costs time this allow priority inheritance to be used.

Spinlock implementations are buggy. You cannot really make a Mutex or a futex do better if it results in either cache line ping pong or priority inversion problems.

Priority inversion on spinlocks is one of the things that cause dead locking of them so your absolute worse performance outcome.

Also your simple benchmark is kind of wrong.

You are recording just the worst and the average. When you watch game benchmarks on FPS.