Bhima (Bhima.Pandava.delete@this.gmail.com) on November 25, 2013 8:01 am wrote:
> Patrick Chase (patrickjchase.delete@this.hp.com) on November 23, 2013 3:37 pm wrote:
> > The real benefit is that with TM you can often get close to that performance with
> > a "naive", coarse-grained locking implementation that takes an order of magnitude
> > less effort to develop. Real world HPC folks care deeply about that sort of thing...
>
> I'm really glad you brought this up because when I read this I was a bit confused with
> that statement as it was my understanding that the motivations to include transactional
> memory in the other Xeons were in part to simplify memory management for the many
> parallel processes that having all those cores & threads (in the regular Xeons) enables.

I think that's about right, though I would change "simplify memory management" to "simplify synchronization". The difference in complexity between:

1. A single big lock that protects everything

and:

2. Fine-grained locking such that no two processes take the same lock unless they're actually going to perform conflicting accesses

Is *huge*. If you can use TM-based speculation to get adequate performance from (1) then you're way ahead in the game, both in terms of development effort and likelihood of correctness.

For example the a single lock is inherently deadlock-free, while fine-grained locking tends to be a disaster. If you always know what set of locks you'll need ahead of time then there's a simple fix (impose a fixed order on all locks in the system and always take the set of locks you need in that order - you can't have a cycle in an ordered graph if you never go backwards) but that's often not the case.

The catch is of course that TM has limited "capacity" (you can only read/write so much data before you're forced to abort), so you probably couldn't use it to mitigate the impact of really "huge" locks such as the dearly departed Linux BKL or the Python GIL.

> Hopefully, this topic will be included in the upcoming article and more fully discussed.
> I find the concept of transactional memory intriguing but clearly I must be
> misunderstanding something important.

I don't see anything in your comments that indicates lack of understanding. I think that a lot of OTHER people made the mistake of evaluating it as though it were a performance enhancement rather than a simplification.