Paul A. Clayton (paaronclayton.delete@this.gmail.com) on September 10, 2012 2:17 pm wrote:
[snip]
> Could this then require separately compiled code that uses locks
> either use the same compiler and options (so that all
> critical/locked sections are in transactions) or limit use to HLE

It seems I was not thinking clearly. It seems that even when using RTM a lock address must be added to the read set--not only to guard against accidental use of locks instead of RTM but also to handle the use of locks in the case of conflict when using RTM. (Unfortunately, this seems to indicate that any failure involving a particular lock will tend to block non-conflicting transactions temporarily. Using retry with a conflict-avoiding delay could avoid having to set the lock under modest contention and when the lock is unset all waiting threads could try using transactions, so the penalty might be small.)

(It seems that in theory a transaction could check the lock at the beginning of the transaction, remove the lock address from the read set, and finally check the lock at the end of the transaction. Any locked sections that are fully enclosed within a transaction will be known to be non-conflicting. One could even conceive of setting up a short duration MWAIT-like operation to wait for the release of the lock so that the transaction could commit, but that seems excessively complex especially with the possibility of [partially] nested transactions.)

It looks like I would need to spend quite a bit more time thinking about this stuff before I even mostly understand it.