Maynard Handley (name99.delete@this.name99.org) on September 25, 2013 10:23 am wrote:
> It's worth noting just WHY IBM cares about eDRAM, and why they are willing to spend so much for it.
> Let's recall that hyper threading sounded like a great idea when it was first proposed, and the first
> commercialized, around 15 years ago. But the results were continually disappointing. Why?
>
> The basic problem is that the bulk of dead time in execution is waiting for loads that miss in the LLC. We
> want our hyperthread to do something useful during that time. BUT unfortunately our hyperthread lands up
> using some fraction of the LLC for its own purposes. And so we have that, yes, the other hyperthread CAN
> work when main thread is waiting on DRAM --- but in the process it shrinks the effective size of LLC, and
> this effect can be worse than the win of using dead time. All this holds regardless of other issues like
> whether or not you do a crappy job of the buffers you provide to your threads, how you schedule them etc.
>
> Exactly how this balances out depends on the workload and the LLC size. As we know, for most workloads,
> and once the mechanical details (buffers, scheduling) don't suck, for Intel sized LLCs (so about 1.5MB
> to 2MB/core) a hyperthread is worth about 25% of a core. Better than zero, but not great.
>
> IBM, selling big iron POWER boxes to people who run lots of threads want to do better than this
> --- they'd like their hyperthreading to work so that every slot of every CPU is constantly active.
> They solve this by throwing a MASSIVE LLC cache at the problem, like 80MB (10MB/core) on the theory
> that "god damn it, now maybe you threads will stop fighting and play nice together". I don't know
> if they've ever given numbers, but this would give 2.5MB/thread (IBM can run up 4 threads/core)
> which beats Intel's 1.5..2MB/core, which in turn suggests that for most code you ARE going to hit
> the goal here --- the actual CPU has basically every slot occupied every cycle.
> I don't think IBM have ever released numbers, but they thought it worthwhile to
> up the thread count from 2 to 4, so presumably the scheme works to some extent.
>
> OK, point of all this is that IBM is trying to solve a very particular problem with their eDRAM. A problem
> that in no way maps onto mobile. I don't know if we have any evidence that a larger than 2MB or so LLC
> per core (or thread, if you were to hyperthread) is worth the cost (especially in energy).


Also look at IBM's market for POWER. Lots of database heavy code that looks a lot like TPC. Cache misses all the way to DRAM are common, which takes forever in CPU cycles, especially when you're clocking at 4 - 5 GHz like IBM is. It makes sense to have a lot of threads for that type of workload, in fact I think they could see further improvement with more than 4 threads, though maybe they figured that was the point of diminishing returns where the added complexities weren't worth it.

Sun apparently thinks that point of diminishing returns is 8, since that's where they stopped with the Sparc T series (at least so far)