Paul A. Clayton (paaronclayton.delete@this.gmail.com) on August 8, 2012 4:02 pm wrote:
> Eric (eric.kjellen.delete@this.gmail.com) on August 7, 2012 6:28 pm
> wrote:
> [snip]
> > As far as I can tell, there are really only two types of
>
> > workloads:
> >
> > 1. Latency-sensitive applications with irregular
> data access
> > patterns.
> > 2. Bandwidth-sensitive applications with
> highly regular data access
> > patterns.
>
> Often removing a latency
> bottleneck will expose a bandwidth bottleneck.
>
> Techniques to hide latency
> like multithreading and prefetching increase the number of memory accesses per
> unit time, but even just decreasing latency will tend to increase the frequency
> of memory accesses by having less effectively idle time (not just from data
> dependencies but from misspeculated control dependencies [and less frequently
> reaching the limits of speculation depth]).
>
> (I fear I am stating the obvious
> and not adequately appreciating the context.)

Well at least it was far from obvious to me, but after reading carefully I think I understand what you mean. =)

But doesn't this just mean that reducing memory latency has diminishing returns if memory bandwidth isn't also increased, which tends to be the case with advancing memory technology (and CPU caches, internal buses etc.) anyway? That is to say, is it really a problem that forces us to re-think memory technology or just a matter of naturally balancing latency and bandwidth improvements to existing memory subsystem models?