Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 10, 2014 2:22 pm wrote:
> David Kanter (dkanter.delete@this.realworldtech.com) on January 9, 2014 6:42 pm wrote:
> > > > I'll further add that I'm willing to wager that I'm correct about the 2 load
> > > > pipelines, and that Nicolas is wrong about a virtually addressed L0 cache.
> > >
> > > What exactly do you mean by 2 load pipelines? If you mean that both vector units would be
> > > able to use a memory operand in the same cycle, then yes, based on Eric's code that seems
> > > to be a necessity. However due to the duplicate loading of the same memory this can be provided
> > > by a dual-ported L0 cache, while the L1 cache itself only requires a single load port.
> >
> > I mean that the KNL core has two AGUs, both VPUs can execute
> > load+op every cycle and that the L1D cache can fill
> > two hits per cycle. I do not believe the cache is dual ported, as I believe it will be heavily banked.
>
> As far as I know that counts as a dual-ported cache.

No it doesn't.

>It's just implemented using multiple single-ported
> banks. Other ways to achieve a dual-ported L1 cache would be to duplicate the storage (like the Alpha 21164),
> or to double-pump a single-ported SRAM (like the Alpha 21264), or have actual dual-ported cells.

So we're making real progress here, because you have identified the terminology problem at hand.

A dual-ported cache is a cache that uses dual-ported cells. This is extremely expensive, and rarely done. Dual-ported cells are nice because you get two accesses/clock and no conflicts. Banking can yield conflicts if the address pattern is unfriendly. OOOE designs use banking, whereas many VLIWs need to use dual porting to avoid potential stalls which can really hit IPC.

As you pointed out, it is also possible to replicate the cache...but that's pretty expensive and has problems for stores. Operating the SRAM at 2X frequency is also possible, but costs power. I think IBM may have done this for POWER7/8, but I cannot recall.

> Anyway, a multi-ported multi-banked L1 cache is a reasonable possibility. I just don't see why it "must" be
> the only possibility, especially with x86 being a load-op architecture as the only explanation in >your article.

> Given that 1 byte/FLOP would suffice and that the code generated by ICC has a lot of duplicate >memory accesses,
> it also seems reasonable to me that there's a single-ported L1 cache and a dual-ported L0 cache.

First, 1B/FLOP isn't sufficient. Look at KNC, it can source 1x64B operand from memory and 2 from registers while performing 8 FMAs. That's 8B for 2 operations or 4B/FLOP.

Haswell also sources 64B/cycle while performing 8 FMAs.

So if you look at these designs, it's very clear that Intel believes that 4B/FLOP is the right amount of L1D bandwidth. Frankly, you can analyze compiler output all you want...but Intel invested millions in making the right decision and that's infinitely more convincing.

> L1 accesses cost a considerable amount of power (http://www.eecs.harvard.edu/~shao/papers/shao2013-islped.pdf),
> and if Intel wants to win the exascale race then they have to consider anything that might
> offer an improvement in performance/Watt. An efficient L0 cache and tuning the compiler
> to maximize its use might be part of such innovations.

It won't be. L1 caches are fine for KNL and the code you're seeing is probably the result of an immature compiler.

Caches might cost power, but the real problem is large scale coherency. The L1D is fine.

> > I'm willing to wager money that there is no L0 as you have described it.
>
> Gamblers wage a lot of money even though they know they have less than a 50% chance of winning
> but choose to ignore it. So your money doesn't mean anything to me even though you say this
> must be the only option for KNL. I won't wager anything because to me it's a coin on it's
> side. I think a dual-ported L1 is not the only option. That doesn't mean I think the other
> option is more likely. And neither does that doubt mean I think it's any less likely.

What I hear is that you aren't very confident of the L0 being the right solution, whereas I'm highly confident that it is the wrong solution.

David