NoSpammer (no.delete@this.spam.com) on January 5, 2014 1:15 pm wrote:
> Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 4, 2014 4:24 pm wrote:
> > I'm not saying Knights Landing can't have two load ports, there might be other reasons why it does. So
> > I'm not betting on anything. I'm just pointing out that one 512-bit load port seems like a possibility
> > as well. If you could elaborate on why it "must have" two load ports, that would be much appreciated.
>
> I thought David was overoptimistic, too.

I don't think he's overoptimistic (about two L1 load ports). I think he's overpessimistic (about one L1 load port).

I'm sure two load ports is quite possible, and even if it weren't necessary the cost would be modest overall. I just don't see any justification why one port can't work and needs no further analysis.

> Let's look at the basics, supporting 4 threads requires
> at least 128 logical registers. The register file needs to have at least 2 (but probably 4-6)
> read ports and 2 write ports. That's one huge beast. Now OTOH 32K of L1 cache would contain
> only 512 register equivalents. Having 1 read and 1 read/write port to L1 seems to be in the
> range of 2-4x (but probably closer to 2x) complexity of the register file. That seems doable
> to me. I guess the challenge is in blocking and power optimizing all those wires.

Doable, yes, but if it's not a necessity for achieving better performance/Watt they would obviously stick with one port. Note that Nehalem still had just one load port, despite a peak IPC of 3 instructions and already having 8 cache banks and fast unaligned 128-bit access support. This tells me the cost still wasn't justified, even though 32-bit x86 code requires lots of operands to be loaded from the stack.

> > Certainly. But what I meant was having *only* one pipeline capable of taking a memory source operand,
> > and the other one being the only one capable of storing the result into memory. Based on typical
> > load/store requirements (see for instance Eric's example above) it's very useful to be able to issue
> > a load and a store each cycle, but two loads per cycle seems much less of a necessity.
>
> IMO it's pointless having two MAC units without being able to feed them both
> most of the time. One load per FLOP in necessary for many kinds of codes.

It's true that FMA units have an increased need for source operands, but for starters AVX-512 has 32 registers to compensate for that. Eric's example code has many FMA instructions but still less than half the total instructions require a source memory operand (http://www.realworldtech.com/forum/?threadid=138897&curpostid=138932). His second example has more source memory operands than that, but they're not unique, so that need can be fulfilled with an L0 cache.

I assume you meant one load per FMA instead of one per FLOP? Still, I don't think there are many kinds of code which need such number of (unique) load operations at all. GPUs have 1 byte/FLOP or less.

> > Kepler can have up to 255 registers per thread, but only at 1/8 occupancy. At full occupancy, 31 registers
> > are available. AVX-512 can have many more than 32 live
> > temporaries (even more than 255), by using the stack.
> > This comes at practically no cost as long as there are available load/store cycles, and only very gradually
> > decreases performance beyond that. In contrast the GPU can suffer significantly from the lower occupancy.
> > Some optimization manuals therefore advise to use local memory to lower register pressure...
>
> That's interesting.
>
> > Anyway, the conclusion is still the same. AVX-512 has a
> > relatively high register count, which lowers the need
> > to access stack memory and thus may be a factor in why Knights Landing might have only one load port.
>
> I think the relevant questions here are only:
> - what are the bandwidth/FLOP requirements on the algorithms intel wants to optimize for
> - what are the actual power/performance ratios for the circuits feeding 2 MAC units from L1 at full tilt

I don't think Intel wants to target any particular algorithms. It's not like they'll have multiple architectures to target different segments, nor is the code being run always the same. Just like the GPU manufacturers, they target anyone who has massively parallel tasks. But that doesn't mean they should add two load ports 'just in case' either. In the end it's all about performance/Watt. I believe code which benefits significantly from loading a unique source memory operand per instruction is quite rare, but everyone else would constantly pay the cost in power (and area) for an extra port they don't use.

Indeed the actual cost for adding a second load port would be very useful to know. And that's my point. We can't dismiss the possibility that Knights Landing will have only one L1 load port without a full cost/gain analysis.

> If they can power optimize L1 with 2 load ports they will do it.

Definitely, if there's a need for it in the first place. An L0 cache seems like an excellent compromise to me. It would add another load port for common cases while being more power efficient than accessing the register file. Two birds one stone.

> The other alternative possibility would be supporting 1 L1 load and 1 L2 load in the same cycle.

No, you'd easily run out of work and stall while trying to hide the higher latency of an L2 access, meaning you're better off waiting just a single cycle to access L1 again. If there's really a need for two (unique) load operations per cycle, it would have to be two ports on L1.