David Kanter (dkanter.delete@this.realworldtech.com) on January 5, 2014 1:44 am wrote:
> Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 4, 2014 4:24 pm wrote:
> > David Kanter (dkanter.delete@this.realworldtech.com) on January 4, 2014 10:58 am wrote:
> > > Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 4, 2014 5:43 am wrote:
> > > > David Kanter (dkanter.delete@this.realworldtech.com) on January 2, 2014 wrote:
> > > > > "AVX3 is a load-op instruction set that can source one operand from memory for each
> > > > > operation (typically assumed to be a fused multiply-add or FMA). This means that the
> > > > > KNL core must have a 512-bit (64B) wide load pipeline for each vector unit."
> > > >
> > > > That seems like a bit of a leap.
> > >
> > > Nope, it's solid analysis. I'm willing to wager $ that I'm correct here.
> >
> > I'm not seeing said solid analysis. You're leaping from AVX3 being a load-op instruction set, to
> > Knights Landing necessarily having two load ports.
>
> That's one obvious reason. The second is that without a second load port, the bytes/FLOP ratio is absolutely
> terrible. Doubling the FLOP/s without doubling the bandwidth is really a pointless exercise. It makes a
> great chip for linpack...but nobody cares about that. Real workloads are bandwidth constrained.

With one 512-bit load port for two 512-bit FMA units, it would equate to 1 byte/FLOP. That's the same as GCN, and more than Kepler. Previous GPU architectures had higher bandwidth per FLOP, so at some point they determined that doubling the FLOPS/s without doubling the bandwidth is not a pointless exercise.

> >All other x86 arithmetic instructions can also
> > have a load operand, so why does Silvermont not have two load ports, using the same reasoning?
>
> Because nobody cares that Silvermont can sustain peak FLOP/s.

It's not just about FLOPS for Knights Landing either. It's about performance/Watt. And if Silvermont was significantly bottlenecked by L1 accesses in scalar integer workloads they would have given it two load ports to improve performance/Watt.

ALUs are cheap, while bandwidth is expensive, so you can easily double the peak FLOPS but you want to be conservative about the number of L1 ports.

> > I'm not saying Knights Landing can't have two load ports, there might be other reasons why it does. So
> > I'm not betting on anything.
>
> So what you're saying is that you aren't very confident in your position then : )

No, what I'm saying is that I'm not taking any position. But for argument's sake let's say my position was that there's a 50/50 chance of Knights Landing having just one L1 load port instead of two. A coin on its edge. That means I'm just open to the possibility, because in my opinion it wouldn't be a terrible architecture. If it does have two ports, then I'm just as fine with that, because it means that Intel carefully analyzed the cost/gain and determined it doesn't take much power and area and there's enough need for two (unique) load operations per cycle to make it worth it. In contrast it seems you're not even willing to consider the possibility of one load port at all. You might end up being perfectly right due to only a very slight net gain for having two load ports. That doesn't mean one load port should have been dismissed as an option that easily. A slight difference in target workloads or process characteristics could have tipped it the other way.

So my only real position is that one load port should be considered as a possibility, and there's too little explanation in the article why that wouldn't be viable. Note that Eric's suggestion of a buffer which caches recently loaded data would allow for both pipelines to have memory operands while the L1 cache would still be single-ported. If I had to take any position I'd say a single-ported L1 with a kind of 'L0 cache' seems most plausible.

> >I'm just pointing out that one 512-bit load port seems like a possibility
> > as well. If you could elaborate on why it "must have" two load ports, that would be much >appreciated.
>
> bytes/FLOP, predictable latency, the list goes on.

Bytes/FLOP is plenty with just one load port in comparison to other throughput architectures. Predictable latency can be achieved with the L0 cache. Any other reasons on your list?

> > > > Wouldn't it be possible for one unit to be able to have a memory
> > > > source operand and the other to be able to have a memory destination operand?
> > >
> > > It's also possible for that to happen with 2 LSUs.
> >
> > Certainly. But what I meant was having *only* one pipeline capable of taking a memory source operand,
> > and the other one being the only one capable of storing the result into memory. Based on typical
> > load/store requirements (see for instance Eric's example above) it's very useful to be able to issue
> > a load and a store each cycle, but two loads per cycle seems much less of a necessity.
>
> Why?

Because typical code only requires one (unique) memory source operand every two instructions, especially when you have 32 registers.

> > > > AVX-512 has a
> > > > relatively high number of registers so there are few stack accesses. GPUs are also fine with
> > > > having low L/S bandwidth per FLOP because of this.
> > >
> > > GPUs can have 100-200 registers per thread. KNL can have 32.
> >
> > Kepler can have up to 255 registers per thread, but only at 1/8 occupancy. I
>
> >At full occupancy, 31 registers
> > are available. AVX-512 can have many more than 32 live
> > temporaries (even more than 255), by using the stack.
> > This comes at practically no cost as long as there are available load/store cycles, and only very gradually
> > decreases performance beyond that. In contrast the GPU can suffer significantly from the lower occupancy.
> > Some optimization manuals therefore advise to use local memory to lower register pressure...
> >
> > Anyway, the conclusion is still the same. AVX-512 has a
> > relatively high register count, which lowers the need
> > to access stack memory and thus may be a factor in why Knights Landing might have only one load port.
>
> Except it will have two load ports, so that's really irrelevant.

You can't call anything irrelevant by postulating your theory as fact. That's backward logic.

> > > > Requiring only two AGUs and one load and
> > > > store port significantly simplifies things and keeps power consumption down, which might be
> > > > more important than ensuring there's never a cycle lost because of load port availability.
> > >
> > > KNL will only have two AGUs, but the configuration is unclear. It could
> > > be 2 LD/ST or 1 LD + 1 LD/ST. There is no way it will be 1 LD + 1 ST.
> >
> > Why?
>
> Because it would be a terrible design for any real workloads.

So every GPU in existence today is a terrible design for any real workload?