Hi Brett,

Brett (ggtgp@yahoo.com) on 4/5/11 wrote:
---------------------------
>Nicolas Capens (nicolas.capens@gmail.com) on 3/9/11 wrote:
>---------------------------
>>To reduce the actual storage needs, the GPU has to become capable of advancing
>>its program counter more quickly. It needs CPU technology for that, like the use
>>of superscalar execution in NVIDIA's latest chips.
>
>Both NVidia and ATI/AMD have introduced new dual chip high end cards in the $700 segment.
>Both are made at 40nm on a TSMC process, so we can compare the tradeoffs and see what the future may look like:
>http://www.xbitlabs.com/articles/video/display/geforce-gtx-590.html
>http://www.xbitlabs.com/articles/video/display/geforce-gtx-590_11.html
>>
>
>ATI has 3000 simple pipelines at 880 MHz.
>NVidia has 1000 out-of-order pipes with part of the chip at twice the clocks.
>RESULT: A Draw. (Pun intended.)
>
>Die size ATI 389 mm2, NVidia 520 mm2, so the ATI chip is 75% of the size of NVidia.
>This means really bad things for NVidia yields and costs and competitiveness.

Superscalar execution is not the same as out-of-order execution. But anyway, you can't evaluate the effect of superscalar execution by comparing two radically different architectures. Note that AMD was already executing multiple instructions from the same thread in parallel, using VLIW. They toned it down from VLIW5 to VLIW4 though. NVIDIA on the other hand went from scalar execution, to 2-way superscalar execution. But if there's one thing we can conclude it's that both architectures became more efficient by investing into more control logic. The next step is adopting technology like out-of-order execution.

Also, while the GF110 has a certain size disadvantage, keep in mind that it's also a much more successful HPC chip (ECC, high DP performance, advanced coherence, etc). Note that the GF114 has a better area/performance ratio for gaming. The high-end market is relatively small, and early adopters don't care as much about price as the mainstream market, so all in all NVIDIA isn't doing badly at all.

>There is a HUGE anomaly in that the 50% wider memory bus on the NVidia chips has
>the same bandwidth as the ATI chips. (384 bits vs 256 bits)
>ATI RAM clocked at 5000MHz vs NVidia 3414MHz, 46% faster.
>
>This screams bus thermal limits getting hit.
>I bet the interface and DRAM thermals just do not matter, its all the on chip bandwidth thermals that matter...?

I haven't done a detailed analysis but keep in mind that slower clocked RAM is cheaper and has a lower latency. Also, there might simply not be higher clocked ECC memory so NVIDIA needed the wider bus anyway. Another explanation is that AMD took a gamble by expecting GDDR5 to scale up to these high speeds while NVIDIA chose to take a less aggressive route. In any case there are a lot of parameters to take into consideration.

>>>Stanford University Online has a truly awesome related iTunes talk by Bill Dally:
>>> http://itunes.apple.com/itunes-u/programming-massively-parallel/id384233322
>>>Watch Track 13, the course "Programming Massively Parallel Processors with CUDA (CS193G)".
>
>You were a heavy poster here, then I posted this link a month ago and you fell off the net... ;)

I didn't fall off the net. :) It was simply taking a lot of my spare time to keep responding to people who kept rejecting my arguments based on short term or small scale observations. Also, my goal was to learn about the main convergence challenges. My overall premise hasn't changed (convergence will continue) but I got a better insight into how things are likely to evolve. So I stopped caring to respond to discussions that didn't appear to lead to new valuable information.

>I am actually a fan of scatter/gather, which you were promoting.
>But the simpler form of short offset off a base, useful for decoding OpenGL display lists and similar compressed data.
>This can be implemented cheaply, with simple generally sequential cache lines reads until the full request is supplied.
>The full four address form requires four address computations and four reads that
>you have to cross check for cache line duplicates to optimize.
>Of course this full form is what you need for texel sampling, which is what you want?

Not just texel fetching, any loop with indirect addressing and independent iterations can benefit from it. Pretty much all software has loops like that, but compilers have a hard time automatically vectorizing them because sequential insert/extract creates a bottleneck. Gather/scatter would make that straightforward and speed things up considerably.

And it doesn't require a lot of hardware. You don't have to compute the full address for each element; you just need to compare the offsets to check whether the same cache line is used. Here's a more detailed explanation: http://www.edaboard.com/thread202326.html

It's perfectly feasible for a future CPU to have a gather implementation which takes between 1 to 4 cycles for an 8-wide AVX gather when all data is in L1 cache. That's way better than the current 18 cycles!

>Larrabee tried a full 16 way scatter/gather, I thought they were crazy. ;)
>Die size and thermals killed Larrabee twice over?
>Intel did impress me by actually designing and manufacturing such a chip.

I doubt gather/scatter is what killed LRB1. I believe Intel underestimated both the hardware and software challenges. Newer versions will probably make a much better impression, but it still takes a lot of software to make good use of its capabilities before consumers will consider buying it. While you can do awesome things with a generic chip like that, the world currently mainly only cares how well a GPU runs Crysis and such.

One way for Intel to get Larrabee in the hands of consumers is by integrating it into a next-gen console. They don't need the highest possible performance at current applications; being future proof and allowing lots of innovation is more important. Just look at how popular the PS3 is despite having a slow GPU by today's standards. The Cell CPU makes up for a lot of that and has allowed the games to look ever more impressive: http://www.technology.scee.net/files/presentations/gdc2009/DeferredLightingandPostProcessingonPS3.ppt

>NVidia and ATI have dedicated texel sample units, this would appear to be the best solution?
>The texel unit is close to its data, does 16 samples from its cache, and passes
>the one result far away to the processor.
>If the processor does texel reads then you blow your data cache on texels, and
>blow the pipeline die size on the scatter/gather unit.
>Bigger pipeline is slower and hotter.
>
>Chopping the task up to localize data and computation gives you a massive thermal advantage.

It's not that simple. With too much specialization you could end up with an extremely efficient chip for running one particular application, but unable to adequately run other applications. People obviously don't want that. They want something that runs all current applications well, and also won't choke on applications released 3 years from now.

The average TEX:ALU ratio is still slowly but surely decreasing. You can decrease the relative number of TEX units, but then it becomes a bottleneck for any code which has a higher than average TEX:ALU ratio. So fixed-function hardware is always overdimensioned and thus takes a significant amount of area. Also note that it doesn't scale well. Even if you had just a single sampler, you still need all the support logic and communication channels. Furthermore, the expectation is that future hardware will perform filtering at full FP32 precision so samplers become even bigger. It all depends on the usage pattern whether it's worth it or not. In the long term, dedicated samplers may no longer make sense.

To give you another example, note that the fixed-function alpha test and fog stages disappeared, despite the fact that dedicated hardware is in theory more efficient. Their usage changed so replacing them with shader code turned out to be more efficient in practice. So there's no guarantee other fixed-function hardware will stay around when the usage changes.

>Larrabee hedged its bets by including scatter/gather AND dedicated texel units.
>If Larrabee 3 ever sees the light of day we may see some design changes here.

Yes, and I'm very curious about what direction AMD and NVIDIA will evolve their architectures into as well!

Cheers,

Nicolas