Hi Brett,

Brett (ggtgp@yahoo.com) on 2/26/11 wrote:
---------------------------
>>>I am talking about adding a 4k to 16k local memory to each ATI pipeline. (Think Cell.)
>>>Which of course would be running at L1 cache speeds.
>>>Not much benefit to moving centralized bulk RAM on chip, as we both agree.
>>
>>How is that different from just increasing the register file (which on GPUs is
>>an SRAM array anyway)? I think you might be underestimating the die area for high
>>speed local storage. Like I said before, what you gain in effective throughput,
>>you lose in compute density. So that's not solving the scaling issue.
>
>ATI has historically just increased the register count.
>But now they are at 128 general registers and 512 constants registers.
>I think that mine is empty now, bigger is slower and hotter, need to add another level.

While multi-level storage indeed offers both high performance and high density by taking advantage of locality of accesses, it still takes extra area, and there's only so many levels you can add. A modern GPU's architectural registers are actually physically stored in SRAM, not in register files. So they're already at the point where the only other level that can be added is eDRAM or similar technology. That won't help for long.

To reduce the actual storage needs, the GPU has to become capable of advancing its program counter more quickly. It needs CPU technology for that, like the use of superscalar execution in NVIDIA's latest chips.

>>What you're describing is tile-based deferred rendering (TBDR). This approach is
>>used by PowerVR's architectures (as you mention below), but hasn't proven to be
>>the silver bullet for high-end GPU scaling. The problem is that while storing frame
>>buffer tiles on-chip is easy, you also need vast amounts of gradient and rasterization
>>data. You can't afford to re-process primitives for every tile.
>
>It would appear that ATIs LDS (Local Data Store) is for holding neighbor done flags for tiling, etc.
>So AMD is already doing as much TBDR as the hardware can, and that the OpenGL compiler can understand and compile for.
>I expect this trend to continue.

Whatever it is you're talking about, AMD does not do TBDR.

>>>There is a Stanford talk on the limits of power, 10 picowatts(?) for a register
>>>read cost as much as an FPU operation, 100 pW for L1 read, 1000 pW for L3 read, 10,000 pW for DRAM read.
>>>
>>>Local data gives you orders of magnitude power reduction.
>>
>>What you're forgetting is that local data is accessed a lot more often. For ever
>>pixel written to DRAM, the shader probably accessed several hundred registers and
>>accessed a dozen textures. So just saving the power consumption of writing the pixel
>>to DRAM doesn't save you much. Especially when you consider that it requires additional
>>on-die storage and thus lowers the compute density, saving power by reducing performance
>doesn't seem that useful to me.
>
>The issue with ATI r700 is that all texture writes spill to the 64 way shared cache.
>Adding a local cache for non-shared data stops a large number of those writes from
>taking place, which can save a lot of power.
>Of course a local cache is also going to cost some power as well.
>
>The ATI r770 switched to local caches, local and remote caches total 5 megs worth.
>(I just found this out, sorry for being out of date.)
>So AMD is doing what I thought would happen.

Yes, previous architectures were not that efficient, while with current architectures they're putting more thought into it to avoid wasting area on storage. But note that this also means they're already running out of options for the next generations. First maybe some eDRAM could be added, but then what? An even bigger eDRAM?

Clearly they have to seriously start considering whether improving single-threaded performance isn't a better option than adding more storage. So convergence toward the CPU is inevitable.

>It looks like you can do full TBDR on the r770, its just a software issue.

It's possible, but it's not efficient due to still very limited on-chip storage. Here's similar research from Microsoft that comes to the same conclusion: http://research.microsoft.com/en-us/um/people/cloop/msrtr-2009-83.pdf

>>>The PowerVR graphics chip in the iPhone is a tile engine, as opposed to the ATI/NVidia stream engines.
>>>Tile engines are the future, today streaming is faster/simpler/easier if you can afford the power cost.
>>>(The PowerVR tile engine has some serious flaws, its not the future yet.)
>>
>>What makes you think that PowerVR's architecture can be scaled into a high-end
>>GPU, and the "serious flaws" are actually fixable? Note that Direct3D 11 has tesselation
>>so you need a way to manage a lot of geometry. I don't see TBDR going anywhere.
>I would love to be proven wrong though.
>
>Tessellation is easy, you can farm that out to lots of vector units, and then farm
>the results to the proper tile units/queues.
>A tile does not care that it is getting polys from multiple vector units.
>That single display list the programmer sees is a myth once the hardware gets ahold
>of those verts and spreads them hither and yon.

You still need lots of storage for all of that!

>I would love to know if TBDR is doomed on the high end.

On low-end hardware TBDR is feasible because you can have high bandwidth compared to the computing performance. So it's ok to store things in global memory. Furthermore, the polygon count is going to be small. But on high-end hardware the computing performance is so high that you need local storage to provide the bandwidth to keep the ALUs busy. But that local storage is too small for TBDR, especially since the polygon count is much higher.

The main benefit of TBDR, overdraw reduction, can be achieved more efficiently with a pre-z pass.

Cheers,

Nicolas