Hi David!

David Kanter (dkanter@realworldtech.com) on 1/25/11 wrote:
---------------------------
>Nicolas Capens (nicolas.capens@gmail.com) on 1/25/11 wrote:
>---------------------------
>>Hardware support for compressed textures is not critical to the viability of software
>>rendering. In bandwidth-limited situations, it only helps by about 10%. Off course
>>in the case of a GPU you want that 10% since every cent you pay has to count, but
>>for a CPU running graphics plus every other application, >it's a different trade-off.
>
>I'm very skeptical. Texture compression should be much more effective than 10%.
>Do you have any numbers to back this up?

Doom 3 on a GTX 460 at 2560x1600 4xAA runs at 53 FPS at Ultra High Detail, and at 56 FPS at High Detail. I was being generous when I said 10%.

To clarify this result somewhat, here's the bandwidth usage of Unreal Tournament 2004 at 1024x768, using compressed textures: http://personales.ac.upc.edu/vmoya/img/bw.png

Although bandwidth usage differs a lot between games, we can observe a few things which reduce the impact of texture compression. First of all, games aren't memory limited all the time. The memory bus is overdimensioned to prevent that from happening too frequently. It means that even if the bandwidth requirement would double, the performance would not reduce in half. Also note how color and Z each take a very significant amount of bandwidth. And the balance tips over even further with deferred rendering.

All these effects combined result in texture compression only helping roughly 10%. Note that in the case of Doom 3, the main reason for not disabling texture compression was VRAM size. When textures had to be swapped in and out over the AGP or PCIe bus the performance hit was huge, but it wasn't a VRAM bandwidth issue.

For a GPU the texture decompression logic is worth it mainly to keep VRAM size smaller (read: cheaper). The bandwidth savings are a welcome bonus but I doubt it would be a justification by itself.

>>That said, it's not at all impossible to add texture >decompression as a CPU instruction...
>
>Won't happen.

That was my conclusion as well. They would never add instructions which could become of little or no use in several years.

I merely mentioned it to note that there's nothing a GPU can do, which a CPU could not possibly do. Also, there are plenty of other alternatives, should it be necessary. And again, it doesn't have to match the efficiency of dedicated hardware to be viable.

Most likely the bandwidth will just steadily keep increasing, helping all (high bandwidth) applications equally. DDR3 is standard now accross the entire market, and it's evolving toward higher frequencies and lower voltage. Next up is DDR4, and if necessary the number of memory lanes can be increased. Either way I think there's plenty of bandwidth already for the market which merely needs minimal graphics support.

>>Exactly. But since software renderers have access to lots of system RAM, that's
>>not really an issue. And the bandwidth issue is largely >compensated by the CPU's big caches.
>
>That's really only true if your textures and rendering algorithms are cache friendly.
>My understanding is that most textures are not that cacheable.

For the case of Crysis, the terrain and vegetation frequently reuse the same textures. So this benefits quite a bit from the large L3 caches.

And again CPU technology is not at a standstill. With T-RAM just around the corner we're looking at 20+ MB of cache for mainstream CPUs in the not too distant future.

And while the expectations for 'adequate' graphics go up as well, it's only a slow moving target. First we saw the appearance of IGPs as an adequate solution for a large portion of the market, and now things are evolving in favor of software rendering.

>>SwiftShader runs 30% faster on a Sandy Bridge chip with >only 55% of the bandwidth.
>>There is no clearer proof that software rendering isn't >bandwidth limited.
>
>That's only meaningful for swiftshader, not all SW rendering. How does it work
>with MSAA and anisotropic filtering and other more advanced techniques?

I don't see why it would only be "meaningful" for SwiftShader. The public demo does not take advantage of Sandy Bridge's new features. So I expect other software renderer's to benefit roughly the same.

I haven't had the chance yet to see how Sandy Bridge performs with multi-sampling, but I'll keep you posted when I do. Anisotropic filtering doesn't seem relevant to bandwidth, since all texels will be in cache due to high temporal coherence. In fact it would make it even more gather/scatter limited.

>Also, what is your comparison against? It sounds like you're saying Sandy Bridge
>has 55% more memory bandwidth than whatever your baseline is...which sounds like
>an awful lot. Sandy Bridge's memory controller isn't that much faster than Arrandale AFAIK.

No, I said Sandy Bridge has only 55% of the bandwidth of my baseline, which is an i7 920 @ 3.2 GHz, using 1600 MHz memory and three channels. So if I'm not mistaken that's 1.85 times the memory bandwidth versus the i7 2600 I tested. Despite that, the 2600 is 30% faster!

So again, software rendering is nowhere near being RAM bandwidth limited. Just like many other compute intensive applications, it needs gather/scatter before any other parameter starts to matter. Note that gather/scatter wouldn't just allow a massive reduction in the number of serial load/store instructions, it would also reduce the number of swizzle operations: http://software.intel.com/en-us/forums/showpost.php?p=139770

>>The trend is that the number of texture accesses is going >down, relative to uncompressed
>>unfiltered memory accesses.
>>So the bandwidth savings is getting smaller. The reason
>>for this is obvious: there's only so many textures you can >slap onto your scene's geometry.
>
>Then you just increase the geometry count, or use tessellation. It seems like
>geometry would scale at the same rate (or maybe a bit slower) than texturing.

With all due respect, you're sounding a bit like the people who said dedicated vertex and pixel pipelines make more sense than a unified architecture because "you just increase the geometry count".

It's clear that telling software developers what (not) to do doesn't result in a succesful next generation hardware architecture. With non-unified architectures, there were numerous applications which were vertex processing limited, and numerous ones which were pixel processing limited. And even those in the middle have a fluctuating workload.

So instead hardware architectures should accomodate to the increasing variety in workloads. This means generalizing the texture units into gather/scatter units and performing filtering in the shader units.

Anyway, you were probably still referring to bandwidth? Then increasing the amount of geometry in the scene won't tip the balance in favor of texture compression. With deferred shading or pre-z passes additional geometry only increases the color and/or z bandwidth needs (on top of the geometry bandwidth of course). Tesselation can make use of compressed textures, but note that it's sampled at a low frequency. Plus it massively increases the vertex processing workload, so in fact you're less likely to get bandwidth limited at all.

>>At the same time, caches are still getting bigger and >bandwidth is still increasing.
>
>That's not the question. It's 'are caches increasing in size sufficiently faster than texturing data'.

Absolutely. Ten years ago the L2 cache was 256 kB, nowadays we have 8 MB of L3 cache. That's 32 times more. Games did not evolve from ~2 to ~64 texture samples per pixel. Even if you count in the increased screen resolution and additional scene complexity (which by the way also scale the computational needs), the texture bandwidth needs did not increase 32-fold. Even for the high-end the memory bandwidth has not increased by that much. And it increased far less in the low-end graphics market. IGPs are focussing on increasing their computing power more than their texturing abilities.

In ten more years caches could be around 256 MB, and that's without taking revolutionary new technologies like T-RAM into account. So it's really hard to imagine that this won't suffice to compensate for the texture bandwidth needs of the low-end graphics market.

SwiftShader 1.0 was first used by a 2D casual game called Galapago. Despite the game's graphical simplicity, it was totally texture sampling limited and it was barely reaching the necessary 10 FPS for playability. That was five years ago. Today we have Crysis running at 20+ FPS.

I don't need any crystal ball to see that gather/scatter will make IGPs redundant.

>>There are even strong indications that T-RAM technology is >nearly ready for production.
>>It combines the density of DRAM with the speed of SRAM.
>
>I know the guys at T-RAM. Their technology is very interesting, but it's not at
>all clear that they are close to production in a CPU or GPU.

http://www.businesswire.com/portal/site/home/permalink/?ndmViewId=news_view&newsId=20090518005181

Anyway, there are multiple high density cache technologies. There's Thyristor-RAM, 1T-RAM, 2nd gen Z-RAM, TTRAM, etc. So regadless of whether it takes one year, two years, three years... does it really matter? Cache technology is expected to exceed Moore's Law at some point. Together with gather/scatter this helps the CPU compensate for its lack of dedicated texturing hardware, allowing it to significantly gain on GPUs. So sooner or later software rendering is going to take over the low-end graphics market.

>>This means that sooner or
>>later the majority of the data used in a frame will fit in >the cache, further reducing
>>the relevance of texture compression.
>
>That depends on how fast the geo, vertex, texture, pixel, etc. data scale. Is
>is faster or slower than cache size increases?

Slower. And that's without factoring in the massive increase in computing power you need for processing all that data. And caches don't need to store all data used in a frame, to already get high reuse.

Seriously, there are more critical things to worry about that determine the viability of software rendering. Currently a lot of cycles are burned on getting the data into the right layout for the SIMD units to process it.

>>We're not there yet, I realize that, but nothing is >stopping the steady convergence toward software rendering.
>
>Power efficiency?

Power efficiency is obviously important, but you have to think about what's possible within a certain power envelope. The acceptable TDP has reached a ceiling, but that doesn't mean only the most power efficient solution is acceptable. It didn't stop audio processing from becoming a software driver, even when Pentium 4's were the least power efficient architecture...

Sandy Bridge is designed to be able to use four cores plus an IGP, all within 95 Watt. During gaming or other intensive applications, that's perfectly acceptable. As you know, Sandy Bridge is amazingly efficient. So it should be equally acceptable to trade the IGP for two more CPU cores, add gather/scatter capabilities, and as a result get a really powerful CPU which can also take care of graphics, all within 95 Watt.

It will take several more generations for gather/scatter support to arrive, so by that time we're on 22 or 16 nm so you get lower power, more performance, and/or lower power consumption.

By the way, currently the IGP part of Sandy Bridge is a total waste of die space when using a discrete graphics card. Instead they could have had a single 6-core architecture which does software rendering for people who have low graphics needs, and it would be a more powerful CPU for people with a graphics card.

>>But what's the next step? Obviously there's a quest for higher quality so it was
>>only a matter of time before compressed HDR formats became available so they actually
>>fit in VRAM. But we don't really want compression; it's just a temporary necessity
>>for dedicated gaphics.
>
>You always want compression for something you store in memory. It increases effective
>bandwidth and reduces capacity used. You might want higher fidelity or lossless
>compression, but you'll still want it. Less memory traffic is lower power and fewer pins.
>
>>Once the bandwidth and VRAM goes up the purpose of >compression
>>diminishes again. There's not much beyond this, and if >there is, it's not likely
>>of interest to the markets that would show an interest in >software rendering first.
>
>Again, only sort of. There's always a benefit to compression.

You don't unconditionally want it. It's still a trade-off. For this reason lossless compression of system memory is hardly more than a conceptual idea. Apparently it's cheaper to increase the bandwidth and capacity the brute force way.

>>To me it's very clear that the lack of parallel load/store >is a huge bottleneck.
>>The ALU's can do 16 operations in parallel, but only 2 >load operations and 1 store
>>operation can be executed per clock. And it gets worse >with FMA.
>
>Um, Sandy Bridge can do 32B/cycle of loads, that's 8 values.

It can load 2 x 4 *consecutive* 32-bit values (per core per cycle). That's not the same as 8 values. As illustrated in the link I gave above, that's worthless for something like a parallel table lookup.

Gather/scatter would make a massive difference for a large range of applications. It's the parallel equivalent of load/store so it would allow parallelizing nearly every loop. It's the one missing thing that prevents auto-vectorization from becoming widely usable.

>>Gather/scatter would speed up texture sampling, vertex >attribute fetch, primitive
>>setup, rasterization, transcental functions, raster >operations. It's incredibly
>>versatile and responsible for a large chunk of the >remaining CPU-GPU gap.
>
>That's part of it...but scatter/gather generally means you MISS in the cache/TLBs and chew up more bandwidth.

Why? It only accesses the cache lines it needs. If all elements are from the same cache line, it's as fast as accessing a single element. But even in the worst case it can't generate more misses or consume more bandwidth.

>Also, I'd point out that there is at least a 4X difference in memory bandwidth between CPUs and GPUs.

So? IGPs are getting pretty powerful, while sharing the same bandwidth as the CPU. So with texture compression only having a marginal effect, bandwidth is not a factor which would prevent software rendering from being viable.

Once the IGP has gone the way of the dodo, more powerful CPUs can be created for people who want mid-end graphics performance. I bet Intel would really love selling quad-channel 12-core CPUs not just to the server market but to gamers as well. Sandy Bridge already pretty much made the low-end discrete graphics cards obsolete, and Intel is not likely going to lose its grip on that massive market.

Note that a 4 GHz 12-core CPU with FMA would be capable of 768 GFLOPS. But while that is achievable in the short term, it's still going to take longer to get all the pieces of the puzzle into place. So by then we're talking about several TFLOPS. At that point legacy rasterization isn't very exciting any more, but the CPUs additional programmability capabilities will really allow software developers to create something unique.

>>So in five years or so people who don't need cutting-edge >graphics will have plenty
>>of graphics power from their CPU cores.
>
>Yes, but they will have even more graphics power available through the integrated graphics.

Which is then an expensive pointless piece of silicon for those who are not interested in even more graphics power.

>>True, but it's evolving from merely a fallback, into a viable alternative. Sooner
>>or later system builders will realize they no longer need >a GPU or an IGP to create
>>a system that's still capable of adequate 3D support.
>
>Sure, but they will save power using an IGP...so for notebooks its a no-brainer.

Not necessarily. Opting for an IGP means accepting a less powerful CPU. I know plenty of people who would buy a laptop with a more powerful CPU instead, if it costed the same and the graphics were adequate for their everyday experience.

Seriously, let's analyze the case of the GMA 950. It sucks donkey balls (pardon my French). There's no excuse for buying it if you care even a little about graphics. Right? Despite having all odds against it, massive numbers of this IGP were sold in dual-core laptops, Mac Mini's, set-top boxes, etc. A lite version (!) of this architecture will even make it in lots of Smart TV's as part of the Intel CE 2110 SoC. Lots of products based on this chip are still to appear. Where does this commercial succes come from? It's cheap.

So people's expectation of adequate graphics can be very low. But Sandy Bridge actually offers more performance than these people need. Software rendering can fill the void, as it's even cheaper.

Let's do some estimates. A measly dual-core Sandy Bridge can achieve 450 points in 3DMark06 with SwiftShader, today. Making use of AVX and gather/scatter, we're probably looking at 1500+ points. GMA 950 scores 150 points, and can't run half of the tests.

>>Low-end does not mean mobile! The fact that mobile devices are getting GPUs does
>>not mean a single thing for the CPU-GPU convergence that is going on in the desktop
>>market. There's a giant market for office systems, which need good CPU performance,
>>but the cheapest possible graphics solution. Since 3D is coming to desktop applications
>>and the web, some level of 3D graphics support is still neccessary, but clearly
>>software rendering can offer the ideal compromise at the >lowest price.
>
>Where power doesn't matter, that is quite reasonable. But if you only pay an extra
>$5 for an IGP, it's probably worth it.

For a 500$ system with a 10% margin, it would increase profit by 10% to not include that IGP. So 5$ is a lot.

Furthermore, gather/scatter doesn't just help make software rendering viable, it helps a whole range of multimedia applications, including some that have not yet seen the light of day. So you'd be selling a very attractive system if you spent that 5$ on generic CPU technology instead.

I fully realize there are many challenges, but the CPU-GPU convergence is still going on and it's more likely to zip together from the low-end up than the other way around.

Cheers,

Nicolas