Nicolas Capens (nicolas.capens@gmail.com) on 1/26/11 wrote:
---------------------------
>Hi David!
>
>David Kanter (dkanter@realworldtech.com) on 1/25/11 wrote:
>---------------------------
>>Nicolas Capens (nicolas.capens@gmail.com) on 1/25/11 wrote:
>>---------------------------
>>>Hardware support for compressed textures is not critical to the viability of software
>>>rendering. In bandwidth-limited situations, it only helps by about 10%. Off course
>>>in the case of a GPU you want that 10% since every cent you pay has to count, but
>>>for a CPU running graphics plus every other application, >it's a different trade-off.
>>
>>I'm very skeptical. Texture compression should be much more effective than 10%.
>>Do you have any numbers to back this up?
>
>Doom 3 on a GTX 460 at 2560x1600 4xAA runs at 53 FPS at >Ultra High Detail, and
>at 56 FPS at High Detail. I was being generous when I said >10%.

That shows nothing about compression, that merely tells about the change in performance due to larger textures. It's also largely about an older game that isn't designed for 2560x1600.

What I'd want to see is for a number of MODERN games:

1. Texture size (uncompressed vs. compressed)
2. Bandwidth usage (uncompressed vs. compressed)

You're claiming that for #2 the difference is 10% and I don't see any real evidence of that. Compression should be vastly more effective.

>To clarify this result somewhat, here's the bandwidth usage of Unreal Tournament 2004 at 1024x768, using compressed textures: http://personales.ac.upc.edu/vmoya/img/bw.png
>
>Although bandwidth usage differs a lot between games, we can observe a few things
>which reduce the impact of texture compression. First of all, games aren't memory
>limited all the time. The memory bus is overdimensioned to prevent that from happening
>too frequently. It means that even if the bandwidth requirement would double, the
>performance would not reduce in half. Also note how color and Z each take a very
>significant amount of bandwidth. And the balance tips over >even further with deferred rendering.

I happen to know the author of that study in question. The data is INCREDIBLY OLD. It's from a simulator that did not use any Z or color compression, so the results cannot be taken seriously.

>All these effects combined result in texture compression >only helping roughly 10%.

Again, I see no evidence of that.

>Note that in the case of Doom 3, the main reason for not disabling texture compression
>was VRAM size. When textures had to be swapped in and out over the AGP or PCIe bus
>the performance hit was huge, but it wasn't a VRAM >bandwidth issue.

>For a GPU the texture decompression logic is worth it >mainly to keep VRAM size
>smaller (read: cheaper). The bandwidth savings are a >welcome bonus but I doubt it would be a justification by >itself.

I think you underestimate the cost of adding pins to your memory controller.

>>>That said, it's not at all impossible to add texture >decompression as a CPU instruction...
>>
>>Won't happen.
>
>That was my conclusion as well. They would never add instructions which could become
>of little or no use in several years.
>
>I merely mentioned it to note that there's nothing a GPU can do, which a CPU could
>not possibly do. Also, there are plenty of other alternatives, should it be necessary.
>And again, it doesn't have to match the efficiency of >dedicated hardware to be viable.

Actually for the mobile space it does have to come close to dedicated hardware. Battery life matters, a lot.

>Most likely the bandwidth will just steadily keep increasing, helping all (high
>bandwidth) applications equally. DDR3 is standard now accross the entire market,
>and it's evolving toward higher frequencies and lower voltage. Next up is DDR4,
>and if necessary the number of memory lanes can be >increased.

More memory lanes substantially increases cost, which is something that everyone wants to avoid.

>Either way I think
>there's plenty of bandwidth already for the market which >merely needs minimal graphics support.
>
>>>Exactly. But since software renderers have access to lots of system RAM, that's
>>>not really an issue. And the bandwidth issue is largely >compensated by the CPU's big caches.
>>
>>That's really only true if your textures and rendering algorithms are cache friendly.
>>My understanding is that most textures are not that cacheable.
>
>For the case of Crysis, the terrain and vegetation >frequently reuse the same textures.
>So this benefits quite a bit from the large L3 caches.

That's one game. How big are the textures anyway?

>And again CPU technology is not at a standstill. With >T-RAM just around the corner
>we're looking at 20+ MB of cache for mainstream CPUs in >the not too distant future.

T-RAM is not just around the corner.

>And while the expectations for 'adequate' graphics go up as well, it's only a slow
>moving target. First we saw the appearance of IGPs as an adequate solution for a
>large portion of the market, and now things are evolving >in favor of software rendering.

I think if you look at the improvement in IGPs, that's a very FAST improving target.

>>>SwiftShader runs 30% faster on a Sandy Bridge chip with >only 55% of the bandwidth.
>>>There is no clearer proof that software rendering isn't >bandwidth limited.
>>
>>That's only meaningful for swiftshader, not all SW rendering. How does it work
>>with MSAA and anisotropic filtering and other more advanced techniques?
>
>I don't see why it would only be "meaningful" for SwiftShader. The public demo
>does not take advantage of Sandy Bridge's new features. So >I expect other software
>renderer's to benefit roughly the same.

Swiftshader is swiftshader - other SW rendering systems work differently. They may (or may not) see similar benefits.

>I haven't had the chance yet to see how Sandy Bridge >performs with multi-sampling,
>but I'll keep you posted when I do. Anisotropic filtering >doesn't seem relevant
>to bandwidth, since all texels will be in cache due to >high temporal coherence.
>In fact it would make it even more gather/scatter limited.

>>Also, what is your comparison against? It sounds like you're saying Sandy Bridge
>>has 55% more memory bandwidth than whatever your baseline is...which sounds like
>>an awful lot. Sandy Bridge's memory controller isn't that much faster than Arrandale AFAIK.
>
>No, I said Sandy Bridge has only 55% of the bandwidth of my baseline, which is
>an i7 920 @ 3.2 GHz, using 1600 MHz memory and three channels. So if I'm not mistaken
>that's 1.85 times the memory bandwidth versus the i7 2600 >I tested. Despite that, the 2600 is 30% faster!

I don't know what an 7 920 is, but if it only has 4 cores, it cannot really utilize the memory bandwidth.

Also, note that there are big differences in the cache hierarchy.

Perhaps a more interesting question to ask is to take Sandy Bridge and start downclocking the memory and seeing what the performance degradation is. Comparing two totally different systems tells you very little about memory bandwidth.

>So again, software rendering is nowhere near being RAM >bandwidth limited. Just like many other compute intensive >applications, it needs gather/scatter before any other >parameter starts to matter. Note that gather/scatter >wouldn't just allow a massive reduction in the number of >serial load/store instructions, it would also reduce the number of swizzle operations: http://software.intel.com/en-us/forums/showpost.php?p=139770

Reducing the number of loads and stores isn't really relevant. It's the number of operations that matters. If you are gathering 16 different addresses, you are really doing 16 different load operations.

>>>The trend is that the number of texture accesses is going >down, relative to uncompressed
>>>unfiltered memory accesses.
>>>So the bandwidth savings is getting smaller. The reason
>>>for this is obvious: there's only so many textures you can >slap onto your scene's geometry.
>>
>>Then you just increase the geometry count, or use tessellation. It seems like
>>geometry would scale at the same rate (or maybe a bit slower) than texturing.
>
>With all due respect, you're sounding a bit like the >people who said dedicated
>vertex and pixel pipelines make more sense than a unified >architecture because "you just increase the geometry >count".

No, that's totally different. With unified shaders you can easily use more or less geometry...until you run out of physical shaders to execute on. You should look at Kayvon's work on micro-polygon rendering.

>It's clear that telling software developers what (not) to >do doesn't result in
>a succesful next generation hardware architecture. With >non-unified architectures,
>there were numerous applications which were vertex >processing limited, and numerous
>ones which were pixel processing limited. And even those >in the middle have a fluctuating workload.

Yes, except a unified shader architecture doesn't really preclude that many options.

>So instead hardware architectures should accomodate to the >increasing variety in
>workloads. This means generalizing the texture units into >gather/scatter units and
>performing filtering in the shader units.
>
>Anyway, you were probably still referring to bandwidth? Then increasing the amount
>of geometry in the scene won't tip the balance in favor of texture compression.
>With deferred shading or pre-z passes additional geometry only increases the color
>and/or z bandwidth needs (on top of the geometry bandwidth of course). Tesselation
>can make use of compressed textures, but note that it's sampled at a low frequency.
>Plus it massively increases the vertex processing workload, so in fact you're less
>likely to get bandwidth limited at all.

>>>At the same time, caches are still getting bigger and >bandwidth is still increasing.
>>
>>That's not the question. It's 'are caches increasing in >>size sufficiently faster than texturing data'.
>
>Absolutely. Ten years ago the L2 cache was 256 kB, nowadays we have 8 MB of L3
>cache. That's 32 times more. Games did not evolve from ~2 >to ~64 texture samples
>per pixel.
>Even if you count in the increased screen resolution and >additional scene
>complexity (which by the way also scale the computational >needs), the texture bandwidth
>needs did not increase 32-fold. Even for the high-end the >memory bandwidth has not
>increased by that much. And it increased far less in the >low-end graphics market.
>IGPs are focussing on increasing their computing power >more than their texturing abilities.

First, you do need to count the increases in resolution. Pixels have definitely increased over time. And TBH, I don't have any numbers on how fast the texture sizes have increased. I suspect quite a bit.

>In ten more years caches could be around 256 MB, and >that's without taking revolutionary
>new technologies like T-RAM into account. So it's really >hard to imagine that this
>won't suffice to compensate for the texture bandwidth >needs of the low-end graphics market.

Because you are imagining that the low-end market stays put. It won't.

>SwiftShader 1.0 was first used by a 2D casual game called Galapago. Despite the
>game's graphical simplicity, it was totally texture sampling limited and it was
>barely reaching the necessary 10 FPS for playability. That >was five years ago. Today we have Crysis running at 20+ >FPS.

What resolution and quality settings?

>I don't need any crystal ball to see that gather/scatter >will make IGPs redundant.

That's because it won't.

>>>There are even strong indications that T-RAM technology is >nearly ready for production.
>>>It combines the density of DRAM with the speed of SRAM.
>>
>>I know the guys at T-RAM. Their technology is very interesting, but it's not at
>>all clear that they are close to production in a CPU or GPU.
>
>http://www.businesswire.com/portal/site/home/permalink/?ndmViewId=news_view&newsId=20090518005181

That means nothing. It means that GF is investigating the technology, not that it's production ready.

>Anyway, there are multiple high density cache technologies. There's Thyristor-RAM,

That's T-RAM.

>1T-RAM,

Not a replacement for SRAM.

>2nd gen Z-RAM

Doesn't work at all.

>TTRAM, etc. So regadless of whether it takes one year, two
>years, three years... does it really matter? Cache technology is expected to exceed
>Moore's Law at some point.

By who? The only people who have made any serious attempt to use a new storage technology is IBM. But that's only in VERY EXPENSIVE chips that are produced in low volume. Not that GF does not share IBM's enthusiasm for eDRAM.

>Together with gather/scatter this helps the CPU compensate
>for its lack of dedicated texturing hardware, allowing it >to significantly gain
>on GPUs. So sooner or later software rendering is going to >take over the low-end graphics market.

It's possible, but they will need to become more competitive from an energy perspective with fixed function stuff.

>>>This means that sooner or
>>>later the majority of the data used in a frame will fit in >the cache, further reducing
>>>the relevance of texture compression.
>>
>>That depends on how fast the geo, vertex, texture, pixel, etc. data scale. Is
>>is faster or slower than cache size increases?
>
>Slower. And that's without factoring in the massive increase in computing power
>you need for processing all that data. And caches don't >need to store all data used
>in a frame, to already get high reuse.

>Seriously, there are more critical things to worry about >that determine the viability
>of software rendering. Currently a lot of cycles are >burned on getting the data
>into the right layout for the SIMD units to process it.

>>>We're not there yet, I realize that, but nothing is >stopping the steady convergence toward software rendering.
>>
>>Power efficiency?
>
>Power efficiency is obviously important, but you have to >think about what's possible
>within a certain power envelope. The acceptable TDP has >reached a ceiling, but that
>doesn't mean only the most power efficient solution is >acceptable. It didn't stop
>audio processing from becoming a software driver, even >when Pentium 4's were the least power efficient >architecture...

Audio consumes an insignificant number of cycles. Miniscule. Perhaps when graphics gets to that point, it will be fine to put it in SW.

And yes, power efficiency matters a lot. You may not think so, but it does.

>Sandy Bridge is designed to be able to use four cores plus an IGP, all within 95
>Watt. During gaming or other intensive applications, that's perfectly acceptable.
>As you know, Sandy Bridge is amazingly efficient. So it should be equally acceptable
>to trade the IGP for two more CPU cores, add gather/scatter capabilities, and as
>a result get a really powerful CPU which can also take >care of graphics, all within 95 Watt.

I don't think you design CPUs for high volume applications. Most don't need scatter/gather and the hardware cost is high.

>It will take several more generations for gather/scatter >support to arrive, so
>by that time we're on 22 or 16 nm so you get lower power, >more performance, and/or lower power consumption.

>By the way, currently the IGP part of Sandy Bridge is a >total waste of die space
>when using a discrete graphics card. Instead they could >have had a single 6-core
>architecture which does software rendering for people who >have low graphics needs,
>and it would be a more powerful CPU for people with a >graphics card.

There will be 6-8 core versions available later this year.

>>>But what's the next step? Obviously there's a quest for higher quality so it was
>>>only a matter of time before compressed HDR formats became available so they actually
>>>fit in VRAM. But we don't really want compression; it's just a temporary necessity
>>>for dedicated gaphics.
>>
>>You always want compression for something you store in memory. It increases effective
>>bandwidth and reduces capacity used. You might want higher fidelity or lossless
>>compression, but you'll still want it. Less memory traffic is lower power and fewer pins.
>>
>>>Once the bandwidth and VRAM goes up the purpose of >compression
>>>diminishes again. There's not much beyond this, and if >there is, it's not likely
>>>of interest to the markets that would show an interest in >software rendering first.
>>
>>Again, only sort of. There's always a benefit to compression.
>
>You don't unconditionally want it. It's still a trade-off. >For this reason lossless
>compression of system memory is hardly more than a >conceptual idea. Apparently it's
>cheaper to increase the bandwidth and capacity the brute >force way.

Really? Have you heard of Vertica? They do an awful lot of lossless compression of data in memory.

>>>To me it's very clear that the lack of parallel load/store >is a huge bottleneck.
>>>The ALU's can do 16 operations in parallel, but only 2 >load operations and 1 store
>>>operation can be executed per clock. And it gets worse >with FMA.
>>
>>Um, Sandy Bridge can do 32B/cycle of loads, that's 8 >values.
>
>It can load 2 x 4 *consecutive* 32-bit values (per core >per cycle). That's not
>the same as 8 values. As illustrated in the link I gave >above, that's worthless
>for something like a parallel table lookup.

Many applications use adjacent values.

A parallel lookup table has control divergence anyway, which is totally different.

>Gather/scatter would make a massive difference for a large >range of applications.
>It's the parallel equivalent of load/store so it would >allow parallelizing nearly
>every loop. It's the one missing thing that prevents auto-vectorization from becoming widely usable.


>>>Gather/scatter would speed up texture sampling, vertex >attribute fetch, primitive
>>>setup, rasterization, transcental functions, raster >operations. It's incredibly
>>>versatile and responsible for a large chunk of the >remaining CPU-GPU gap.
>>
>>That's part of it...but scatter/gather generally means you MISS in the cache/TLBs and chew up more bandwidth.
>
>Why? It only accesses the cache lines it needs. If all >elements are from the same
>cache line, it's as fast as accessing a single element.

And exactly as fast as using AVX! i.e. no improvement and more complexity/power.

>But even in the worst case
>it can't generate more misses or consume more bandwidth.

It sure can. Now instead of having 1-2 TLB accesses per cycle, you get 16. How many TLB copies do you want? How many misses in flight do you want to support?

>>Also, I'd point out that there is at least a 4X >>difference in memory bandwidth between CPUs and GPUs.
>
>So? IGPs are getting pretty powerful, while sharing the same bandwidth as the CPU.
>So with texture compression only having a marginal effect, >bandwidth is not a factor
>which would prevent software rendering from being viable.

You haven't convinced me that texture compression is marginal at all.

>Once the IGP has gone the way of the dodo, more powerful >CPUs can be created for
>people who want mid-end graphics performance. I bet Intel >would really love selling
>quad-channel 12-core CPUs not just to the server market but to gamers as well. Sandy
>Bridge already pretty much made the low-end discrete graphics cards obsolete, and
>Intel is not likely going to lose its grip on that massive >market.

I'm sure they'd love to, but they won't. Some things run better on dedicated hardware than not. It's possible some dedicated hardware will migrate into the core...but not guaranteed.

I think one of the points people are raising is that CPUs are generally latency oriented devices, whereas scatter/gather tends to be more useful in the context of throughput devices. That suggests that to some extent scatter/gather in the GPU (i.e. throughput device) is the right approach...instead of trying to make the CPU into a throughput device.

>Note that a 4 GHz 12-core CPU with FMA would be capable of 768 GFLOPS. But while
>that is achievable in the short term, it's still going to take longer to get all
>the pieces of the puzzle into place. So by then we're talking about several TFLOPS.
>At that point legacy rasterization isn't very exciting any >more, but the CPUs additional
>programmability capabilities will really allow software >developers to create something unique.

>>>So in five years or so people who don't need cutting-edge >graphics will have plenty
>>>of graphics power from their CPU cores.
>>
>>Yes, but they will have even more graphics power >>available through the integrated graphics.
>
>Which is then an expensive pointless piece of silicon for >those who are not interested in even more graphics power.

Not really. It's a more power efficient approach to graphics. And that matters. You may not believe it does, but using specialized hardware drives down energy/operation, which extends battery life.

>>>True, but it's evolving from merely a fallback, into a viable alternative. Sooner
>>>or later system builders will realize they no longer need >a GPU or an IGP to create
>>>a system that's still capable of adequate 3D support.
>>
>>Sure, but they will save power using an IGP...so for >>notebooks its a no-brainer.
>
>Not necessarily. Opting for an IGP means accepting a less >powerful CPU.

It doesn't though - it merely means fewer CPU cores. Look at Sandy Bridge for consumers versus the EP version. The difference is in the core count. You cannot simply afford to create different CPU cores for folks who want IGPs and those who don't. THat's bad economics.

>I know
>plenty of people who would buy a laptop with a more >powerful CPU instead, if it
>costed the same and the graphics were adequate for their >everyday experience.

I think you're missing the point. Right now, it appears like the trade-off is # of CPU cores vs. graphics.

>Seriously, let's analyze the case of the GMA 950. It sucks donkey balls (pardon
>my French). There's no excuse for buying it if you care >even a little about graphics.
>Right? Despite having all odds against it, massive numbers >of this IGP were sold
>in dual-core laptops, Mac Mini's, set-top boxes, etc. A >lite version (!) of this
>architecture will even make it in lots of Smart TV's as >part of the Intel CE 2110
>SoC. Lots of products based on this chip are still to >appear. Where does this commercial success come from? It's >cheap.

It's also power efficient. At this point, it's not clear that SW rendering is comparably efficient and I'd argue it's unlikely to be without specialized hardware.

>So people's expectation of adequate graphics can be very >low. But Sandy Bridge
>actually offers more performance than these people need. >Software rendering can fill the void, as it's even cheaper.

You're assuming that people's appetite for graphics stays constant, which isn't a good assumption.

>Let's do some estimates. A measly dual-core Sandy Bridge >can achieve 450 points
>in 3DMark06 with SwiftShader, today. Making use of AVX and >gather/scatter, we're
>probably looking at 1500+ points. GMA 950 scores 150 >points, and can't run half of the tests.

I don't have enough familiarity to assess whether these projections are realistic.

>>>Low-end does not mean mobile! The fact that mobile devices are getting GPUs does
>>>not mean a single thing for the CPU-GPU convergence that is going on in the desktop
>>>market. There's a giant market for office systems, which need good CPU performance,
>>>but the cheapest possible graphics solution. Since 3D is coming to desktop applications
>>>and the web, some level of 3D graphics support is still neccessary, but clearly
>>>software rendering can offer the ideal compromise at the >lowest price.
>>
>>Where power doesn't matter, that is quite reasonable. But if you only pay an extra
>>$5 for an IGP, it's probably worth it.
>
>For a 500$ system with a 10% margin, it would increase >profit by 10% to not include that IGP. So 5$ is a lot.

Yes, but you can charge more for the system since it gets better battery life.

>Furthermore, gather/scatter doesn't just help make >software rendering viable, it
>helps a whole range of multimedia applications, including >some that have not yet
>seen the light of day. So you'd be selling a very >attractive system if you spent
>that 5$ on generic CPU technology instead.

I totally agree that scatter/gather is a great capability to have. But what's the cost in die area, power and complexity? Not just to the core, but also the memory controller, etc.

>I fully realize there are many challenges, but the CPU-GPU >convergence is still
>going on and it's more likely to zip together from the >low-end up than the other way around.

In that context, why wouldn't you use the shader arrays in the IGP? Seems like you could get really good performance out of them, especially since they have some neat data formatting/conversion tricks.

David