Nicolas Capens (nicolas.capens@gmail.com) on 1/12/11 wrote:
---------------------------
>For the record, I have an i7 920 clocked at 965 speeds. As you certainly know,
>CPUs get ridiculously more expensive for a small increase in performance. But it
>also works the other way around! You don't have to lose a lot of performance when
>you opt for a cheaper CPU. Plus, any system needs a CPU anyway, so you got to take
>that into account when comparing CPUs against GPUs. The latter still needs a CPU or it's worthless...

Honestly, I still have to see a system - outside of embedded applications - that doesn't have a GPU. And no, if you go low in the CPU scale you lose a lot of performance. You have a high-clocked quad-core, how many of those end up in laptops? Cheap OEM PCs? CULV laptops? How fast would SwiftShader run 3DMark06 on a dual-core 1.4GHz CULV processor? Here's my guess: much slower than the pitiful IGPs you get with those machines.

>This weekend I had the opportunity to run SwiftShader on an i7 2600. It scored
>820 3DMark06 points at stock speed. So that's 32% faster than my overclocked 920!
>What's more, the 2600 is considered a mainstream CPU and it's price is expected
>to drop fast once AMD launches Bulldozer. And it comes with a ridiculously small
>heatsink. Note that the GMA X4500, still sold in massive numbers, only scores about 950 3DMark06 points.
>
>So while I won't deny there's still a gap in performance and power consumption, it's getting smaller every generation!
>
>But that's not all. Sandy Bridge wastes die space on a GPU. It could have had two
>extra cores instead, exceeding the X4500's performance. Furthermore, SwiftShader
>doesn't take advantage of AVX yet. And with gather/scatter support lots of graphics
>operations would become a whole lote more efficient.

Well, Sandy Bridge IGP, which is a really nice surprise coming from Intel, scores over 4k in 3DMark06, and that's with early drivers:

http://nl.hardware.info/reviews/1947/45/intel-core-i7-2600k-i5-2500k-i5-2300-sandy-bridge-review-gpu-benchmarks

That's 10 times what you'd achieve with SwiftShader on those two cores. That does look like a very good compromise to me, not a waste especially since it's also using less power to do so. And 3DMark06 is a basically CPU-less load, with many games you wouldn't have that luxury, StarCraft 2 for example which is pretty light for todays standards burns you 2-3 cores just for IA and physics. So no, putting two additional cores on Sandy Bridge instead of an IGP would have been a terrible idea for 99% of the users. Having a decent IGP on the other hands enables them to play games, Oh, and removing the IGP would also means doing video decoding on the CPU which would have shortened battery life when you're on the move, etc...

>So it's already well within reach to have a GPU-less system and still have ample
>3D graphics performance for the same market that APUs target. Last but not least,
>with extra generic CPU cores developers can create other diverse and demanding applications.

Honestly, a 6-core SB using software rendering wouldn't be able to play games with entry-level settings, a 4-core SB as it is does and offers a plethora of other nice, low-power features when dealing with graphics.

>Actually I believe AMD's GPUs have a higher transistor count, on a smaller die.

No, they have higher density but lower transistor counts, that's a design choice.

>This is probably due to NVIDIA's architecture using a higher clocked shader domain,
>requiring faster and thus larger transistors. Furthermore, depending on the exact
>chips you're comparing, NVIDIA has higher double-precision performance. Together
>with the new Fermi capabilities this means NVIDIA is way ahead of AMD in the HPC
>market. It's obvious this comes at a bit of an extra die size cost.
>
>But clearly that difference is insignificant in comparison to what I pointed out:
>It achieves the same actual performance with half the GFLOPS! The importance of
>this is that efficiency is critical, even for graphics, which was previously regarded
>as "ridiculously parallel".

You seem to have completely missed the point: AMD GPUs use VLIW processors. VLIW has been since its inception a design methodology for processors that trades off complexity for higher maximum throughput. VLIW processors will have lower utilization *by design* compared to a RISC approach, but they will make up for it by offering higher maximum throughput and compute density because they are simpler and thus smaller to implement. The low utilization of AMD GPU's is a *design choice* which actually allowed them to hit the same performance targets with a smaller die.

>And since there are lots of GPGPU applications which
>don't even achieve 10% of the theoretical performance of the GPU, while they use
>the CPU to it's fullest, it's clear that CPUs are really efficient at juggling tasks
>around and keeping the data flowing.

We already knew that, but that doesn't change a thing for graphics.

>So how AMD and NVIDIA compare today is pretty irrelevant to the discussion.

You were the one to bring this argument to the topic, not me.

>What
>matters in the long term is that applications, including graphics applications,
>are getting more complex and a CPU architecture is more suited for this.

Then why GPUs are becoming more and more prevalent? First it was only desktop computers, now it's desktop and laptops, now you get them even in DVRs, cellphones, tablets, you name it. If they are on-die or not doesn't change a basic fact: if you want to do graphics at decent performance level and in a decent power envelope you need dedicated hardware. Complex, programmable dedicated hardware but still dedicated.

>Note that software rendering frees game engine developers from the graphics API restrictions, which in turn results in higher performance in practice: http://graphics.cs.williams.edu/archive/SweeneyHPG2009/TimHPG2009.pdf

I had already read that presentation and while Tim Sweeny has already predicted the demise of GPUs many times GPUs have been entering new markets where they were inexistent in the past. So no, this is a purely theoretical argument that doesn't only happen in practice: it's also being proven wrong by the fact that we're getting *more* dedicated hardware, not less.

>Retirement buffers are pretty small. But instead of speculative execution, GPUs
>opt for massive simultaneous multi-threading, which means they need register space
>for all these extra threads. Shaders which use more registers than what the hardware
>was designed for, can really decimate performance. Developers also want a true call
>stack, so you need massive caches to store all this context.
>This situation is not sustainable. Some really drastic measures need to be taken
>to reduce the thread count.
> But you don't need to look any further than CPU architectures.
>Speculative execution, branch prediction, forwarding, register renaming, etc. can
>come to the rescue.

GPU caches are smaller than CPU caches, how's that comes? GPU register files are pretty large, but very dense, do you think that a ROB is a small/cheap structure? Or a branch predictor history table? Or OoO logic? They're not and for throughput-oriented workloads they do not make much sense. How comes thread count on CPUs is going up? Besides you are the first person I hear saying that current GPU trends are not sustainable, if it came from a GPU HW designer it might make sense but from you it sounds like wishful thinking.

> But when GPUs sacrifice computing density for higher efficiency
>at complex generic tasks, it's obviously also a really interesting option to just
>start with a multi-core CPU architecture and give it powerful graphics capabilities by adding gather/scatter.

Yes, they tried that, it didn't work out.

>While texture sampling is a highly specialized operation (it's the only thing Larrabee
>really has dedicated hardware for), the general trend is still to generalize them
>into load/store (gather/scatter) operations.

No, the general trend is to have more, more powerful texture units in GPUs. Look for yourself at the latest architecture from *any* graphics vendor if you don't believe me.

>There are several reasons for this:
>
>Shaders need ever fewer texture samples (TEX:ALU ratio). So the amount of die area
>use on texture samplers has steadily decreased. But this means that TEX heavy shaders
>become bottlenecked, and for ALU heavy shaders they are underutilized...
>
>But while the TEX:ALU ratio decreases, the shaders do make more unfiltered memory
>accesses. So modern GPUs also have specialized access to local memory, shared memory,
>global memory, etc. Each of these can again be a bottleneck!

Actually they're no bottleneck at all, they are part of the specialized hardware that makes GPUs so fast at graphics. They have tremendous bandwidth, they are optimized for texture access patterns and they are usually fully associative. No matter how you slice it, TEX:ALU ratio has decreased and still TUs have increased in number and functionality. They are not going away, they are getting better and they're both faster and more efficient at doing their job as a CPU will ever be. On top of that they use less space on the die.

>Also, filtering is useless for GPGPU, while graphics on the other hand wants full
>FP32 filtering (possibly even FP64). These diverging needs are hard to combine into
>texture units or other highly specialized load/store units.

All recent GPUs can do full-speed FP32 filtering, and filtering is very important for GPGPU too because many GPGPU applications are actually doing graphics and so they can also benefit from texture hardware.

>So it becomes very tempting to just slap all these different forms of memory access
>together, have generic load/store units, and let the cache hierarchy take care of local and temporal coherence.

Tempting for who? All hardware available now or in the design pipe is going in the other direction, how do you fail to see this basic, hard fact?

>As for compressed textures, they're not as critical as they may seem. Due to licensing
>issues, popular S3TC based formats are still not part of core OpenGL.

Compressed textures have been in core OpenGL since version 1.3. Specific compression techniques are left to the implementer, better get your facts straight.

>Loads of applications don't bother using them.

Name a relevant application - especially games - that is not using compressed textures at all.

>And low-end and mid-end hardware have higher bandwith:computing ratios so it's not that helpful.
>
>Where compression does help significantly is to fit everything in VRAM.

Absolutely untrue, on low-end hardware you *need* the bandwidth savings from compressed textures even more than in the high-end. Especially on IGPs where VRAM constraints are not there since they're using system memory.

>Ironically people with high-end hardware tend not
>to like the quality loss of texture compression so they sometimes disable it.

How do they do that? Well, I'm gonna tell you: they don't unless the application supports it. That's because assets in games are shipped in compressed form, so there's no way to "turn it off". Most high-quality assets are actually higher-resolution compressed textures.

>The general trent is toward higher quality.

Yeah, there's new compression standards coming out for that purpose:

http://www.nvidia.com/content/nvision2008/tech_presentations/Game_Developer_Track/NVISION08-Direct3D_11_Overview.pdf

Check out BC6H and BC7.

>But for software rendering that's far less of an issue
>since you typically have about four times more system RAM. Note also that an i7
>2600 has lower memory bandwidth than an i7 920 yet it's significantly faster so
>software rendering isn't bandwidth limited.

Yes, because it's much slower than hardware-accelerated rendering and you're hitting a different bottleneck: processing power. Let's put it this way, when doing rendering on the CPU you're wasting so many cycles on calculations that you're never gonna be bandwidth limited. That's why adding scatter/gather will not make CPU-based rendering competitive. Look at it from another angle: SB IGP's ALU throughput is *less* than the CPU's ALU throughput while still being at least 5 times faster in 3DMark06 according to your numbers.

>Why not try it instead of "basing" yourself on the 3DMark06 score? I have yet to
>find a game in Steam's casual section which doesn't run fine with SwiftShader on
>a quad-core CPU. Some even run smoothly with anti-aliasing.
>3DMark06 is a synthetic
>stress test designed to benchmark the hardware at the time of release, and still
>offer a challenge to future graphics solutions. Even today's casual games are nowhere
>near as complex as 3DMark06. And that's not likely to change fast. Causal games
>are designed to run on the lowest common denominator, which includes older IGP's like the X3100.

Pointers? I'm playing Defense Grid right now, pretty light game. If I try it on my laptop with its measly Radone HD2400 it can run at 1024x768 with all detail turned off. It's far from pretty and still dog slow but it runs, what about running it with SwiftShader on the processor, a C2D? Defense Grid is totally unplayable on an X3100. And that's a casual games which is already two years old. If by casual games you mean 2D games using 3D hardware then that's another story but honestly those run well *everywhere*.

>For your information, SwiftShader will be used in future versions of Flash to enable 3D casual games on the web, when you don't have adequate graphics hardware: http://blogs.adobe.com/flashplatform/2011/01/digging-more-into-the-molehill-apis.html

Well that's nice to hear, but offering a fallback for incompatible hardware or drivers doesn't mean that software shading is competitive with IGPs, let alone threatening them.

>But that's one outer end of the application area. You appear to be stuck thinking about what's possible with today's systems. Sandy Bridge's performance per Watt is already double that of the previous generation: http://www.pcper.com/article.php?aid=1057&type=expert&pid=12.
>> Now imagine that in two more generations we will have four times higher performance
>per watt, and gather/scatter support, and tell me again it will never be viable
>to have GPU-less systems even for the low-end market.

The lowest end market with the tightest space, power and area constraints - mobile phone - is going from GPU-less to GPU-enabled. You keep ignoring the very basic fact that we're getting more GPUs and faster ones precisely in the low-end. They're not going away, they offer better performance at lower price and power, no scatter/gather instruction will ever make it different.

>Even if it doesn't happen in four years, it's still bound to happen one day. Every
>single parameter is evolving in favor of software rendering.

Power constraints - the main driver in computer evolution lately - heavily favors hardware accelerators. The market has been moving in that direction faster and faster, it's the other way around, no matter how many times you repeat it, reality is going in the opposite direction.

>Systems with fixed functionality, ask for dedicated hardware. No argument there.
>But consumer systems can take on many roles. Graphics is increasingly becoming less
>of a focus. Once you can run a wide range of casual games (and beyond) on the CPU,
>which is there anyway, there's very little demand for a piece of dedicated hardware.

Then care to explain why every piece of hardware - including CPUs - are getting *more* dedicated hardware. Maybe you missed it but besides an IGP SB got a video decoder and encoder. If there's so little demand as you suggest then why every other vendor is providing *more* dedicated hardware?

>Let me put it this way: Professional DJ's will always buy dedicated sound cards.
>But nowadays it's clear that's a worthless argument against audio codecs for the
>consumer market. Likewise, mobile phones may have dedicated DSPs for sound today,
>but with an ever wider range of applications a powerful CPU is needed and so in
>several years it will be a lot cheaper to have an audio codec for these power-constrained devices as well.
>
>Dual-core CPUs for phones have only barely started to hit the market, but much
>against some people's expectations they're more power efficient. Two cores at 800
>MHz consume less power during everyday use than one core at 1000 MHz, and can deliver
>higher peak performance when it matters. It's the MHz race to multi-core revolution
>all over again. Multi-core wins at performance per Watt.

And dedicated hardware wins over multi-core. Low-end mobile phone SoCs do not have dedicated hardware for audio, not high-end ones, and guess what: because they're cheap. It's the other way around, just look at those hardware documentation, there's not a single SoC in the market that has been losing functionality, they're gaining more and more.

>Clarkdale/Arrandale didn't have an on-die memory controller like Nehalem, but was
>connected to the GMCH with a QPI link. Nehalem didn't have a GPU. So with Sandy
>Bridge for the first time the CPU and GPU use the same memory hierarchy.

And that's all, they're not even coherent.

> Let me
>put it this way; the (logical) distance between two CPU cores is the same as the
>distance between a CPU core and a GPU core. You could say they do exactly the same
>thing: processing data. For as second, think of Sandy Bridge's GPU as a CPU core
>with a graphics-oriented instruction set.
>
>That's a huge leap toward fully uniform CPUs doing software rendering. At a hardware
>level it may only seem like juggling things around, but it's a new milestone at
>a conceptual level. Previously the GPU was considered a device you send some data
>and some graphics commands and it did its magic. Fire and forget. It may as well
>have been on a different planet. But with Sandy Bridge these heterogenous cores are sitting side by side.

It's not, it has it's own control mechanism like any other GPU. But if you like to think about it this way then not only GPUs but many, many hardware accelerators are actuall CPU cores with domain-specific instructions. Tensilica cores for example are very popular as a basis for hardware accelerators, most video decoders and possibly encoders are based on them including AMD's ones IIRC. And you still send commands to them like you did before. Nothing really changed at that level. They are so much heterogeneous that if you put in a discrete graphics card you also loose the video decode/encode functionality.

>It doesn't consume anywhere near 100 times as much power. Buy yourself an i7 2600
>and measure the wall socket power consumption while running Crysis with SwiftShader.
>It should be about 150 Watt. Next, find a system with the same performance in Crysis,
>but using the GPU. The closest thing I can think of is an ASUS N10 netbook, with
>a GeForce 9300M GS (http://www.youtube.com/watch?v=GqzhtI_xhPY). It consumes up
>to 30 Watts, so that's only a 5x difference.
>So while today there is still a gap, it's not nearly as big as you think it is.
>You really have to keep in mind that GPUs are worthless without a CPU and the rest of the system.

Yeah, right. I suppose that taking into account the screen, drive and other components makes a very good comparison of the relative power required for rendering. Honestly, this is the biggest strawman of your whole argumentation. But let's get this right with a quick test:

http://techreport.com/articles.x/20088/7

SC2 is doing ~50FPS on a GTX570, consuming ~200W at the wall for the graphics card alone. All detail is turned on and the game is running at 2560x1600 with 4xAA and 16xAF. If this 5X number you completely made was true it means that on a ~100W CPU you should be able to run that benchmark using SwiftShader at ~5FPS. Want to give it a try?

>And once again, it's only going to get better. Gather/scatter and a complete set
>of integer AVX instructions will considerably improve the performance. Also note
>that not nearly as many man-hours have been spent on SwiftShader compared to the
>development of GPUs and their drivers, so there's still room for some improvement.
>And finally, games are getting ever more complex so future GPUs will need more registers
>and caches and such so they'll burn more Watts to keep the efficiency up.
>
>So this 5x gap is going to close from both ends. It's still considerable, but note
>that software rendering is very cheap in comparison so even today there's already a market for it.

Just like CPUs, your argument works both ways. And there's no such thing as a 5x gap, you made up that number out of thin air, and it's not shrinking, GPUs have become more power-efficient as time went by just as CPUs did.

>Wrong: http://www.youtube.com/watch?v=4bITAdWvMXE

Oh. An Intel sponsored video. How convincing, now what about this:

http://www.cad.zju.edu.cn/home/rwang/publication/09GPUGI.pdf

Or this, offline rendering, GPU-based software shipping now:

http://www.studiogpu.com/
http://furryball.aaa-studio.eu/
http://www.mentalimages.com/products/iray.html

>Deferred rendering and virtual texturing are nice techniques but they're not silver
>bullets by any means. The number of ALUs still grows and despite these techniques
>today's games use more API calls per frame than those five years ago. So I wouldn't
>say that DLP is becoming less and less of a problem at all. Like I said, the situation
>is stagnating (at best), and to achieve higher performance it's useful to start
>looking at more than just data level parallelism.

Compared to the amount of data on the screen current games are using way less calls per instance than in the past years.

>The writing on the wall is that both NVIDIA and AMD have released architectures
>capable of task level parallelism. Clearly they don't trust DLP enough for the foreseeable
>future and are willing to invest transistors in logic to exploit other types of parallelism.

Yeah, that's a nice feature which happens not to be used for graphics.

>The causes for Larrabee's demise are not an argument against the viability of software
>rendering on the CPU. Here's why: Larrabee is an expensive piece of additional hardware
>they attempted to position against fierce competition in the high-end market. In
>the end what really killed Larrabee is performance per dollar. For Larrabee single
>digit percentages mattered a lot. So big investments into brilliant architectural
>changes which only start to pay off in five years when developers take advantage
>of the new possibilities, are a big no-no.

Look, it wasn't single-digit percentages. Not by a large shot. You're into this market, ask around, you can probably get information from first-hand sources about what happened so you know for yourself.

>Again, systems with a single purpose ask for dedicated hardware. It's not an argument
>against audio codecs for the consumer market, and it's not an argument against software rendering.

Yes it is since GPUs have been breeding in each and every CPU, mobile, tablet or DVR SoC. We're getting *more* hardware accelerators, not less, deal with it.

>A heterogenous architecture is generally considered to use separate instructions
>streams with a different ISA. You use an API to communicate with the slave components.
>So the FPU is not a heterogenous component, except maybe in the early days when
>it was a separate chip you could consider x86 and x87 to be different ISAs and the
>instruction steam is separated by the ESC prefix (which caused an interrupt which
>you could consider a very low level API call). But regardless of where you put the
>dividing line it proves my point that it became fully homogenous.

OK, so answer this question. I have this SoC which has four CPU cores, they all use the same ISA, two of them can run an OS and run user programs, two of them are part of hardware accelerators and are accessible only through an API (and have dedicated memories, etc...). I would call this - common - case heterogeneous in spite of all the cores sharing the same ISA. The definition is not as clear cut as you make it.

>A customized CPU is not a heterogenous architecture either.
>
>And that's the beauty of it. Just because we call something a CPU doesn't mean
>we can't give it a customized instructions set to make it efficient at processing graphics!
>
>You could even add complete texture sampling instructions. You may even find some
>use for them outside of graphics. But personally I don't think this is a good idea.
>As I detailed above things are evolving toward generic gather/scatter operations
>anyway, which are a lot more useful outside of graphics as well.

Then add a few dedicated caches for texture access, and you have a GPU! That's how GPUs are made, they are customized CPUs with customized instruction sets and "weird" execution units that actually deal with textures, rasterization and such things. And that holds true for many other accelerators, as I mentioned many video decoders/encoders are customized Tensilica cores. Wake up, what you're talking about is already here, hidden behind an API!

>It does go away. We used to have vs_1_1 and ps_1_3 instruction sets which were
>physically implemented in separate vertex and pixel processing pipelines.
>
> [snip]
>
>They're
>on the same die today, using the same memory subsystem, you'd be removing a balancing
>bottleneck, you'd enable the CPU to run other existing and new/unknown compute intensive
>applications, and as a graphics processor you'd gain endless possibilities...

OK, so, vertex and pixel shaders were merged, they were very different beasts. Basically both doing FP32 calculations. And this is a strong proof that GPUs will be replaced by CPUs. On the other hand GPUs are popping up in every and other device that used to not have them, and this fact can be discarded as meaningless. Honestly do you believe your own argumentation?

>There's an article on this very site which shows that offloading anything other than graphics to modest GPUs is a bad idea and you need to cheat to make it look even remotely interesting: http://www.realworldtech.com/page.cfm?ArticleID=RWT070510142143
>Also, what sort of content creation are you talking about? If it's graphics realated, it's not GPGPU.

Yeah, that's why the commercial product I've mentioned above are being sold, because they're cheating. Right. Anyway everything that is not rasterization _is_ GPGPU by definition. And it's getting more and more traction, now render farms are being built with GPUs in the mix for this specific reason and that's a field that was for a long time CPU-only.

http://www.theregister.co.uk/2011/01/10/datacentre_cooling_and_power_constraints/

I snipped the rest of your post because you just repeat baseless predictions that say that CPUs will somehow get better why GPUs stagnate. No current trend supports it so unless you come up with something better it's just your prediction against the real world. I pick the latter.