Hi David,

>>>>David Kanter (dkanter@realworldtech.com) on 2/7/11 wrote:
>>>>---------------------------
>>>You are talking about eliminating throughput cores. You claim to have a basic
>>>understanding of hardware, so it should be readily apparent how CPU cores and throughput
>>>cores (e.g. Niagara, GPU shaders) differ.
>>
>>Throughput-oriented is a term which originates from server systems, long before
>>graphics chips were even called GPUs! It merely means a focus on data rate, potentially
>>at the cost of latency. Any multiprocessor system, is a throughput oriented system.
>>Clock frequency and ILP are latency-oriented, while DLP and TLP are throughput-oriented.
>>
>>Today's x86 CPUs exploit DLP and TLP (SIMD, multi-core and Hyper-Threading) and
>>have a less aggressive clocking than several years ago.
>
>Actually the clock speeds are about the same 3-4GHz.

Indeed. You appear to have missed the word "aggressive". At 90 nm, 3.8 GHz was really pushing it. Actually, the Pentium 4 had an integer core at twice the frequency. It required very short pipeline stages and Low-Voltage Swing circuits, which takes a lot of extra transistors. So at 32 nm, 3.8 GHz is a breeze.

>>So they are definitely throughput-oriented
>>architectures already and would become more efficient at with the addition of gather/scatter
>>support. Parallel load/store is the main thing setting them apart from GPUs.
>
>That's simply false. There are many more differences in terms of circuit design
>and the latency of individual instructions.

Sure, but those things don't make it not thoughput-oriented.

>>GPUs are, in the words of NVIDIA's chief scientists, "aggressively throughput-oriented
>>processors". Note though that GF104 features superscalar >execution, intended to
>>lower the latency. And aside from reducing bandwidth, caches also reduce latency.
>>So GPUs are forces to become less aggressive at using throughput-oriented techniques,
>>because reducing latency somewhat also reduces the amount of on-chip storage you
>>need. It's a balancing act, because obviously reducing >latency costs transistors as well.
>
>The two architectures are leagues apart in terms of latency? What is the latency
>of a dependent chain of adds in a GPU? What about a CPU? What is the branch coherence? What is the memory latency?

The latency of a multiply-add on GF114 is 18 cycles I believe. On Sandy Bridge it would take 8 cycles. That's roughly a 2x difference, or 4x in absolute time.

Note though that on GT200 it was 24 cycles, so there's some convergence taking place.

Furthermore, the latency of the CPU is based on making use of argument forwarding, to bypass the register file. On a GPU the latency includes accessing the register file, which for AMD's architecture takes half of the total latency (base on what you wrote about AMD Cayman here: http://www.realworldtech.com/page.cfm?ArticleID=RWT121410213827&p=5. What this means is that CPUs are not pushing their ALUs to perform 4 times faster than a GPU ALU. Note also that an AMD shader core is a fairly complex VLIW core which can do more than a MAD, while the CPU's multiply+add latency is based on just the scalar latency for a multiply and an add. It leaves the pipeline as soon as the result is ready. So even in terms of absolute latency, CPUs are not pushing their execution units.

Last but not least, note that using a forwarding network is a design choice which increases the area of the execution pipeline, but decreases the register file size! In conclusion, CPUs are indeed latency optimized, but that doesn't mean it can't be a throughput oriented architecture.

>The memory latency makes this especially obvious, as there is a huge difference.

There's indeed a big difference, but it has to be compensated using additional on-die storage, which itself lowers compute density.

So again, CPUs can achieve practically the same compute density, while at the same time being latency optimized too. Note that with FMA, the CPU cores would have a 2x compute density *advantage* over the IGP. Of course with software rendering you need that to implement the fixed-funtion operations, but in combination with the advantage of not being bottlenecked by any dedicated components it's clear that software rendering is viable.

And once more the conclusion is that modern CPUs are compute oriented. The GPU is still "aggressively" compute oriented, but that's slowly changing as this aggressiveness lowers effective efficiency for complex workloads.

>>>Even to someone without circuit design
>>>expertise, it should be blinding obvious - the clock speeds are about a factor of 2-4X different.
>>
>>Then why did NVIDIA decide to put its aggressively throughput-oriented cores into
>>a higher clock domain? The GeForce GTX 560 Ti has a shader clock of 1645 MHz, while
>>the Radeon HD 6950 has a clock of 800 MHz. Does that mean >NVIDIA's architecture is not throughput-oriented?
>
>1.5GHz is still half the speed or less of a modern CPU. AMD has GPUs that run
>at 900MHz. And those speeds have stayed constant for the last several generations (since about 65nm).

You missed the point. There's a design space for compute oriented devices, which includes a wide range of clock frequencies (3.2 GHz for Cell BE at 90 nm). So you can't conclude that modern x86 CPUs aren't compute oriented, based on their clock frequency (or latency).

Unless of course you want to imply that NVIDIA's GPUs are any less of a GPU due to their higher clock frequency?

Seriously, whether you like it or not modern CPUs are compute-oriented. And they're about to become more compute-oriented with FMA, and more efficient at it with gather/scatter, while GPUs become less compute-oriented (and in the case of AMD, less effient at it due to bottlenecks causing low utilization, unless they change course).

>>Clock speed alone isn't an indication of being throughput-oriented or not. It's
>>a design decision which doesn't have to compromise *effective* throughput, as proven
>>by NVIDIA vs. AMD. Another example is Cell BE, which clocks at 3.2 GHz but even
>>at 90 nm was considered strongly thoughput-oriented. x86 CPUs have come a long way
>>since the single-core Pentium 4 days, and they're not about to stop increasing throughput
>>efficiency (AVX, Bulldozer, FMA, etc.).
>
>Listen, no matter what you say, CPUs are still optimized for latency. It's blatantly
>obvious if you have ever written code for a CPU. It's also blatantly obvious that
>GPUs are not optimized for latency. This means different circuits, different architectures,
>pipeline arrangements, etc. etc. and most importantly a different style of programming.

I never denied CPUs are latency optimized! But they're no longer "aggressive" about it (i.e. they don't sacrifice everything else for the sake of latency reduction). The MHz-race is over. It has made way for a focus on effective performance/Watt, leading to a healthy balance between latency and throughput optimization.

Yes the design decisions are still different from GPUs, but it's converging, and even today it's already well within the design space of throughput-oriented architectures. Except for the "aggressive" varieties, it's orthogonal to being latency-oriented.

And no, it doesn't result in a different style of programming. SwiftShader takes shaders (GPU programming style) and compiles them into a sequence of vector instructions. It's abstracted from the application developer, in the same way the GPUs driver and hardware abstract the parallelism. The only thing that's missing to make the translation from a throughput-oriented programming language into explicit vector instructions more effective, is gather/scatter. But this happens within the abstraction layer. CPUs already support the programming model without it.

>Consider the performance degradation of running a latency sensitive workload (e.g.
>SPECint) on a GPU vs. a CPU. It's going to be huge.

Still not proving that CPUs aren't throughput-oriented.

>>As a matter of fact the CPU's clock frequency has remained nearly constant. The
>>i7-2600K has a 3.8 GHz Turbo Mode ( but under multi-threaded workloads it's not
>>that high), the same as the 2004 Pentium 4 Prescott. In the same time period, NVIDIA's
>>GPUs have increased their clock frequency by over a factor 3. There's no indication
>>this is going to change. For NVIDIA to conquer the HPC market, it needs to continue
>>investing into latency reduction. To prevent an excessive growth in die size, it
>>needs to increase the clock frequency. GF100 had some thermal issues, but they got
>>that under control with GF110, which has more cores >enabled and even higher clocks.
>
>Again, frequency is just one aspect. There are so many other things to consider.
>Pipeline depth, result forwarding, dependent instruction latency, etc. etc.

Yep, covered all of that. Still the conclusion is that aside from being latency optimized, CPUs are also thoughput optimized. One doesn't cancel the other if they're not aggressive about it.

>>So while it's "blinding obvious" that there's a clock frequency difference today,
>>it's also "blinding obvious" they're on a collision >course. Gather/scatter support
>>is still several years out, so by that time they'll have >converged even closer and gather/scatter is the keystone.
>
>Scatter/gather is helpful, but it will not make CPUs as efficient as GPUs. It
>doesn't help make the ALUs lower power. It doesn't help make the front-end lower power.

Higher utilization does improve power consumption of the rest of the pipeline. Idle logic consumes power as well.

Furthermore, you have to take into account that gather/scatter involves only the load/store units, while a sequence of extract/insert instructions involves both the load/store units and ALUs pipelines. With gather/scatter these pipelines become available for more useful work, improving throughput beyond what would be achieved if gather/scatter was an ALU operation!

Emulating gather/scatter involves 2.25 instructions per element. That's a lot of data moving though the pipelines just to exchange one element. Gather/scatter mainly only needs a small network to move the elements where you want them, and nothing else. I'm sure this is more power efficient than the 2.25 instructions that need to pass through the entire pipeline (which involves shuffle units as well). And on top of that it frees up an ALU. You could use this ALU to achieve higher thoughput (which itself consumes more power), or you could simplify the execution units (e.g. toss out the duplicate blend unit) and use the freed up cycle to achieve the same total throughput, keeping power consumption the same.

And it doesn't stop there. There's already people complaining about the fact that Sandy Bridge's dual load units can't load two 256-bit words, making AVX code more L1 cache bandwidth limited than SSE code. This is a problem which is going to get worse with FMA. Which is why FMA support was delayed till Haswell (i.e. when the transistor budget becomes available and applications which use AVX are around). Haswell will double the width of all load/store units. But since dual 128-bit gather/scatter suffices, a large part of the logic can overlap. This means that even when you're not using gather/scatter, the logic isn't a waste of transistors.

So any way you look at it, gather/scatter is an absolute improvement in power efficiency for throughput computing.

>Look at what CPUs have that GPUs do not:
>Bypass networks
>Branch predictors
>Prefetchers
>Out-of-order execution
>Low latency instructions
>
>etc. etc.
>
>The differences at the circuit level are just as big.

I've already shown that being throughput-oriented is a far more global property than any of those things could change.

Regarding prefetching, Fermi already has CUDA prefetch instructions, which are proven to be useful: http://comparch.gatech.edu/hparch/nvidia_kickoff_2010_kim.pdf. However, it takes additional instructions to calculate the fetch address and issue the prefetch. Here's an entire paper on the subject: http://arch.ece.gatech.edu/pub/asplos15.pdf. But note that successful prefetching makes a latency bound workload compute bound again. So it's better not to waste these cycles and use a small, conservative hardware prefetcher instead, which generically and automatically helps everything that could benefit from prefetch. Simple stream prefetchers have been around in CPUs for ages, so I'm sure it's a tiny amount of logic (in comparison to other solutions).

With increasing RAM latencies and increasing workload complexity, including for graphics, it's only a matter of time before GPUs feature speculative prefetching logic. Prefetching makes a throughput-oriented architecture a *more* succesful throughput-oriented architecture.

>>>You cannot simply tack on scatter/gather to a latency optimized CPU core and expect
>>>it to look like a throughput core in terms of power efficiency. At least, there
>>>is definitely a lack of evidence for any such claims. Moreover, you need to preserve
>>>the power efficiency for workloads that cannot be vectorized.
>>
>>An architecture which balances latency and theoretical throughput, can still achieve
>>high effective thoughput. It's how NVIDIA achieved to >outperform AMD with only half the FLOP density.
>
>That's so wrong it isn't even funny. That's because of AMD's use of static scheduling
>for their VLIW, and because Nvidia is much more optimized for scalar memory accesses.
>Has nothing to do with latency vs. throughput.

Hold your horses. The scheduling efficiency of the VLIW5 core was 3.4 operations (http://www.anandtech.com/show/4061/amds-radeon-hd-6970-radeon-hd-6950/4). So the move to VLIW4 made it very efficient. Also note that instead of one fat ALU taking the role of SFU, three ALUs can together operate as an SFU. This unification means there's even better scheduling opportunity.

So it would be pretty ridiculous if low scalar memory access performance alone was responsible for lowering the efficiency to 50% of that of NVIDIA's architecture. You'd think someone would have noticed that and fixed it by now.

The reality is that AMD's architecture often can't use its full computing capacity. It's not a well balanced architecture for executing a wide range of workloads, but it compensates for that with raw computing power while executing tasks that don't get bottlenecked. Fighting Amdahl's Law by using a massive amount of tiny cores works for a while, but eventually it stops scaling.

Don't get me wrong, it's an excellent GPU for contemporary games. But as NVIDIA proves, an architecture can be both good at games and other workloads, with half of the peak computing power.

>>The way things converge, tacking on gather/scatter support >does put the GPU within
>>striking distance, starting with the IGP. For someone not >>time, a balanced homogenous architecture is the most cost >effective solution for all his processing needs.
>
>I don't believe that a homogeneous architecture is optimal at all, and you have
>yet to show that in any meaningful way. In fact, you have admitted that it is sub-optimal
>for power consumption...which means that as long as graphics consumes a non-trivial
>amount of power, that an IGP will be a superior solution. If there is a day when
>graphics is merely 1-2% of all cycles, then perhaps it might happen...but I don't see that ever happening.

I've shown you that homogeneous architectures can work, through the example of vertex and pixel shading unification. If there was a fixed ratio in the workload, keeping them separate whould have been more efficient, but because that's not the case, all desktop and laptop GPUs have moved to unified shaders.

Now, while gaming with an IGP, you're practically always going to be GPU limited. There might be some peaks in the CPU usage for physics and AI and such, but much of the time the CPU is waiting on the IGP. While not gaming, which is going to be the majority of the time for people who chose a system with an IGP, it's the IGP that sits idle. In particular, it's of no use during CPU intensive tasks. Also, they're both using the same floating-point and integer operations. So it's clearly worth unifying them (by wich I mean ditching the IGP and adding gather/scatter to the CPU).

That would give us a 6-core mainstream CPU. And with FMA already on the roadmap we're looking at 650 GFLOPS of parallel processing power within grasp. That's nothing to sneeze at; comparable to a GF106 in GFLOPS, transistors and TDP. So the global properties of a compute-oriented architecture are all there. Only the lack of gather/scatter support would hold it back, which is easy enough to fix.

>>Note that widening the vectors amortizes the cost of >things like out-of-order execution.
>>At the same time, AMD has reduced its VLIW width from 5 to >4, in order to achieve
>>higher efficiency.
>
>So what?

Widening the vectors makes the CPU more efficient at compute-oriented workloads. Performance/transistor increases.

AMD reduced the VLIW width but not the front end or register file. This results in a lower computing density. Cypress XT achieves a theoretical 2.7 TFLOPS with 2.15 billion transistors, while Cayman XT requires 2.65 billion transistors to do the same. The effective thoughput increased by 10%, but still the performance/transistor went down.

Convergence. That's what.

>>GPUs also introduced concurrent kernel execution and >scalar execution,
>>and have growing register files and caches. So they're >investing more transistors
>>into latency reduction and programmability than raw FLOPS. >GF110 has a 0.52 FLOP/transitor
>>ratio. With G92b that was still a 0.94 ratio.
>
>Register file size is increased for better throughput...the registers per vector lane have been decreasing.

That's not what you wrote here: http://www.realworldtech.com/page.cfm?ArticleID=RWT121410213827&p=5

I'm assuming you're talking about GF100 then. Note that GT200 doubled the register count, without touching the execution core. GF100 was able to reduce that again, but at the cost of increased scheduling complexity. Quoting your article on GF100: "Again, each pipeline is still scalar, but there are now two for added throughput. Despite the notion that GPU cores are simpler than CPU cores, the schedulers have to tackle considerable complexity."

This proves two of my points. The first one is that GPUs either lose computing density by adding more registers, or they lose computing density by adding complex scheduling. And secondly, it indicates that the use latency-oriented techniques doesn't make a modern CPU less of a comput-oriented architecture.

>What you are saying is obvious - GPUs are becoming more programmable. But the
>reality is that they are not even remotely optimized for latency. Where do you compile your code? On a CPU or a GPU?

They're not very optimized for latency yet, but they are becoming more optimized for it. They've got no choice since even graphics is running out of easy to exploit DLP. And to run some of the parallel workloads the CPU is currently better at requires reducing lantecy as well.

Note that it wouldn't take a whole lot of logic to make the GPU vastly more latency optimized. They've already got generic caches, but lack speculative prefetch, which is tiny in comparison to the cache itself. And they've already got superscalar execution, but they lack reordering. A small instruction window would allow to hide some of the latency within the same strand, which allows reducing the register file and reduces cache contention.

So you see, the design space allows to make them much more latency optimized without making them much less thoughput optimized, if at all. I still won't use the GPU for compilation any time soon, but there's a whole range of GPGPU applications which would be vastly more successful on a more latency optimized GPU. And NVIDIA's push for the HPC market forces them to come up with efficient techniques to balance the efficiency at complex workloads.

CPUs do trade some potential thoughput for latency, but note that they've come a *very* long way since the Pentium 4 days. By adding FMA and gather/scatter support they retain their efficiency at latency-bound workloads, but become even optimized for thoughput workloads, including adequately running graphics to make the IGP redundant.

>The bottom line is that to achieve optimal throughput you must sacrifice latency
>(look at the memory subsystem), and vica versa. You can refuse to believe this,
>but it's simply true. While GPUs may become more programmable, this is all relative
>to an architecture that started with 0 programmability. The gap between GPUs and
>CPUs may shrink, but it will never disappear and the efficiency differences will always be sizable.

Saying something is "simply true", doesn't make it true. I've given you lots of proof of the contrary already. Refusing to optimize for latency, is not something that works for long due to Amdahl's Law. It may cost theoretical thoughput but unagressive latency optimization improves effective throughput. Only the latter is relevant.

Regarding programmability versus efficiency differences, you're forgetting that pixel processing was still largely fixed-function when vertex processing became programmable. Yet it evolved into full unification!

The gap will not stop shrinking, because of the simple fact that you can't get high utilization out of a million cores for any realistic workload in consumer applications. So there is no other option but to *increasingly* focus more on latency optimization.

>>It's easy to see where your preconceptions come from though. NV40 had a 0.24 ratio,
>>which G92b increased by a fourfold in a few years time. But you got fooled into
>>thinking that this is a trend which can be sustained. Widening a component only
>>increases the overall throughput desity of that component till it reaches 50% of
>>the die area. And the components themselves get fatter to increase programmability
>>as well, and the rest of the architecture needs to support >the same throughput.
>
>Of course it's easy to see where my preconceptions come from. It's reality.

Keep telling yourself that.

[snip]

>>>Let's take another example. Icera makes a very cool SDR. However, to meet the
>>>performance and power efficiency requirements, they use a custom designed chip to
>>>run the SDR. So, the 'dedicated hardware' is used by many different radio protocols,
>>>in exactly the same way that GPU shaders are used by many different shader types.
>>It's still dedicated hardware though.
>>
>>Does Icera's SDR support IEEE-754? I guess not, so *this* >is irrelevant.
>
>What does IEEE-754 have to do with latency?

Who's talking about latency? This part of the discussion was about dedicated versus generic/unified hardware.

I mentioned IEEE-754 because among other criteria it only makes sense to unify things when you're using the same generic operations (i.e. floating-point and/or integer math). There's singificant opportunity for using dedicated hardware for an SDR since it's a very defined workload. You can't compare that to graphics, which uses the same IEEE-754 operations as supported by the CPU, and there's a varying workload.

Also, dedicated SDR hardware belongs in the ultra-mobile market. This market has very different characteristics. Power consumption is of the utmost importance, while cost is much less of an issue because the required logic is tiny anyway.

>>It's nothing personal, but face it, you're running out of >arguments and start handwaving
>>and reaching for absurd examples which I'm easily able to >debunk.
>
>Only because you totally fail to understand and refuse to acknowledge reality.

No, I do understand and acknowledge the reality of a dedicted SDR for the ultra-mobile market.

What you're failing to understand and acknowledge though is that its charachteristics have no immediate relevance to the markets I'm making software rendering claims about.

>>>In case you haven't noticed, modern CPUs are filled with >>idle silicon. Floating
>>>point units, AES crypto blocks, virtualization support, >>real mode support, etc. Many of these were added recently.
>>
>>Floating-point is useful to graphics, so this isn't an argument against software rendering.
>>
>>As for AES, virtualization, real mode, etc. they certainly don't "fill" the CPU
>>with idle silicon.
>
>Microcode?

Tiny, and not idle.

>>Unless you can prove me otherwise, AES doesn't take die space
>>proportional to the GPU's, ROPs, texture samplers or >rasterizers.
>
>The ROPs are used for general purpose workloads, as are the texture sampling units.
>Where do you think loads and stores are executed? And atomic operations? The
>rasterizer is not useful for general software, but how much power does it consume? How much area?

Fine, then lets compare AES to anti-aliasing or anisotropic filtering.

On the one hand you're trying to tell me dedicated hardware is an absolute necessity but on the other hand CPUs are not allowed to spend a tiny bit of die space on things like AES?

Don't mistake my claim about software rendering on a homogenous architecture, in specific markets, for a claim that all dedicated hardware should be banned.

>>And like I said
>>before, fast AES support is important for generic encrypted disk and network access,
>>and gather/scatter speeds up software AES so the dedicated >hardware can be removed.
>
>You said that, but you're wrong. You cannot remove it for compatibility reasons, and also for security reasons.

Wrong. AES-NI has its own CPUID bit. Software has to check support for it before using it. So there's no compatibility issue. And any security attach is utterly impractical. But even for the paranoia it doesn't mean you're out of options without AES-NI hardware support. The AESSE implementation only uses registers so it's not succeptible to cold-boot or cache-timing attacks.

>>VT-x and real mode are even supported by Atom cores, so >it's doubtful this takes
>>any noticable die space on a desktop chip, and it's >obviously indispensable for the software that make use of >it.
>
>Why is virtualization support in hardware? VMware was doing fine with their binary
>translation. Maybe it was added to improve performance and efficiency!!!! Just like rasterizers!

First of all, you're talking about totally different pieces of dedicated hardware. You can't conclude from the potential need for virtualization hardware that dedicated rasterizers are a necessity for graphics.

That said, virtualization may not be needed at all: http://www.vmware.com/pdf/asplos235_adams.pdf. Obviously it's not an efficiency improvement when performance is much lower. Once again though, it's a CPUID bit. So if they ever felt like it's not worth it, they can leave it out. So far it looks like they intend on keeping it, but you can't conclude anything about graphics hardware from this.

>>Besides, like I said before GPUs also have lots of programmability features which
>>may or may not be used. For instance it's doubtful I'll ever use my GeForce GTX
>>460's double-precision computing capabilities. But that's fine, it's relatively
>>small and it's not worth designing a separate chip for the people who do use it.
>>
>>So I have nothing against dedicated hardware in general, but like I said it has
>>to offer a high enough efficiency advantage, weighed against its utilization. The
>>problem with some of the GPU's dedicated hardware is that even during it's key application,
>>graphics, it's often either a significant bottleneck or >mostly idle.
>
>You have said that, but frankly, you've said a lot of things that are simply wrong.
>
>How about you provide some hard data on modern high performance GPUs (e.g. most
>recent generation from NV or AMD) on the utilization of the rasterizer. They have
>performance profilers, so it shouldn't be too hard. Then you can find out how much
>power the rasterizers use, and we can compare it to the power consumption of SW
>rendering. Then you will have actually a marginal understanding of the relative efficiency.
>
>And I'm fairly certain that you will find that comparison to be very unattractive for SW rendering.

For software rendering, rasterization and gradient setup combined takes on average 1.4% of CPU time running the Crysis benchmark. That's all the data you need from me. The rest of the claim is yours, so you prove it.

>>Unifying vertex
>>and pixel processing removed the bottleneck between them >and increased utilization.
>>Texture sampling is useless to generic computing and >having too few texture units
>>is a bottleneck to graphics, while the importance of FP32 >texture filtering increases,
>>so it makes lots of sense to start doing the filtering in >shader units and have
>>more generic gather/scatter units. And support for >micropolygons would require substantial
>>hardware to sustain the peak throughput, but it's again >idle during other workloads
>>and even for graphics it's full capacity isn't used all >the time. Make it smaller,
>>and it's a bottleneck when drawing micropolygons. Again >unification seems like the better option here to me.
>
>You haven't even quantified the gains from utilization at all for rendering, or the cost in terms of power consumption.

What you're asking here is probably worth a doctoral dissertation. So you're going to have to wait for detailed data than what I've already provided, or come up with it yourself. In the meantime, I've given you plenty of arguments to make it at the very least plausible for software rendering to make the IGP redundant once gahter/scatter support is added.

Face it. You haven't presented a single smashing evidence of the countrary. You started with the preconception that hardware rendering is an order of magnitude more efficient, but that clearly crumbled as you had to look for deeper differences, which don't affect the global efficiency nearly as much, and you felt the need to come up with ever more contrived examples from different markets than the one that's relevant. Seriously, this entire discussion has only made me more confident in what I do. Thanks for that.

>>>>What you're also forgetting is that the software evolves as well. In 2001 people
>>>>were really excited about pixel shader 1.1. Today, a desktop GPU with only pixel
>>>>shader 1.1 support would be totally ridiculous, regardless of how power efficient
>>>>it is. I've said it before; we don't need more pixels, we >need more exciting ones.
>>>>Which means increasing generic programmability.
>>>
>>>So let the shaders evolve, and stay separate.
>>
>>I sincerely hope you're not being serious. There's no way >GPU manufacturers will un-unify their architectures.
>
>Please read what I wrote, carefully and think about it. "Stay separate" implies
>they are already separate. What are they separate from? You seem to assume I'm
>talking about the vertex/pixel/geo shaders being separate from one another, but that's hardly clear.
>
>What was meant is that the shaders should stay separate from the CPU (which is the state today, even in IGPs).

I misunderstood that, but ironically it's not all that very different from asking GPU manufacturers to un-unify vertex and pixel shaders. You're not aknowledging the motivations and advatages behind that unification.

So tell me, why should vertex and pixel shaders stay unified while unifying the CPU and IGP would be a bad idea?

>>>Every single fact that I've seen tends to suggest that software rendering is a demonstrably bad idea.
>>
>>You haven't demonstrated anything.
>
>Sure I have. CPUs are not optimized for throughput and have roughly 4X lower performance
>efficiency. In fact, in some cases that's a vast understatement.

No, you have not demonstrated that CPUs are not optimized for throughput. You demonstrated they are optimized for latency, and wrongly implied from that they can't be optimized for throughput.

>A Tesla has roughly 2.2 GFLOP/s per W (DP). A high performance Westmere has roughly
>0.75 GFLOP/s per W. Cayman is roughly 2.7 GFLOP/s per, although a real workstation
>card would be lower, probably around 2.5 GFLOP/s per W.
>
>So the reality is that the performance per watt is much worse on CPUs than GPU,
>by a factor of 3-4. So to achieve the same throughput, the power consumption would
>be 3-4X higher. So...um...CPUs aren't throughput optimized.

Westmere doesn't even have AVX, and FMA doubles the GFLOPS rating again. Furthemore, as proven by NVIDIA, Cayman's effective throughput is only half the theoretical throughput. So there's your 3-4X smashed to piece. And while GPUs make use of their fixed-function hardware as well during graphics, you're neglecting that they're no longer able to scale their effective throughput aggressively.

>>And "tends to suggest" coming from someone who's clearly basing things on prejudice
>>is just more handwaving. I've proven you WRONG about the necessity for dedicated
>>texture decompression, using real data.
>
>You have no real data. You had bad data from an old simulator that the author
>of the simulator thought was BS. Garbage in, garbage out.

I do have real data, but I might have forgotten to mention it in this thread (I mentioned it in two other posts though): Crysis at High detail at 1680x1050 performs 22 million compressed texture accesses per frame. Assuming no magnification and no texture reuse, this means that using uncompressed textures instead would have costed only 4 GB/s of extra bandwidth at 60 FPS. But no IGP runs Crysis at these settings at 60 FPS (not even my GTX 460), so it's way less than that, and in reality there is some magnification and quite a bit of texture reuse in the foilage.

[snip]

>>And finally I've shown that an IGP does cost
>>quite a bit and is worthless for non-graphics applications.
>
>That you definitely haven't shown. And IGPs are useful for the same general purpose
>applications that a GPU is. Fusion parts will have OpenCL and compute shader. So will Ivy Bridge.

Then show me one real-life example of an application using the IGP for something other than graphics, and achieving an advantage over using properly optimized AVX code (I'll settle for SSE if you must).

>>Yes, GPUs are evolving too, toward a more CPU-like >architecture! I've proven that many times now.
>
>Yes and the relative gap in performance is still HUGE.

Is it? Again, show me this huge performance gap for anything other than graphics running on the IGP. Also, the gap for SwiftShader is only 5x. It's not using AVX yet, there's no FMA, and no gather/scatter. Are you still comfortable claiming the gap will be huge with these three throughput-oriented technologies in place?

[snip]

>The bottom line is that while it's true that GPUs and CPUs are evolving towards
>one another, that says nothing about how vast the distance between the two is.
>The reality is that there is roughly a 4X gap in performance efficiency between
>GPUs and CPUs on many throughput workloads, and the gap is even larger on latency sensitive workloads.

There's a 4X gap today but not for long. And no the gap isn't larger on lantency sensitive workloads. There's software pipelining to deal with that. There's nothing in terms of latency a GPU can do, that a CPU can't.

>Throughput means more than just scatter/gather although it is one key aspect.
>But to simply throughput down to scatter/gather is pure ignorance and naivete, and
>shows an acute lack of understanding of the substantial differences in circuit design, microarchitecture and software.

With all due respect, you missed the fact that the 4X throughput gap is soon gone and you're telling me I have an acute lack of understanding the substantial differences? Please.

Take care,

Nicolas