Hi David,

David Kanter (dkanter@realworldtech.com) on 2/18/11 wrote:
---------------------------
>>>You don't understand anything I write, so I'm done discussing this with you.
>>
>>What exactly didn't I understand? That a high-end discrete GPU with almost three
>>times more transistors having nearly an order of magnitude higher bandwidth over
>>a CPU+IGP, really is relevant to the viability of having a >homogenous CPU take over the role of said IGP?
>
>No, you don't understand circuit design or any of the fundamental trade-offs that
>come into play when designing hardware (let alone the economics of selling such hardware).

That's your opinion. You haven't actually proven that I don't understand these things.

Let's try again. How is a high-end discrete GPU with almost three times more transistors having nearly an order of magnitude higher bandwidth over a CPU+IGP, relevant to the viability of a homogeneous CPU taking over the role of said IGP?

I'm an intelligent guy with a masters degree in computer engineering so if I'm ignorant about this matter I'm sure an analyst like you can explain it to me in glorious detail. Either that, or this order of magnitude difference is nowhere near as relevant as you're trying to imply.

>Not only that, but you consistently refuse to face reality and consider HARD metrics
>and data that is contrary to your beliefs.

The bandwidth of a discrete GPU is indeed a hard metric, but I don't refuse to face that and it's not contrary to my 'beliefs'. It's just utterly irrelevant to a homogenous CPU taking on the task of the IGP. Unless you have other hard metrics that prove software rendering would be bandwidth starved while the IGP is not?

>>Don't be ridiculous. You're done discussing because you >ran out of arguments. You
>>failed to look at the bigger picture and take the (triple) >convergence into account,
>>and you failed to take the benefits of unification into >account (*).
>
>You're certainly welcome to your near religious beliefs.

Don't try to turn the whole discussion into something personal. I've backed each of these arguments with evidence. I'll sum things up once more:

1) Convergence #1: CPUs will improve their compute density with FMA, and gather/scatter support improves effective throughput.
2) Convergence #2: GPUs will lose compute density because Amdahl's Law prevents linear performance scaling for the same workload. Investing into latency optimizations to prevent bottlenecks, and frequency optimizations to speed up sequential tasks, becomes ever more vital.
3) Convergence #3: The workload is shifting too. There's an increasing need for programmability and GPUs are expected to do more than classic rasterization alone. This forces them to spend less die area on throughput as well.
4) Unification: An IGP still needs a CPU so there's always one of them going to be partially idle. With a homogenous architecture you have the whole chip at your disposal.

All of these things help favor software rendering in the not too distant future. You can call them "religious" all you want, you haven't debunked any of them.

>>And that's forgivable. The current gap between the CPU and IGP can easily lead
>>one to disregard progress and believe CPUs can never adequately handle graphics.
>>But I expected you would at least have the decency to admit I might have a few valid
>>points about how that current situation will change in >favor of software rendering,
>>instead of saying I don't understand anything you write.
>
>I told you exactly when that would happen. When the % of CPU cycles required to
>do mainstream rendering* is <5% of the CPU cycles. That's what it took for us to get rid of sound hardware.

You're wrong. First of all, playing an MP3 on an AC'97 solution took up to 50% of CPU usage in the early years. This dropped as CPUs got faster, but then EAX and other tehnologies were introduced and it went up again. So for a considerable amount of time the CPU usage was well over 5% yet software audio processing quickly dominated the market.

Secondly, I've already shown you that gaming with an IGP takes relatively high power consumption as well: http://www.legionhardware.com/articles_pages/intel_core_i5_2500k_and_core_i7_2600k_sandy_bridge,14.html So a software rendering solution does not have to take less than 5% of CPU cycles to be viable.

>>That would be pretty terrible in and by itself. You're an >analyst so you're supposed
>>to explain things in a way people can easily understand >it. So either you failed
>>to make an engineer understand "anything" you wrote, or >some of the things you wrote are false or irrelevant.
>
>Unfortunately, when you are faced with someone who:
>1. Doesn't understand key relevant concepts
>2. Refuses to learn
>3. Cannot agree on basic and widely accepted metrics for comparing different approaches
>4. Constantly pushes for hypothetical scenarios with no data or evidence
>
>There's really not a lot to be done. I can't debate someone over beliefs that
>take on the nature of religious fervor and blind faith.

Again you're just deflecting technical matter into a personal attack. I might just as well say you don't understand key relevant concepts (or don't understand why some are or little or no relevance), you refuse to learn, you cannot agree on anything, and you constantly push for comparisons against products for distinct market segments. That doesn't advance the discussion since it's one professional's opinion against another. So I prefer to think you're an intelligent guy, who actually does understand the things I say and has the ability to explain in detail the things I might be wrong about (and explain how it's relevant to the bigger picture).

So stop demeaning yourself and wasting time by changing the subject from technical into personal. Even in the odd chance that I'm totally ignorant about all this, it's arrogant and rude to say I'm wrong and not take my arguments into account. Heck, if it was readily obvious to everyone that I'm wrong about everything, the right thing to do would be to just ignore me and let the future speak for itself, not try to insult me.

>>So I kindly suggest you do take another look at my previous response, and either
>>show how I misunderstood that big differences at the hardware level equally affect
>>software rendering efficiency of future CPUs, or start putting some doubt into your
>>premature conclusions. I believe we settled on using Crysis as a benchmark for effective
>>throughput, so don't give me theoretical data unless you also prove that it's significantly
>>relevant to the viability of a homogenous CPU doing >graphics.
>
>I already explained, at length, repeatedly. Others have explained as well.

Not really. Noone has explained how a latency optimized device with wide vectors and FMA can't possibly be throughput optimized as well. And given the unification benefit it doesn't need to have the same compute density to count as a throughput optimized device. And let's not forget the three convergence arguments which continue to move them closer together.

Furthermore, you nor anyone else has given me benchmark numbers for the IGP which indicate that the effective performance of a CPU with FMA and gather/scatter can't compete with that.

So stop saying you've "already explained, at length, repeatedly".

>For instance, we all asked how many gate delays, power and area would the hypothetical
>gather/scatter cost to implement. How would it impact single threaded code (e.g.
>SPECint). I have yet to hear anything.

This is the first time I've read anyone explicitly ask about gate delays. But okay; there would be zero extra gate delays. That's because selecting four elements from a cache line instead of one, can be done in parallel. Also, calculating which of the elements are located within the same cache line, and computing their offset, can be done in parallel as well and isn't actually needed till the cache line has already been fetched. Of course fan-in/out increases at some points but (progressive) transistor sizing and/or input reordering can deal with that if necessary. Note that even if I'm wrong about any of this, you'll still also have to prove that the required extra gate delays are actually significant. A contemporary CPU can do 64-bit adds in one cycle and the gate delay for that is between 30 to 60 inverters (and that's without counting other delays to perform back-to-back operations). Since the CPU's clock frequency is increasingly less aggressive you can expect the gate delay budget to go up with every process node.

Next, I already detailed somewhere that emulating gather/scatter takes 24 uops which each pass over 128 bits of data around, and starves some of the other logic so it consumes idle power. Dedicated gather/scatter support would not occupy any of the ALU pipelines, and keeps the data being moved around to a bare minimum. So it's a big win in terms of power.

As for the area requirement, the element selection isn't all that different from the permute units in the ALUs. Other than that four 25-bit comparators and four 7-bit adders are needed (per LSU) but as indicated these can be relatively slow (and thus quite compact). Logic to keep multiple misses in flight is already in place. Of course the exact area requirement is hard to determine but feel free to fill me in if you want to prove it's too expensive. Note that Cell BE has 8 high-frequency 2-cycle-latency independent load/store units at 241 million transistors (looking at just the SPEs), while Sandy Bridge has four times the transistor budget, so asking for 8 load/store units capable of 4-element gather/scatter doesn't seem like a big deal to me.

And I've already pointed you to a paper which indicate that SPECint would be able to benefit from gather/scatter by about 20% thanks to more succesful parallelization of loops. SPECfp obviously benefits even more. So even for non-graphics workloads the area investment is clearly well worth it.

>Moreover, you make utterly unsubstantiated (and simply wrong) claims about the
>applicability of gather/scatter to ordinary code and claims about performance implications
>for SW rendering WITHOUT any substantiation. How do you know that single cycle
>scatter/gather is achievable? You don't even know the number of gate delays required.

You wrongfully assumed I didn't know the gate delays, and clearly you didn't know yourself since otherwise you wouldn't have asked.

And I did substantiate the applicability by noting that texel fetch is essentially a gather operation, which currently takes many uops to emulate. Lookup tables for transcendental functions also fit in L1 cache lines so those operations benefit substantially as well. And that's just scratching the surface.

Furthermore, the 20% benefit for SPECint merely requires the use of the 'restrict' keyword to statically determine dependencies (instead of dynamically as done by the paper). Even if static analysis yields lower benefit, it's highly doubtful that the investment isn't worth it. If that's not enough substantiation for you, feel free to take the opportunity to prove me wrong. Simply saying I don't provide enough positive proof does not count as negative proof.

[snip]

>Moreover, the data you provided was simply WRONG and BAD and DISCREDITED by the
>author of the simulator you used. Merely presenting spurious data isn't helpful to the conversation.

First of all, you haven't given me any indication at all what's wrong about it, nor how wrong it really is. The texture bandwidth of contemporary games may just as well be less than what's shown in that old ATTILA graph. So merely telling me the data is wrong doesn't prove your claims at all.

Secondly, I *measured* the number of uncompressed texture accesses for Crysis, and *conservatively* determined that it does not have a significant impact on bandwidth.

>>(*) P.S.: Just in case you still don't see it, the benefit >of unification is that
>>a homogenous CPU does not need to have the same compute >density as the IGP, to exceed
>>its effective performance. You basically get area for >free, which would have otherwise
>>remained largely idle (and you can trade it for power).
>
>How do you know it would have remained idle?

On my system with a GTX 460 and 3.2 GHz Nehalem, the Crysis CPU benchmark peaks at only 27% of CPU cycles. The average CPU usage is lower, and clearly with an IGP limiting the framerate the CPU usage would be lower still.

So the vast majority of CPU cycles are left unused. You can't ignore the unification benefit when comparing compute density.

>I agree it would be nice to have
>a unified architecture, but I think the reality is that processing in general is
>going to trend towards two distinct resources: latency optimized and throughput
>optimized, driven by the fundamental power and area efficiencies for separating the two.

There's two things separating latency optimized and throughput optimized hardware: architecture and design. Architecture isn't much of an argument though, due to the unification benefits. A homogenous architecture would have lower theoretical compute density than a GPU but like I said before that's fine since you already paid for the architectural latency optimizations to make it function as a CPU. For instance a bypass network takes extra wires and control logic but you need those for the CPU functionality anyway *and* it reduces the necessary register file size and reduces cache trashing. So at the architecture level the latency optimizations don't have to cost you anything thanks to unification.

So we're left mainly with design level differences. An area optimized ALU can be substantially smaller than an aggressively latency optimized one, exceeding any effective architectural benefit from the lower latency. That said, the MHz-race is long behind us, and things continue to converge on no less than three fronts! At 32 nm, the CPU's latencies are not aggressive at all. There's insane overclocking potential (even on air) and I believe all dynamic logic is gone. So Intel is optimizing for area and power as well. By the time gather/scatter is added there will be very little difference between CPU logic and IGP logic. Your lantency comparisons indicating an order of magnitude difference are invalid since you haven't taken the bypass network into account nor the instruction complexity and other stages.

Cheers,

Nicolas