Nicolas Capens (nicolas.capens@gmail.com) on 1/5/11 wrote:
---------------------------
>Do you happen to have any sources which explain just how complex it is? I'm sure
>it's not trivial, but given that LRBni already has many other features which I'd
>consider relatively complex, I'd be surprised if gather/scatter took a disproportionately
>large area. Intel was able to fit 32 of these feature rich cores with 512-bit vectors
>onto a chip roughly double the transistor count of Sandy Bridge...

Unfortunately Intel didn't disclose details on LRB die AFAIK so we don't know what was the weight of every single feature. Yet the fact that LRBni doesn't include a full-vector permute tells you that doing a crossbar of that size is prohibitive.

>What I meant is that the two 128-bit load units could each have their own 4x32-bit
>crossbar. It means they sometimes load the same cache line, but that's ok. I assume
>that's what already happens anyway with vmovaps, and it considerably simplifies
>the crossbar. So while Larrabee requires one massive 16x32 crossbar an architecture
>based on Sandy Bridge would have a near optimal gather/scatter implementation with two simpler 4x32 crossbars.

Yeah, that sounds feasible to me.

>At the same or higher image quality level. On a Core i7 965 SwiftShader scores
>620 3DMark06 points. That's more than a GMA X3100.

It is nice to hear that 3DMark06 can be run in software but an i7 965 is a pretty high end processor and beating a an old and lowly GMA X3100 doesn't tell me much. At 1280x1024 decent IGPs are already scoring in the thousands and will have vastly better performance/W, performance/$ and performance/area than an i7 965 doing software rendering.

>CPUs have always run tasks which they're not specifically designed for. There's
>simply a point where a finely tuned generic processor is more interesting than a
>highly specialized one with limitations. As I've mentioned before, GPUs have also
>unified their vertex and pixel pipelines into more generic shader cores, because
>with the latter you can run a much larger range of workloads efficiently. It also
>spurs creativity to be able to go beyond what everyone else has been using the hardware for.
>
>Also note that there's like a 100 fold difference in performance between the high-end
>and low-end GPUs on the market today. So clearly the consumer's expectations vary
>wildly. People who don't really care about gaming (except maybe causal games), would
>be better off with extra CPU cores for higher performance in the applications they
>do care about. Efficient software renderers running on modern CPUs can already take
>care of occasional 3D applications (3D on the web will be a huge thing - Adobe has
>licensed SwiftShader for future versions of Flash, for when the hardware is inadequate).
>And despite the fact that cutting edge games have an insational hunger for faster
>GPUs, everything else is not a fast moving target. Today's casual games aren't all
>that much heavier than those from several years ago, while in the same time period
>mainstream CPUs have become many times faster.
>
>At the same time, GPUs also continue to become more generic. The days of fixed-function
>rendering are long gone and games continously push the boundaries of the hardware.
>NVIDIA has managed to create an architecture which can outperform an AMD GPU, with
>only half the theoretical GFLOPS. So clearly other factors start to matter a lot.

You could also say that AMD managed to create an architecture which matches NVIDIA's with vastly lower die area and transistor count. The differences between their hardware boil down to the fact that AMD is using VLIW shader processors which by design will have lower achievable throughput but being simpler much higher compute density.

>GPUs need ever larger caches, reduced branch granularities, ALUs with more generic
>operations, support for call stacks, etc. Mark my words, one day they'll need speculative
>execution just to prevent the register file from growing out of proportion.

Speculative execution will not cause the register files to shrink, it will cause them to grow. The larger the instruction window the more physical registers you need to store results of not yet retired instructions.

>Likely
>before that we'll also have programmable texture filtering, which means the texture
>units become more like generic gather/scatter units and the actual filtering is
>done at full FP32 precision in the shader cores (AMD already does the texel address
>calculations in the shader).

Texture address calculation has always done partly in the shader core and partly in the TU since R6xx IIRC, and technically speaking you could consider the TU a coprocessor of the shader processor being completely tied to it. Add to this that both nVidia and AMD continuously improved the filtering speed of their TU as well as adding specialized functionality (like Gather4) to accelerate in hardware common parts of custom filters. Besides TUs also implement on-the-fly decompression of compressed textures, not just addressing and they often have specialized caches (very granular, often with fully associative addressing). That is not something you can easily merge with a more generic architecture. Doing so would certainly yield lower throughput, and higher power at the same performance level.

>The only reason we haven't seen a sign of Larrabee as a GPU yet, is because it's
>too soon, and because it approaches the market from the wrong end. It's too soon
>because despite the enormous opportunities a generic architecture like that offers,
>developers aren't just going to invest in developing applications for it if it has
>too little market share. And it has too little market share because Intel tried
>to sell it as a high-end GPU. It's an architecture with huge potential but it's
>facing a massive chicken-and-egg problem. We first need the rest of the GPU landscape
>to evolve in the same direction before developers will take the plunge. There's
>simply no strong software ecosystem yet for things that go beyond the classic graphics
>APIs. OpenCL and such are cautious first steps but it will take many more years
>before you can write code in the language of your liking and have it run on either
>a CPU or a GPU. It takes more steps toward full convergence for this to become a reality.

I agree with this POV but I don't think that software rendering will ever be viable, even in the low end, but more on this later.

>Software rendering for the low-end market on the other hand is viable today. There's
>bound to be people who rather have a more powerful CPU instead of an on-die GPU.
>And these powerful CPUs are attractive to developers because they can safely extend
>the existing software ecosystem without big risky investments. This software further
>increases the demand for powerful CPUs, and these will be capable of more than just low-end graphics...

I don't think so. Basing ourselves on your 3DMark06 number means that most of the casual games available lately on services like Steam wouldn't be able to run at playable frame-rates even at the lowest setting and resolutions. And that would be on high-end, quad cores, what would this mean for people with CULV laptops?

>If all this still seems unlikely to you, just look at what happened to sound processing.
>In the early days, mixing and modulating multiple channels at high quality was a
>heavy task for the CPU to perform. So your best option was to get a discrete sound
>card. Then this sound processor migrated to the chipset. And nowadays it has been
>reduced to just an I/O chip and the actual processing takes place in an audio codec
>run by the CPU, thanks to its vastly increased performance. Not even power efficiency can reverse that trend.

Well, in fact it already did and I am glad that you brought up the topic. Look at mobile phone SoCs, almost all of them have CPUs with excellent performance/W yet except for the cheapest ones they all sport a custom DSP for sound decoding and processing, usually coupled with a custom path to the DAC so that large portions of the chip can be powered down. The net result is vastly longer battery life when doing the common activity of playing back an MP3. Also SoCs are a pretty good example of custom hardware growing instead of shrinking, video hardware decoders have been growing in functionality and are now augmented by encoders. A peek at the schema of a BluRay-player SoC will also show you a disproportionate amount of hardware accelerators. The reason for this is very simple: as far as performance/W and performance/area go nothing beats custom hardware and that is becoming ever more important with the continuous growth of power-constrained devices.

>The same thing will happen with graphics. Sandy Bridge already makes the GPU share
>the CPU's caches and memory controllers. So it's not that big a step to perform
>the actual processing on the CPU cores.

Well, actually it *is* a big step. The only thing that changed is that now the IGP can access the L3 in a non-coherent manner, the memory controller was already shared as the IGP was on the northbridge. The IGP is still a processor of its own, with a specialized instruction set and hardware.

>Once the CPU can run Crysis at high quality,
>which I expect to happen well within this decade, the number of people willing to
>pay for a GPU will have reduced considerably.

Really? Even if running Crysis on the CPU will consume 100-1000X as much power as running it on a GPU for a similar performance level?

>And this entices more developers to
>make use of the powerful generic capabilities the CPU offers, meaning that this
>software doensn't run very efficiently on a GPU unless it evolves even closer to
>the CPU's architecture. It's an unstoppable spiral which will cause the GPU to go
>the way of the dodo in the long end, just like sound cards are exhaling their last breath.

See above for the sound card analogy, as for the GPU-going-to-the-CPU code I beg to differ. For a long time I have been skeptical of the possibility of offloading non-rasterized workloads on GPUs but with the impressive stride of ray-tracing, path-tracing and full radiosity implementations coming to the GPUs it seems to me that it's going the other way around. GPUs are simply better for graphics, no matter how you slice it.

>The DLP for graphics workloads is stagnating. The motto "batch, batch, batch" has
>reached its limits and to efficiently run more diverse applications you need to
>take advantage of more than just data level parallelism.

The way graphics workloads have been evolving DLP is actually becoming less and less of a problem. With deferred rendering and virtual texturing many recent engines can actually draw an entire scene with only a minimal number of API calls.

>And like I said before we don't want dedicated hardware for filtering. Everything
>is pointing towards more generic gather/scatter units and filtering in the shaders.
>Neither do we want dedicated hardware for rasterization. Tesselation is an early step toward programmable rasterization.

I suggest you dig a little deeper LRB's demise as a GPU, lack of hardware rasterization and poor filtering performance had both an important part in it.

>None of this is happening overnight and indeed it's not an easy task, but if you
>look what GPUs were like several years ago, and extrapolate that several years into
>the future, it's readily apparent that not a single graphics-specific feature will survive the test of time.

Customized hardware will always beat software-based methods on generic hardware in every possible metric so no matter what happens, if a workloads is regular enough and gobbles enough processing time it will be a candidate for hardware offloading. Just look at IBM's wire-speed processor presentation, the thing has encryption, regexp and XML parsing accelerators on die (which on a side note tells me our whole industry is going down the drain, I never though I would see the XML acronym slapped on a piece of a die micrograph).

>Please name one co-processor on the same die, which after many years still remains a heterogeneous component.

Well, I guess it depends on what you define as heterogeneous. FPUs still have separate instructions and register files, that sounds heterogeneous to me. A plethora of other hardware accelerators have been living on the same die as the CPU in SoCs without being absorbed (and some of them actually gained a customized CPU in the meantime).

>The fact of the matter is that no developer really likes heterogenous architectures.
>Software development is complex enough as it is to avoid having to deal with multiple
>programming models.

I wholeheartedly agree with that. Yet it's not going away, in fact it's getting more complex. Die area matters, and power even more so even if we developers don't like it.

>Also keep in mind that the GPU's programming models haven't
>even settled down. Supporting multiple brands and even multiple generations from
>the same manufacturer is a huge pain. So together with the 100 fold difference in
>performance between the high-end and low-end, we're nowhere near using the GPU as
>a reliable vector co-processor. Targetting SSE and AVX offers a lot more guarantees.

That I also agree with, however we went from being completely unable to use GPUs for other workloads to being able of using them. As far as content creation goes I've been seeing more and more CPU-only applications offering GPU-assisted offloading which is something that (pleasantly) surprised me as I think it's the field where GPGPU makes more sense.

>The only thing missing to make the CPU's SIMD support complete, is gather/scatter.
>Once that's available, the GPU makes little chance as a co-processor.

Honestly, you seem to be ignoring what happened with LRB and that's a piece of hardware on which some very fine hardware designers and programmers worked on and it tanked exactly because bolting gather/scatter to a CPU is not enough to replace a GPU, far from it.