Hi Seni,

Seni (seniike@hotmail.com) on 2/18/11 wrote:
---------------------------
>>>The IGP's share of the other costs (fab buildout, etc.) is not entirely clear.
>>>
>>>The fabs are the #1 cost. But how much of that cost can be spread out per chip
>>>depends on how many customers you can find to buy one chip each.
>>>If there are lots of customers, smaller chip are gonna cost less.
>>>If there are few customers, you can only utilize the fab fully by making big chips,
>>>in which case the extra area per chip has no effect on fab costs.
>>>If you have multiple generations of fabs at once, then the cost of idling a fab is more complicated.
>>>An increase in the area of one product at a new fab may cause another product to
>>>be shifted to an older fab that would have been idling.
>>>
>>>So, for example, by moving Sandy Bridge's IGP on chip, they consume 32nm capacity
>>>at a greater rate and free up some 45nm capacity.
>>>They then use up the extra 45nm capacity by delaying the 32nm shrink of Atom.
>>>Would it be any cheaper if they made IGPs on 45nm and both Atoms and non-IGP Sandy Bridges on 32nm? Hard to say.
>>>And if they did that, but replaced the 45nm IGP with software and idled the 45nm fabs?
>>
>>I don't see how any of this is relevant. Either you have an IGP at 32 nm, or two
>>more CPU cores at 32 nm. The fab cost is the same.
>
>You proposed two designs: one with 6 cores and one with 4.
>The 6-core supposedly would be the same die area as the 4+IGP, and hence similar cost.
>The 4-core was supposed to be cheaper, but by how much? This is what I was talking about here.
>I argue that the cost of the chip is not proportional to die area, and that the savings is quite likely very small.
>Now that I point that out, you suddenly pretend that the you never proposed the 4-core design.

What I proposed is that you either replace the IGP with two cores, or get rid of it entirely. The 6-core design is slightly cheaper due to R&D and production savings and all that, while the 4-core design also saves proportional die area on top of that.

By reducing the die area you can get more functional dies out of a wafer. Not just because you can fit more dies onto it, but also due to yields (a lesser percentage of defective dies).

So seriously, the IGP does not cost just 5 $.

>>>The IGP's cost of design, v&v, drivers, and so on is shared between current IGPs
>>>and any future products that use the same IGP (e.g. future Atom),
>>>and to some extent future products with next-generation IGPs based on modified
>>>versions of the current IGP. (e.g. Ivy Bridge)
>>
>>That applies to CPU cores between generations as well. But within one generation,
>>having more cores is a negligible R&D cost while having both CPU cores and an IGP is a higher cost.
>
>There is no savings whatsoever to be had by shutting down the IGP R&D program,
>unless you do it completely, across the entire product line.

I do think that in time it's possible to shut down the entire R&D program. Ivy Bridge will have a DX11 IGP so that leaves plenty of architectures to shrink down for Atom till it too can switch to software rendering (as I noted before, ironically it does software vertex processing today). In fact Atom doesn't have the out-of-order execution overhead so equipped with AVX, FMA and gather/scatter it would be more throughput oriented than desktop CPUs!

>Besides, saving money by shutting down R&D programs on a product line where they
>have ~50% market share of a huge market? This is your plan?

Intel's bottom line does not depend on its IGP market share. You think they're sad they no longer sell ISA bus sound cards? Of course not. Sound and soon graphics are just applications which can be run on the CPU so they drive CPU sales.

A 6-core homogenous CPU is more interesting than a 4-core CPU+IGP because it's a more powerful CPU that can still do graphics, while a 4-core homogenous CPU without IGP is considerably cheaper while still offering adequate graphics. Both are a better answer to the needs of people who only occasionally play games (again, no serious gamer would choose an IGP), and it's a better answer to the needs of serious gamers who have no need for an IGP in the first place. So despite losing graphics market share in the strict sense, total sales would go up.

>>No, not at all. Programmability does not mean the end of APIs. Look at OpenCL for
>>CPUs, and the many other choices of APIs and programming languages application developers
>>get to choose from. But while some will pick a high-level API or even a full-fledged
>>engine to create their applications, some will create totally new possibilities.
>>So the software ecosystem expands instead of starting over.
>
>So your plan is to have exposed software rendering competing with driver-based software rendering DirectX and OpenGL.

Yes. Application specific implementations, commercial middleware, open-source frameworks, runtime libraries, operating system APIs, IHV drivers, the lot.

Currently developers are still severely limited by having just one graphics API (seriously, Direct3D and OpenGL are one and the same - I'm part of the Google ANGLE development team), and what the hardware can actually do. OpenCL and CUDA are just a glimpse of what is to come; low-level access to fully generically programmable high-throughput hardware. For the low-end market, CPUs with FMA and gather/scatter can play that role.

Note that a dedicated software renderer can be much faster than one implementing D3D or OGL. For instance lots of games to date use software based visibility determination. Integrating this more closely with the rendering process would result in superior performance.

>>Also note that software rendering would enable both backward compatibility AND
>>forward compatibility. Running a game which demands DirectX 12 would require merely a software upgrade.
>
>A software upgrade of what software? The OS? The drivers? The game?

Anything. I've played Half-Life 2 (Direct3D 9.0) on a laptop with an ATI Rage LT Pro (Direct3D 6.0) by dropping in SwiftShader. It would have worked just the same if it came with the game or was part of the O.S. or a driver.

SwiftShader has been licensed for individual applications, but it has also been licensed for an API like Adobe Molehill (meaning every 3D Flash application can use it). So it depends on the situation what makes most sense, but software rendering clearly leaves all options open.

>>>Each sampler does a 4-texel quad at a time, so 2 of them do 8 texels per cycle.
>>
>>Why would samplers process a quad at a time? Do you have any proof of that?
>
>Texels are processed in quads because it is needed to calculate mip-levels for dependent texturing.
>This of course only applies to GPUs that support dependent texturing, so anything DirectX 8+.

That doesn't mean it's processed a quad at a time. You can have a single scalar pipeline process each operation for a quad of pixels sequentially. So there's no reason to assume that the IGP's samplers are really configured as a quad of samplers.

>>I was wrong about the 2 samplers though, it's actually 4 (32-bit textels): http://www.techarp.com/showarticle.aspx?artno=88&pgno=23
>
>I seem to have been wrong about some of the numbers. I took your word for it that
>there were 2 samplers, but it looks like there is actually only one, so it's only 4 texels/clock.
>Which is staggeringly poor. An original Geforce from 11 years ago could do 8 per clock (at like 150MHz though).

The ALU:TEX ratio in shaders is much higher than in the early GeForce years. So the higher frequency is sufficient to keep up with the texturing needs of modern games. As I've mentioned before on this forum, the number of pixels hasn't increased all that much and there's only so many texels you can slap on a pixel (plus the pre-z pass keeps shading to a minimum).

>>Still, 5.4 GT/s peak isn't a whole lot, especially considering that the IGP will
>>be arithmetic limited much of the time.
>
>I doubt it. Those texturing numbers look terrible.
>High end GPUs do 64+ texels per clock. This does 4 per clock.
>Unless you feed it characteristically different workloads than what high-end GPUS
>see, it's mostly going to be texture-limited, except when it's bandwidth-limited.

It's not mostly texture-limited. At 1024x768 and 60 FPS, 5.4 GT/s suffices to use 229 texels per pixel. That's insanely high for the type of games you can run on the IGP at this resolution and framerate.

A GeForce GTX 580 can do 49.41 GT/s, but it also has an order of magnitude higher GFLOPS and an order of magnitude higher bandwidth. So the IGP is very weak in comparison but the texturing capabilities are actually quite proportional.

The reason GPUs have such insane texture sampling capabilities, is because specific shaders are more texture heavy. For instance a full screen effect performing a blur operation could have a 1:1 TEX:ALU ratio. Even if just one in ten shaders are like that, you need to prevent them from dominating the frame time. So to avoid such bottlenecks the samplers are overdimensioned.

A CPU with gather/scatter wouldn't suffer from that. Either it's used for texture sampling, or gather/scatter can be used for the lookup tables to evaluate transcendental functions, or heavy vertex attribute fetching, or parallel rasterization, or pretty much every other task requiring memory accesses!

>>>If you beef up the gather unit so it can pull multiple source texels from a single
>>>L1 cache access in a single cycle, you could catch up a little.
>>>But this would require a swarm of enormous multiplexers in the middle of the L1D/LSU
>>>datapath, which would probably add 2 or 3 cycles to L1 latency and would devastating to all non-graphics code.
>>
>>It wouldn't be all that costly. There's already a 512-bit to 128-bit 'shift' networks
>>with byte granularity in place for each of SNB's load units. Turning this into a
>
>But there's only one of it. You would need 4 or 8 of them to do fast gathers.

No, SNB has two 128-bit load units per core.

>>network capable of selecting 4 x 32-bit ins't that much more complicated.
>
>Shouldn't that be 8x32?

4 x 32-bit per load unit suffices for a maximum of 256-bit gather per cycle.

>>It's somewhat comparable to the multiplexers already in the execution units to
>>support all the permutation type instructions. Note also that these have a latency
>>of 1 cycle. So both in terms of area and timings supporting gather/scatter shouldn't be a big hurdle.
>>
>
>But again, there's only one or two of these in the execution units. You would need a lot of them.

Selecting four elements per cache line instead of one is relatively simple. Note that CPUs have been able to extract data from cache lines since the dawn of caches (more than two decades ago for consumer CPUs). So asking for four accesses per LSU on a future process isn't really pushing it.

>>>Also, it's not clear that other supporting parts of the chip could keep up, in
>>>particular, the parts of the LSU and TLB associated with enforcing memory ordering,
>>>coherency, memory protection checks and so on.
>>
>>It's only accessing one cache line per clock (per LSU).
>
>This stuff isn't done per cache line. It's done per access.

It's done per cache line being accessed. If today you have a variable straddling two cache lines, one part could be in L1 cache while the other part could be swapped out to disk. They're loaded sequentially, not simultaneously, and thus it doesn't require any additional ports to the TLB and such. Neither does gather/scatter put any more stress on the per cache line operations.

Also note that gather/scatter typically makes no guarantees about memory ordering (unlike unaligned accesses which straddle a cache line). Fence instructions can be used to flush the load/store buffers.

>>So there's no extra pressure
>>on any of those parts. It merely needs to be able to collect four elements out of
>>a chace line, which isn't all that hard. And computing which elements are stored
>>within the same cache line is easy too. You don't even need the full address per
>>element for that. Just compare the upper 25 bits of the offsets and compute the
>>lower 7 bits of the address to determine the position within the cache line (and 'overflows').
>
>This is not a 32 bit processor, so addresses aren't 25+7. They are 57+7.
>To check to which elements are stored in the same cache line, you just have to
>run ALL POSSIBLE PAIRS in parallel through separate 57-bit comparators.
>So, for 8 lookups per AVX word, that's 36 comparators.
>
>With all 8 57-bit numbers running as a single 456-bit bus, and then each comparator
>running out taps to the 114 input bits it needs, I get 456 bits wide and 114*36= 4104 bits tall.
>This is a big thing. And that's just the wiring feeding into the comparators.
>
>Maybe it's not as easy as you thought?

No, it's far simpler than you think. Let's say we have a base address of 0x0123456789ABCDEF, and two of the offsets are 0x00000000 and 0x00000080. Are these two elements located in the same cache line?

You don't need to add each of the 32-bit offsets to the 64-bit base address to determine that. I can tell right away that the elements are stored more than a cache line width apart by comparing the upper 25 bits of the offset.

Are offsets 0x0000003F and 0x00000040 located on the same cache line? That actually depends on the sum of the lower 7 bits of the offsets and the base address.

Also note you don't need to compare all possible pairs. You just start with the first element and see which other elements are on the same cache line. If one isn't on the same cache line, it's loaded in the next cycle and the other offsets are compared to this element's offset to determine which ones share this other cache line, etc. No need to compare everything in one cycle.

So all you need per LSU is four 25-bit comparators and four 7-bit adders. And since you have an entire cycle to determine whether any other cache lines need to be fetched, these comparators and adders (which are already very narrow) can use a particularly compact implementation with low power consumption.

>>>So basically, what we're looking at is that gather/scatter alone needs to make
>>>the cores 4 times faster to hit your target of two cores=1 IGP.
>>
>>Not really. 5.4 GT/s corresponds to 86.4 GB/s bandwidth to the texture cache. With
>>two 128-bit gather units a 3.4 GHz CPU would have 108.8 GB/s bandwidth to L1 cache,
>>per core! So even if you reduce the clock speed to keep the power consumption in
>>check, 6 cores amply exceed the minimal bandwidth required to match the IGP performance.
>>
>
>Only because the IGP is worse than I thought.

Yes, and it's not going to get much better any time soon: http://www.anandtech.com/Show/Index/4083?cPage=23&all=False&sort=0&page=13&slug=the-sandy-bridge-review-intel-core-i7-2600k-i5-2500k-core-i3-2100-tested

"The 82.4% increase in clock speed resulted in anywhere from a 0.6% to 33.7% increase in performance."

The CPU is catching up and clearly there's little point in investing more die space into the IGP. So CPU manufacturers should look into using the die space for something more interesting, or cut it out entirely.

>>>The point is there is are two separate the market segments here served by separate product lines.
>>>Gulftown is high-core-count, high TDP, no IGP product, appropriate for mainly for
>>>certain uses. (High-end Desktops, Servers...)
>>>Sandy Bridge is a moderate-core-count, moderate TDP, CPU+IGP product, appropriate
>>>for other different uses. (DTR Laptops, IGP Laptops...)
>>
>>There's not supposed to be separate market segments, other than high-end, mid-end, and low-end.
>
>What???
>You must be masterly with the English longbow.

I think I've clarified by now that eventually there's just 'computing'. A homogenous CPU with FMA and gather/scatter can handle both latency-sensitive and high-throughput workloads (and most applications are a complex mix of these). So the main thing separating the markets is price.

>>An operation is an operation. You need the same ones for any market segment. HPC
>>and desktop applications alike are written in C-like languages which use generic
>>integer and floating-point operations, memory accesses, control flow, etc. So there's
>>little point diversifying for other market segments, other than performance. Server
>>chips could be made more power efficient by having more cores and clocking them
>>lower, but it can essentially be the same architecture.
>
>I don't think you can just claim "All code is the same."
>Granted, Turing completeness and all, differences are limited to such things as
>speed and power efficiency. But the differences here can be huge.

Again, most applications are a complex mix of workloads. But even if some tend to be hugely latency-sensitive or high-throughput, there's no telling what application(s) you'll run tomorrow.

Besides, the IGP is only tolerably good at graphics. Don't even think about letting it perform any other task. To make it capable of some worthwhile GPGPU processing, it needs to become far more programmable and have a memory subsystem capable of handling arbitrary data flows. Basically you need to turn it into something much more CPU-like. So you might as well relieve the poor thing of it's suffering and give the CPU cores gather/scatter support.

>>>There is just too much room for further improvement in graphics quality in games.
>>
>>It goes in multiple directions. Serious gamers will never have enough graphics
>power. But casual gamers won't pay extra to play a ray-traced Super Mario.
>
>Ray-traced Super Mario sounds pretty awesome. I'd get it.

Well, actually, due to the wildly varying latencies occuring in ray-tracing, you need an architecture capable of coping with that optimally. A homogenous CPU with gather/scatter can handle fine-grained dynamic balancing of the complex mix of workloads.

>>> So seriously, the group of people for which low-end graphics suffices, is growing.
>
>But is the group for which it does not suffice shrinking? Or are they both growing together?

I don't have exact numbers at the moment, but I'm under the impression that the percentage of people buying high-end graphics cards is slowly shrinking. There isn't a single game on the market today which I can't fully enjoy on my GTX 460. Even Crysis 2 won't compel me to upgrade, and is likely going to be the most demanding game for some years to come.

As a matter of fact AMD decided to produce smaller sized GPUs, to better match the market demand. The high-end solutions consists of dual-GPU cards (and CrossFire). NVIDIA's tactic is to continue producing large GPUs but to also sell them as HPC solutions. If there was a high demand for high-end graphics, they'd probably have separate architectures for both markets.

They need these high-end products to create a halo effect and satisfy the bragging needs of loud enthusiasts, but what the average consumer actually buys are the mid-end and low-end products.

>>Whereas previously IGPs were considered insufferable, they're now considered adequate for a very sizable market segment. The statistics don't lie: http://unity3d.com/webplayer/hwstats/pages/web-2011Q1-gfxcard.html
>
>How representative is unity3d webplayer?
>Last I checked, the stats from Steam hardware survey show a quite different picture.
>Not that Steam is representative either...

The casual gaming market is booming, and the potential of WebGL and Molehill is hard to overestimate. The web allows to reach a whole new market of people, who are hesitant to set foot in a game store or install Steam. Mark my words; the next World of Warcraft hype may very well be an HTML5 or Flash game. The emerging Smart TV market also consists of devices with weak graphics chips (shameless plug: http://gametree.tv/platform/).

Unity3D already walks the line between casual gaming and more serious entertainment. So yes I think its statistics are fairly representative of the hardware profile of the future gaming market. Don't get me wrong, there will still be lots of people enjoying games like Crysis 2 (me included), but they need a discrete GPU and a powerful CPU without an IGP.

>>>The wildcard here is the stalled console cycle. This hasn't happened before since
>>>the 1970s, IIRC, so it's unclear what effect it will have.
>>>If pc games continue to be mostly console ports, even when there is no new console
>>>for 7+ years in a row, it could create a temporary plateau effect.
>>>There might be no popular game demanding enough to need more than an IGP, thus killing the GPU market altogether.
>>
>>Exactly. What we need is innovation. A homogenous CPU with FMA and gather/scatter
>>would give developers the freedom to develop some new exciting applications that were previously unthinkable.
>
>I'm not so sure about this.
>Mostly what we've gained since the NES is better graphics, but you can't have better
>graphics unless you hire more/better artists, and that costs a lot of money.
>
>I think this is a big part of why so few games are being made for high-end pcs.
>The art cost of better graphics hasn't scaled well.
>Performance won't solve this problem.

Actually that's not entirely true. Currently artists are responsible for working around many of the limitations of the classic rasterization pipeline. Want realistic foilage? Please produce 3-4 sets of geometry. Want realistic fight animations? Please motion capture every possible move. Want realistic lighting? Please produce multiple sets of radiosity lightmaps. Want a huge gaming world? Please place portals and occluders...

Higher (generic) computing performance does help these things. You can do more things procedurally, you can have advanced rigid and soft body physics, you can speed up offline preprocessing, you can compute visibility in real-time, etc. It allows the artists to concentrate more on the actual content. These things have been proposed as GPGPU tasks as well, but I have yet to see really succesful implementations.

The thing is, it takes years to develop fixed-function hardware, while it only takes days to do something in software. For example unifying the vertex and pixel pipeline so you can sample textures in a vertex shader took probably four years from concept to products on the shelves. With a software renderer it takes less than a day to call the texture sampling function from within the vertex processing routine, and a couple days later it can be sent to a client who issues a patch and lets every customer enjoy more exciting gaphics. Granted, that's an old example, but it should illustrate that the possibilities are endless.

Programmable shaders triggered a revolution in computer graphics. But we've far from reached an endpoint.

Cheers,

Nicolas