First, we're just about at the limits of single CPU speed. In the last ten years, we've gone from 300 MHz clocks to 3GHz clocks. That growth is over. We're not going to see 30GHz clocks in the next ten years, at least not in anything commercial.

There's an NSA effort (http://www.nitrd.gov/pubs/nsa/sta.pdf) to get clock rates up to the 50-100GHz level, and it's worth reading that paper to get an sense of what's required to increase clock rates from current levels. They're proposing huge supercomputers that use Rapid Single Flux Quantum devices, superconducting components, and run in liquid helium at 4 degrees Kelvin. Cooling gear struggles to keep the temperature down near absolute zero as the unit generates a kilowatt of heat inside the liquid helium tank. Half a million data streams at 50 Gbps each are required between the superconducting processors and room-temperature SRAM.

There may be some desperate military need for this. Its' not a technology that will take over the commercial data center world, let alone the desktop.

So we're stuck with a technology that gets smaller and cheaper, but not much faster.

This forces us in a few directions. We only have three ideas that really work: clusters, shared memory multiprocessors, and massively parallel graphics processors. All the other architectures that have been tried, from the nCube to the Connection Machine to the BBN Butterfly, have been duds. Even the Cell, the only exotic multiprocessor to go into mass production, has been a disappointment.

The future of personal computing may be cheaper and lighter portable devices, not more powerful ones. We should not assume that personal computers will continue to increase in power. We will see all that chip real estate used for memory, not more CPUs, to produce cheap single-chip computers. There will be more $100 laptops than 100-CPU laptops.

In larger machines, it's worth thinking about how to organize tightly coupled clusters better. We probably are going to get clusters on a chip for servers. At some point (probably around 20 CPUs, from SGI's experience) the scaling problems of shared memory multiprocessors get in the way, and it's better to have a separate address space. So we need to think about how to make inter-computer communication for such systems better. We need interprocess messaging that's supported by hardware to make it efficient without violating protection boundaries. We need operating system support for such messaging. Using XML or JSON over TCP/IP over simulated Ethernet to pass a message to another cluster on the same chip is not a good answer. Neither is shared memory, which leads to "one fails, they all fail" clustering. What programmers usually want in interprocess communication is a subroutine call; what the OS usually gives them is an I/O operation. Code is layered on top to work around that mismatch, inefficiently. We need to get past that. (Look at Infiniband, and at QNX messaging for successes.)

The other interesting area of work is graphics-type processors, which are turning into useful number crunching engines. I've been to talks at Stanford on compiling Matlab into graphics engine code. That may be the future of number crunching. It's definitely the past and future of game machines, which have almost always had most of their crunch power in the special-purpose engines. I can't really speak to that issue.