David Kanter (dkanter@realworldtech.com) on 9/19/09 wrote:
---------------------------
>Wilco (Wilco.Dijkstra@ntlworld.com) on 9/19/09 wrote:
>---------------------------
>>David Kanter (dkanter@realworldtech.com) on 9/18/09 wrote:
>>---------------------------
>>>To achieve the performance of a 2S Nehalem, you're going to need a ton of ARMs.
>>>And they all need to have ethernet to communicate. Those cables are not free and
>>>use space and power, as do the switches they connect to.
>>
>>The current Cortex-A9 doesn't compete with Nehalem indeed. However the gap with
>>x86 is closing fast, and I'd expect the next generation to be far closer.
>>
>>What low power enables is packing many sockets in a single rack - 8-16 sockets
>>should be possible. As long as they can talk to each other, you don't need more
>>than 1 network connection per board.
>
>So how do you want them to talk? Coherency for 16S isn't trivial (look how long
>it took intel to get there), and if it's not coherent and you are using a network
>stack...then you still need an on-board switch of some sort.

ARM doesn't have something like HyperTransport, and the market is too small to be worth adding inter-socket coherency. So you could use an on-board switch and use an ethernet connection in each socket, or run the network stack on one socket, and DMA the packets to the relevant cores via a fast serial link.

>>Stacking 2 - 4 boards in a 1U rack is feasible
>>if you solder cores and DRAM on the board - no sockets, >heatsinks or DIMMS means
>>much more vertical space (I don't know whether the failure >rates of DRAM are low
>>enough for this to be cost effective - but avoiding >sockets/DIMMS should improve reliability).
>
>I think Arun mentioned that SGI did this for a lot of their large altix systems.

OK

>>Assuming a conservative 2 boards, 8 sockets, 4 cores per >socket, you could have
>>64 cores in a 1U rack (ie. 8 times more than a typical 2S >quad core today).
>
>That's still a lot of hypothetical infrastructure. 2S x86s exist today, and they will have 12 cores per socket soon.

It's an interesting thought experiment though - all it needs is a startup making it happen. How much power does a 12 core x86 chip use? Frequency would be below 2GHz for a reasonable power budget. Not being power limited allows ARM to close the performance gap even quicker.

Wilco