Wilco (Wilco.Dijkstra@ntlworld.com) on 9/19/09 wrote:
---------------------------
>David Kanter (dkanter@realworldtech.com) on 9/19/09 wrote:
>---------------------------
>>Wilco (Wilco.Dijkstra@ntlworld.com) on 9/19/09 wrote:
>>---------------------------
>>>David Kanter (dkanter@realworldtech.com) on 9/18/09 wrote:
>>>---------------------------
>>>>To achieve the performance of a 2S Nehalem, you're going to need a ton of ARMs.
>>>>And they all need to have ethernet to communicate. Those cables are not free and
>>>>use space and power, as do the switches they connect to.
>>>
>>>The current Cortex-A9 doesn't compete with Nehalem indeed. However the gap with
>>>x86 is closing fast, and I'd expect the next generation to be far closer.
>>>
>>>What low power enables is packing many sockets in a single rack - 8-16 sockets
>>>should be possible. As long as they can talk to each other, you don't need more
>>>than 1 network connection per board.
>>
>>So how do you want them to talk? Coherency for 16S isn't trivial (look how long
>>it took intel to get there), and if it's not coherent and you are using a network
>>stack...then you still need an on-board switch of some sort.
>
>ARM doesn't have something like HyperTransport, and the market is too small to
>be worth adding inter-socket coherency.

Agreed.

>So you could use an on-board switch and
>use an ethernet connection in each socket, or run the >network stack on one socket,
>and DMA the packets to the relevant cores via a fast >serial link.

So basically you are thinking of using something like XAUI between chips?

I guess the thing that makes me leery is that network overhead may not be small for an ARM, but who knows.

>>>Stacking 2 - 4 boards in a 1U rack is feasible
>>>if you solder cores and DRAM on the board - no sockets, >heatsinks or DIMMS means
>>>much more vertical space (I don't know whether the failure >rates of DRAM are low
>>>enough for this to be cost effective - but avoiding >sockets/DIMMS should improve reliability).
>>
>>I think Arun mentioned that SGI did this for a lot of their large altix systems.
>
>OK
>
>>>Assuming a conservative 2 boards, 8 sockets, 4 cores per >socket, you could have
>>>64 cores in a 1U rack (ie. 8 times more than a typical 2S >quad core today).
>>
>>That's still a lot of hypothetical infrastructure. 2S x86s exist today, and they will have 12 cores per socket soon.
>
>It's an interesting thought experiment though - all it >needs is a startup making
>it happen.

Right...so my opinion is that whoever funds a start up like would be crazy.

You can't go for HPC, because NV and ATI will just crush you in efficiency.

You're probably stuck going after the same markets that Afara was, and they didn't seem to do so well. Admittedly, you can go for a lower budget since you aren't taping out a chip.

There are probably a couple of companies doing what you're suggesting, but I don't know if they are using ARM.

> How much power does a 12 core x86 chip use? Frequency >would be below
>2GHz for a reasonable power budget. Not being power >limited allows ARM to close the performance gap even >quicker.

AMD will probably hit 2GHz, and keep in mind, the cores are quite beefy there. Not sure how they compare in specint to ARM.

David