By: Michael S (already5chosen.delete@this.yahoo.com), November 12, 2014 2:53 am
Room: Moderated Discussions
Ronald Maas (rmaas.delete@this.wiwo.nl) on November 11, 2014 4:21 pm wrote:
> Michael S (already5chosen.delete@this.yahoo.com) on November 11, 2014 9:23 am wrote:
> > Huge memory capacity of Xeon-E5 is important for other applications, but for HPC capacity of Xeon-E3
> > is probably sufficient. Taking as example, TSUBAME-KFC that you mentioned earlier, it has 2,560 GB
> > of RAM = 64 GB per node = 32 GB per socket = 16 GB per GPU. Xeon-E3 can address 32 GB per socket.
> >
> > As to bandwidth, if we build system with 1 GPU per Xeon-E3 then it will have the
> > same theoretical peak CPU-side bandwidth per GPU as TSUBAME-KFC, but slightly higher
> > practically achievable bandwidth, due to lower latency of unbuffered DIMMs.
> >
> > I think, that it's not memory capacity or bandwidth that keeps Xeon-E3 away from Top500,
> > but what I said in my previous post - bigger nodes are simply more economical.
> >
> > BTW, you say that XG1 can address vastly more memory compared to Xeon E3. I don't know if it is true in
> > general, but XG1-based HP ProLiant m400 supports only 64
> > GB, which is twice more than Xeon-E3 and AMD Opteron
> > X1150, the same as Intel Atom C2730 and many times less than even the most castrated Xeon-E5.
> >
> >
> >
>
> Agree both technical and economic factors ultimitately dictate if ARM adoption for GPU based
> supercomputers will actually happen or not.
It will happen. But on NVidea terms.
> But the fact HPC solution builders like Cirrascale
> and E4 are clearly on board with XG1, give at least some credibility to the idea.
I think, they are taking dangerous bet.
Intel is integrating COMM functionality into KNL. NVidea has no choice, but to follow the suite either in their next Tesla or in one after next. When they integrate IB they will undoubtedly integrate few Arm64 cores as well. And if we are going to believe NV PR, they not just forced into this route, they actually want to go there.
So, everybody who is now heavily investing in multichip Tesla+ARM SOC+IB solution will find themselves in worse position, financially, than those who waited for integrated NV solution and in the mean time were selling safe Xeon-E5+Tesla+IB-on-PCIe combos.
>
> XG1 with 4 memory channels supports 512 GB according to http://arxiv.org/pdf/1410.3441.pdf
I would not consider this article a reliable source for technical capabilities X-Gene.
If they were saying 384 GB, I'd scratch myself into believing, but 512 GB sounds like they simply do not know what they are talking about.
Anyway, max. memory capacity is not a bottleneck for typical HPC node.
>
> Ronald
> Michael S (already5chosen.delete@this.yahoo.com) on November 11, 2014 9:23 am wrote:
> > Huge memory capacity of Xeon-E5 is important for other applications, but for HPC capacity of Xeon-E3
> > is probably sufficient. Taking as example, TSUBAME-KFC that you mentioned earlier, it has 2,560 GB
> > of RAM = 64 GB per node = 32 GB per socket = 16 GB per GPU. Xeon-E3 can address 32 GB per socket.
> >
> > As to bandwidth, if we build system with 1 GPU per Xeon-E3 then it will have the
> > same theoretical peak CPU-side bandwidth per GPU as TSUBAME-KFC, but slightly higher
> > practically achievable bandwidth, due to lower latency of unbuffered DIMMs.
> >
> > I think, that it's not memory capacity or bandwidth that keeps Xeon-E3 away from Top500,
> > but what I said in my previous post - bigger nodes are simply more economical.
> >
> > BTW, you say that XG1 can address vastly more memory compared to Xeon E3. I don't know if it is true in
> > general, but XG1-based HP ProLiant m400 supports only 64
> > GB, which is twice more than Xeon-E3 and AMD Opteron
> > X1150, the same as Intel Atom C2730 and many times less than even the most castrated Xeon-E5.
> >
> >
> >
>
> Agree both technical and economic factors ultimitately dictate if ARM adoption for GPU based
> supercomputers will actually happen or not.
It will happen. But on NVidea terms.
> But the fact HPC solution builders like Cirrascale
> and E4 are clearly on board with XG1, give at least some credibility to the idea.
I think, they are taking dangerous bet.
Intel is integrating COMM functionality into KNL. NVidea has no choice, but to follow the suite either in their next Tesla or in one after next. When they integrate IB they will undoubtedly integrate few Arm64 cores as well. And if we are going to believe NV PR, they not just forced into this route, they actually want to go there.
So, everybody who is now heavily investing in multichip Tesla+ARM SOC+IB solution will find themselves in worse position, financially, than those who waited for integrated NV solution and in the mean time were selling safe Xeon-E5+Tesla+IB-on-PCIe combos.
>
> XG1 with 4 memory channels supports 512 GB according to http://arxiv.org/pdf/1410.3441.pdf
I would not consider this article a reliable source for technical capabilities X-Gene.
If they were saying 384 GB, I'd scratch myself into believing, but 512 GB sounds like they simply do not know what they are talking about.
Anyway, max. memory capacity is not a bottleneck for typical HPC node.
>
> Ronald
