By: jp (jipe4153.delete@this.gmail.com), July 28, 2012 11:04 am
Room: Moderated Discussions
aaron spink (aaronspink.delete@this.notearthlink.net) on July 28, 2012 7:05 am wrote:
> Emil Briggs (me.delete@this.nowherespam.com) on July 28, 2012 6:40 am
> wrote:
>
> > By ORNL do you mean Oak Ridge National Laboratory? I ask
> >
> since I am a large user at ORNL. Currently only some of the Jaguar/Titan nodes
>
> > are equipped with GPU's but there are enough of them installed to do
> realistic
> > evaluations. For certain workloads (and when properly
> programmed) they beat
> > CPU's pretty handily performance wise. And that
> data comes from a real world
> > application not LINPACK. The section of the
> code that I adapted for GPU's runs 3
> > to 4 times faster than it does on
> CPU's. It's also possible with this particular
> > application to overlap some
> operations on the CPU and GPU and hide the latency
> > of PCI-E data
> transfers. Obviously not all applications can benefit in the same
> > way and
> it's not easy to do so even when possible but GPU's can offer some very
> >
> nice performance gains in some cases.
> >
>
> I'm not denying that there are
> some workloads and some kernels that have an advantage, I am however saying that
> it is generally the exception based on data published from both PRACE and ORNL,
> et al.
>
> BTW, what % of peak are you seeing on the GPUs with your code?
>
> >
> That being said I do think that the
> > cost of moving data across the PCI-E
> bus and the difficulty of the programming
> > model are some real downsides
> to GPU's. How all that plays out will be
> > interesting and I'm looking
> forward to getting my hands on some Intel MIC
> > hardware to see what we can
> do with it.
> >
>
> MIC still has the disadvantage of PCI-E being a limiter but
> it should have a better programming model overall than GPUs. Hopefully we'll
> eventually see x32/x40 PCI-E interfaces or dual QPI interfaces. Certainly would
> be nice if the GPUs/MICs had simple coherent access to memory.
Nvidia GPUs do have simple coherent access to memory. For example data allocated on the CPU (host side) can be seamlessly pulled over to the GPU kernel ( just by passing a pointer). It's more a question of PCIe speed than ease of use.
But on a majority of the real world applications I've worked on the computational time exceeds the copy_to and copy_from time meaning that you can hide almost all of the transfer time by overlapping compute & transfer.
So to my experience the PCIe has not been a real limiter.... And according to bias little me the programming model isn't very difficult compared to trying to write a highly optimized multi threaded CPU application utilizing SSE. It's just that CUDA/OpenCL doesn't allow you to write stupid code to begin with, you get punished immediately for that.
In the future we'll have ARM+GPU (ie Maxwell) on the same die which means the discrete graphics cards would be able to also handle the strictly serial components of the algorithms, the big gain here being that a closely coupled GPU+CPU (a bit like fusion) via an interconnect or L3 cache would mean that even smaller workloads could be offloaded. This is border line speculation but it does seem to be on the horizon.
> Emil Briggs (me.delete@this.nowherespam.com) on July 28, 2012 6:40 am
> wrote:
>
> > By ORNL do you mean Oak Ridge National Laboratory? I ask
> >
> since I am a large user at ORNL. Currently only some of the Jaguar/Titan nodes
>
> > are equipped with GPU's but there are enough of them installed to do
> realistic
> > evaluations. For certain workloads (and when properly
> programmed) they beat
> > CPU's pretty handily performance wise. And that
> data comes from a real world
> > application not LINPACK. The section of the
> code that I adapted for GPU's runs 3
> > to 4 times faster than it does on
> CPU's. It's also possible with this particular
> > application to overlap some
> operations on the CPU and GPU and hide the latency
> > of PCI-E data
> transfers. Obviously not all applications can benefit in the same
> > way and
> it's not easy to do so even when possible but GPU's can offer some very
> >
> nice performance gains in some cases.
> >
>
> I'm not denying that there are
> some workloads and some kernels that have an advantage, I am however saying that
> it is generally the exception based on data published from both PRACE and ORNL,
> et al.
>
> BTW, what % of peak are you seeing on the GPUs with your code?
>
> >
> That being said I do think that the
> > cost of moving data across the PCI-E
> bus and the difficulty of the programming
> > model are some real downsides
> to GPU's. How all that plays out will be
> > interesting and I'm looking
> forward to getting my hands on some Intel MIC
> > hardware to see what we can
> do with it.
> >
>
> MIC still has the disadvantage of PCI-E being a limiter but
> it should have a better programming model overall than GPUs. Hopefully we'll
> eventually see x32/x40 PCI-E interfaces or dual QPI interfaces. Certainly would
> be nice if the GPUs/MICs had simple coherent access to memory.
Nvidia GPUs do have simple coherent access to memory. For example data allocated on the CPU (host side) can be seamlessly pulled over to the GPU kernel ( just by passing a pointer). It's more a question of PCIe speed than ease of use.
But on a majority of the real world applications I've worked on the computational time exceeds the copy_to and copy_from time meaning that you can hide almost all of the transfer time by overlapping compute & transfer.
So to my experience the PCIe has not been a real limiter.... And according to bias little me the programming model isn't very difficult compared to trying to write a highly optimized multi threaded CPU application utilizing SSE. It's just that CUDA/OpenCL doesn't allow you to write stupid code to begin with, you get punished immediately for that.
In the future we'll have ARM+GPU (ie Maxwell) on the same die which means the discrete graphics cards would be able to also handle the strictly serial components of the algorithms, the big gain here being that a closely coupled GPU+CPU (a bit like fusion) via an interconnect or L3 cache would mean that even smaller workloads could be offloaded. This is border line speculation but it does seem to be on the horizon.
