Michael S (already5chosen.delete@this.yahoo.com) on July 16, 2020 1:17 am wrote:
> dmcq (dmcq.delete@this.fano.co.uk) on July 15, 2020 3:18 pm wrote:
> > Michael S (already5chosen.delete@this.yahoo.com) on July 15, 2020 11:35 am wrote:
> > > dmcq (dmcq.delete@this.fano.co.uk) on July 15, 2020 9:54 am wrote:
> > > >
> > > > Jeez. I'm not sure what you'd believe but how about Anandtech
> > > >
> > > > NVIDIA Volta Unveiled: GV100 GPU and Tesla V100 Accelerator Announced
> > > >
> > > > "Critically, NVLInk 2 also introduces cache coherency allowing the GPUs to be cache coherent
> > > > with CPUs. Expect to see this play a big part in the eventual Power 9 + Volta systems."
> > > >
> > >
> > > I read a little about it. It's more than typical PCIe "I/O coherence",
> > > but less than "full cache coherence" found in CPU-based SMP.
> > > A "more" part is that POWER9 can cache locations that physically reside in memory, attached to GV100.
> > > A "less" part is that GV100 can not cache locations that physically reside in memory, attached to POWER9.
> > > Due to "less" part the latency of normal POWER9 L3 miss is not impacted because of presence of
> > > GV100. And that always was the biggest concern about off-die cache-coherent accelerators.
> >
> > Pages are migrated to the GPU if needed and the CPU can continue accessing them. There is also a
> > mechanism for having them eventually migrate out again if the usage changes. The major thing is
> > that one doesn't have to program memory transfers,
>
> OS+drivers are doing it. In software. With help of various statistical counters. In principle,
> it's all exactly the same as on traditional "IO-coherent" PCIe. The only major difference is
> link bandwidth - 25 GB/s per/link/direction * 2 links to host vs ~15 GB/s per link/direction
> on PCIe3 a * 1 link. That certainly helps a lot, but is not related to cache-coherence.
>
> > and barriers and atomics do whats expected of
> > them. It is cache coherency no ifs or buts,
>
> As long as one is not allowed to cache, barriers work, sure. Always worked.
> Atomics? Like Fetch&add or CaS or LL/SC? I don't think that they work on GV100-to-hostmem accesses.
> Looking at wikichips description I don't see how they could possibly work.
> Anyway, I am not even sure that Volta has such instruction.
>
> > it is efficiently supported by the hardware and programmers
> > don't have to worry about it any more than they do for normal multitasking programs.
> >
>
> Performance profiles a vastly different from "normal multitasking programs". Even with huge bandwidth
> of NVlink, page migration from host memory to GPU memory or back is order(s) of magnitude slower
> than cache line migration in traditional SMP, even dual or quad-socket. May be, on SGI Altix-UV
> HPE Superdome Flex it is similarly slow, but very few programmers have experience with it.
> So, programmers certainly have to worry about it, esp. when
> they define layout of their shared data structures.
>
> > L3 misses is not a big problem when using a distributed directory based cache coherence which
> > is what sysems use once they get to any size.
>
> Not in IBM world. I am not 100% sure about POWER8/9, but from 4 to 7 cc worked on broadcasts. Not
> simple broadcasts that can be described in 2 sentences, but broadcasts nevertheless. Any directory
> is an optimization that favors "non-shared" case at cost of slowdown of "shared" case. In-memory
> ECC bit-stealing directory is probably the most radical at that. Such bias almost certainly hurts
> OLTP performance and OLTP was main thing for IBM POWER at least from 4 to 7. May be, still is.
>
> > I can't really see why you've mentioned it.
> >
>
> May be, because it matters?
>
> BTW, I wonder how they implement a "more" thing (i.e. cache-coherent access from POWER9 to GV100's HBM).
> Most likely GV100 maintains in-memory directory in HBM, likely through ECC bit-stealing, which allows
> them to handle most common "certainly non-shared" case of L2 misses without touching NVlinks. I can't
> think about anything else that wouldn't suck. But so far I found no confirmation and no details.

I can only assume you've only looked at Volta + PCIe as used on x86 and not the Volta + NVLink connection to he CPU. Please see this short presentation hwhich has a cople of slides on the difference.

NVidia Volta Architecture

Directory based coherence doesn't rule out snooping locally. As to ECC bit stealing I just can't see IBM wanting to go anywhere near bargepole distance of anything like that!