dmcq (dmcq.delete@this.fano.co.uk) on July 15, 2020 3:18 pm wrote:
> Michael S (already5chosen.delete@this.yahoo.com) on July 15, 2020 11:35 am wrote:
> > dmcq (dmcq.delete@this.fano.co.uk) on July 15, 2020 9:54 am wrote:
> > >
> > > Jeez. I'm not sure what you'd believe but how about Anandtech
> > >
> > > NVIDIA Volta Unveiled: GV100 GPU and Tesla V100 Accelerator Announced
> > >
> > > "Critically, NVLInk 2 also introduces cache coherency allowing the GPUs to be cache coherent
> > > with CPUs. Expect to see this play a big part in the eventual Power 9 + Volta systems."
> > >
> >
> > I read a little about it. It's more than typical PCIe "I/O coherence",
> > but less than "full cache coherence" found in CPU-based SMP.
> > A "more" part is that POWER9 can cache locations that physically reside in memory, attached to GV100.
> > A "less" part is that GV100 can not cache locations that physically reside in memory, attached to POWER9.
> > Due to "less" part the latency of normal POWER9 L3 miss is not impacted because of presence of
> > GV100. And that always was the biggest concern about off-die cache-coherent accelerators.
>
> Pages are migrated to the GPU if needed and the CPU can continue accessing them. There is also a
> mechanism for having them eventually migrate out again if the usage changes. The major thing is
> that one doesn't have to program memory transfers,

OS+drivers are doing it. In software. With help of various statistical counters. In principle, it's all exactly the same as on traditional "IO-coherent" PCIe. The only major difference is link bandwidth - 25 GB/s per/link/direction * 2 links to host vs ~15 GB/s per link/direction on PCIe3 a * 1 link. That certainly helps a lot, but is not related to cache-coherence.

> and barriers and atomics do whats expected of
> them. It is cache coherency no ifs or buts,

As long as one is not allowed to cache, barriers work, sure. Always worked.
Atomics? Like Fetch&add or CaS or LL/SC? I don't think that they work on GV100-to-hostmem accesses.
Looking at wikichips description I don't see how they could possibly work.
Anyway, I am not even sure that Volta has such instruction.

> it is efficiently supported by the hardware and programmers
> don't have to worry about it any more than they do for normal multitasking programs.
>

Performance profiles a vastly different from "normal multitasking programs". Even with huge bandwidth of NVlink, page migration from host memory to GPU memory or back is order(s) of magnitude slower than cache line migration in traditional SMP, even dual or quad-socket. May be, on SGI Altix-UV HPE Superdome Flex it is similarly slow, but very few programmers have experience with it.
So, programmers certainly have to worry about it, esp. when they define layout of their shared data structures.

> L3 misses is not a big problem when using a distributed directory based cache coherence which
> is what sysems use once they get to any size.

Not in IBM world. I am not 100% sure about POWER8/9, but from 4 to 7 cc worked on broadcasts. Not simple broadcasts that can be described in 2 sentences, but broadcasts nevertheless. Any directory is an optimization that favors "non-shared" case at cost of slowdown of "shared" case. In-memory ECC bit-stealing directory is probably the most radical at that. Such bias almost certainly hurts OLTP performance and OLTP was main thing for IBM POWER at least from 4 to 7. May be, still is.

> I can't really see why you've mentioned it.
>

May be, because it matters?

BTW, I wonder how they implement a "more" thing (i.e. cache-coherent access from POWER9 to GV100's HBM). Most likely GV100 maintains in-memory directory in HBM, likely through ECC bit-stealing, which allows them to handle most common "certainly non-shared" case of L2 misses without touching NVlinks. I can't think about anything else that wouldn't suck. But so far I found no confirmation and no details.