Data integrity of L1 caches

By: --- (---.delete@this.redheron.com), September 16, 2022 1:42 pm
Room: Moderated Discussions
dmcq (dmcq.delete@this.fano.co.uk) on September 16, 2022 1:41 pm wrote:
> --- (---.delete@this.redheron.com) on September 16, 2022 11:28 am wrote:
> > anon2 (anon.delete@this.anon.com) on September 15, 2022 7:04 pm wrote:
> > > Everybody knows the data integrity problems with parity protected write-back arrays. ECC has also
> > > seemed to be a difficult problem for L1 data cache that seems like nobody has solved very well.
> > >
> >
> > > What is expensive about L1 ECC which is less costly in
> > > L2? Keep in mind you need write-through, so L2 has to
> > > receive all the stores.
> >
> > ECC saves bits (relative to simpler options like replication) by performing math across
> > the ENTIRE line. This means that each time you modify an element of L1, something (whether
> > in the core itself or in the L1) will have to read the entire line (whether before or
> > after the write) to calculate the new ECC value. That's a lot of extra power.
> >
> > L2 is different because the entire line is written at once
> > from the L1 to the L2, so there's a one-time calculation.
> >
> > Now, could you avoid this by gathering writes in the store queue before sending them to L1?
> > To some extent yes, but
> > - you can't be sure that an entire line will be accumulated before there is some reason
> > to push the line out to L1. So you still have to have the above machinery.
> > - there are various constraints in the x86 memory model that limit the extent to which this
> > sort of store gathering is permitted. I don't know the details, but my understanding is that
> > the ARM memory model allows for much more aggressive gathering than is feasible under x86.
> >
> > I think (but don't trust this in the slightest!) that, for example, an unbroken sequence
> > of stores to the same line by x86 can be gathered, but if an out of sequence store is
> > then performed to a different line, the previous gathering has to terminate (and be single
> > unit to be pushed to L1) before a new gathering in that same line can begin.
> > Conversely ARM does not care about gathering that bounces around between multiple lines in any order; the
> > gathering in each line will continue till heuristics of whatever sort decide it's appropriate to write to
> > L1. [Of course barriers or snoops modify behavior in both cases, but I'm describing the basic case.]
>
> I think even ARM suffers that problem. As far as I can see because of lock free programming one has to assume
> most of what you are saying about the x86. I think it could be fixed if all lock free programming used load
> acquire store release for loads and stores. That would probably impact the lock free timing unfortunately
> but I think it would be worth it. Even better would have been lock free load and lock free store instructions
> for pointers to make the whole business obvious and take advantage of any difference.
>
> As far as I can see the ARM server designs use ECC for L1D and parity for L1C. They also offer
> ECC as an option for a lot of their other designs, in particular for their microcontrollers.

What exactly are you claiming?
ARM Ltd may well somewhat limit the amount of load/store reordering and aggregating they do relative to what the spec says, based on helping incorrect code to continue working.
But I don't think Apple cares in the slightest about helping incorrect code continue to work.

We've already had some cases of Asahi uncovering bugs in Linux that already existed but were previously not triggered by less aggressive CPUs. (Similar to finding bugs in SW that was supposedly page-size agnostic but...)

https://threadreaderapp.com/thread/1554031265728569344.html discusses this in general,

https://asahilinux.org/2021/03/progress-report-january-february-2021/ gives some particulars.

https://lore.kernel.org/linux-arm-kernel/90ea3e27-b2ef-41ac-75c7-5f0686603b2a@marcan.st/ gives a particular bug in load/store ordering semantics
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Data integrity of L1 cachesanon22022/09/15 06:04 PM
  Data integrity of L1 cachesGroo2022/09/15 10:46 PM
    Data integrity of L1 cachesanon22022/09/16 08:00 AM
      Data integrity of L1 cachesgroo2022/09/16 10:06 AM
        ECC outside critical path?hobold2022/09/16 12:03 PM
          ECC outside critical path?Mr. Camel2022/09/16 02:39 PM
            ECC outside critical path?anonymou52022/09/16 04:01 PM
          ECC outside critical path?anonymou52022/09/16 03:50 PM
            ECC outside critical path?hobold2022/09/17 05:57 AM
        Data integrity of L1 cachesanon22022/09/16 04:45 PM
  Data integrity of L1 cachesanon.12022/09/16 05:51 AM
    Data integrity of L1 cachesanon22022/09/16 08:04 AM
      Data integrity of L1 cachesBrett2022/09/16 11:12 AM
  Data integrity of L1 caches---2022/09/16 10:28 AM
    Data integrity of L1 cachesdmcq2022/09/16 12:41 PM
      Data integrity of L1 caches---2022/09/16 01:42 PM
    Data integrity of L1 cachesanon22022/09/16 04:49 PM
      Data integrity of L1 caches---2022/09/16 05:25 PM
        Read the thread (NT)anon22022/09/16 05:55 PM
        Data integrity of L1 cachesanon22022/09/16 05:57 PM
    Data integrity of L1 cachesMichael S2022/09/17 04:02 PM
  Data integrity of L1 cachesDavid Kanter2022/09/16 08:44 PM
    ECC word not necessarily full cache linePaul A. Clayton2022/09/17 09:59 AM
      ECC word not necessarily full cache lineDavid Kanter2022/09/18 11:29 AM
        ECC word not necessarily full cache lineAnon2022/09/18 11:54 AM
          ECC word not necessarily full cache linehobold2022/09/18 05:32 PM
            ECC word not necessarily full cache lineMichael S2022/09/19 07:47 AM
              ECC word not necessarily full cache linehobold2022/09/20 05:38 AM
                ECC word not necessarily full cache linedmcq2022/09/21 04:10 AM
                ECC word not necessarily full cache lineMichael S2022/09/21 05:55 AM
                  ECC word not necessarily full cache linehobold2022/09/21 12:59 PM
  Data integrity of L1 cachesDavid Hess2022/09/17 09:03 AM
  Data integrity of L1 cachesMichael S2022/09/17 04:12 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell tangerine? ūüćä