Notes on Graviton 2 store behavior

By: Travis Downs (travis.downs.delete@this.gmail.com), February 24, 2021 12:36 am
Room: Moderated Discussions
I have been playing around with Graviton 2 a bit and was trying to characterize its load-store performance.

For stores, I found the following:

1) It can apparently execute 2 stores per cycle (I believe this has been reported).

2) It can sustain a write to at most one unique aligned 32-byte chunk per cycle: however, several program writes can be satisfied in the same cycle. Writes going to a 32-byte chunk don't have to be contiguous in address (space) or in the store sequence (time). So, for example, a series of stores with 32-byte stride will result in 1 store/cycle, rather than 2, because of this limit. Stride 16 (or less) can achieve 2 stores per cycle, however. A pattern like 0, 32, 8, 40 (then repeated with all offsets increased by 64 bytes each time) can also achieve 2 stores per cycle since that's 4 stores across 2 32B chunks, even though the stores are not contiguous (space or time). Stores to a 32B chunk cannot be combined across (in time) a store to a different cache line.

3) Consecutive (in time) stores to different cache lines can commit to L1D at 1 cycle per store, at best. On the other hand if two consecutive (in time) stores are to the same cache line, they can commit in the same cycle. This is similar to Intel Ice Lake behavior. So more than one store per cycle can only be achieved if consecutive stores at least sometimes fall into the same cache line. Only two stores can be committed this way per cycle, not three or more (tested by a 3-1 pattern of stores to two lines). So an offset pattern like 0, 64, 0, 64 will take 1 cycle per store because each adjacent store is to a different line.

4) Misaligned stores appear to be handled reasonably well: in some cases two can execute and commit per cycle, although in most cases the throughput is somewhat lower, perhaps due to bank conflicts (the pattern was suggestive of bank conflicts). However, in most cases the throughput is still better than 1 per cycle even after conflicts. Misaligned stores which cross a cache line have a best-case throughput of 1 per cycle.

5) Even aligned stores sometimes suffer (apparent) bank conflicts or some other penalty. The penalty looks like 4 cycles. As an example, repeated stores at offsets 0, 8, 0, 8, ... to the same line take 1.5 cycles each rather than the expected 0.5.
 Next Post in Thread >
TopicPosted ByDate
Notes on Graviton 2 store behaviorTravis Downs2021/02/24 12:36 AM
  Notes on Graviton 2 store behaviorAdrian2021/02/24 03:06 AM
  Notes on Graviton 2 store behaviornone2021/02/24 06:37 AM
    Notes on Graviton 2 store behavioranon2021/02/24 07:30 AM
      Notes on Graviton 2 store behaviorTravis Downs2021/02/24 12:59 PM
  Notes on Graviton 2 store behaviorJeff S.2021/02/24 10:15 AM
    Notes on Graviton 2 store behaviorJon Masters2021/02/25 12:16 AM
    Notes on Graviton 2 store behaviordmcq2021/02/25 07:21 AM
      Notes on Graviton 2 store behaviorJon Masters2021/02/25 10:34 AM
      Notes on Graviton 2 store behaviorJeff S.2021/02/25 11:55 AM
        Notes on Graviton 2 store behaviordmcq2021/02/25 05:36 PM
          Notes on Graviton 2 store behaviorJeff S.2021/02/26 08:55 AM
            Notes on Graviton 2 store behaviorJon Masters2021/02/26 10:41 AM
              Notes on Graviton 2 store behaviorJon Masters2021/02/28 04:07 PM
                Notes on Graviton 2 store behaviordmcq2021/02/28 04:54 PM
        Notes on Graviton 2 store behaviordmcq2021/02/25 06:02 PM
    Notes on Graviton 2 store behaviorTravis Downs2021/02/26 10:05 PM
      Notes on Graviton 2 store behaviorLinus Torvalds2021/02/27 12:50 PM
        Notes on Graviton 2 store behaviorTravis Downs2021/02/28 12:22 AM
          Notes on Graviton 2 store behaviorLinus Torvalds2021/02/28 12:44 PM
            Notes on Graviton 2 store behaviorAnon2021/02/28 01:15 PM
              Notes on Graviton 2 store behavioranon2021/02/28 02:20 PM
                Notes on Graviton 2 store behaviorTravis Downs2021/03/01 12:28 AM
                  Notes on Graviton 2 store behavioranon2021/03/01 03:42 AM
            Notes on Graviton 2 store behaviorGabriele Svelto2021/02/28 01:36 PM
              Notes on Graviton 2 store behaviorJon Masters2021/02/28 07:42 PM
            Notes on Graviton 2 store behavioranon22021/02/28 05:32 PM
              Notes on Graviton 2 store behaviorJon Masters2021/02/28 07:43 PM
            Notes on Graviton 2 store behaviorTravis Downs2021/03/01 12:44 AM
      Notes on Graviton 2 store behaviorJeff S.2021/02/28 10:22 AM
        Notes on Graviton 2 store behaviorTravis Downs2021/03/01 12:08 AM
          Notes on Graviton 2 store behaviordmcq2021/03/01 07:31 AM
            Notes on Graviton 2 store behaviorTravis Downs2021/03/01 10:54 PM
          Notes on Graviton 2 store behaviorJeff S.2021/03/01 10:21 AM
        Notes on Graviton 2 store behaviorJon Masters2021/03/01 10:25 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?