Context of ISA doesn't matter

By: Paul A. Clayton (paaronclayton.delete@this.gmail.com), February 26, 2021 1:03 pm
Room: Moderated Discussions
anon2 (anon.delete@this.anon.com) on February 22, 2021 7:17 pm wrote:
> blaine (myname.delete@this.acm.org) on February 22, 2021 9:37 am wrote:
[snip]
>> Well I can think of at least one area where a little complexity would be worth it to reduce SW complexity.
>> Optimal memcpy is often different between multiple implementations of the same ISA.
>
> Optimal-many-things are often different between multiple implementations of the same
> ISA, as well as otherwise identical implementations with an ISA difference.
>
> That's one problem, improved (but probably never really solved) with processor
> designs like OOOE, instructions, and some toolchain techniques.
>
> I don't think memcpy, memset instructions are bad per se, though I still don't understand
> the fascination with them, unless their proponents are going to move on to do-daxpy, route-ip-packet,
> gzip-memory, etc instructions when/if one day Intel's rep ; mov finally doesn't suck. But
> I digress, the point was not a totally open-ended "ISA does not matter".

memset and memcopy are relatively straightforward to improve in hardware (exploiting cache interface width, cache block granularity, etc.). These operations are semantically relatively simple even though they can include a lot of work.

While it would be quite possible for these operations to be detected as idioms, four instructions (store byte, increment address, decrement count, branch backwards four instructions if count is not zero) is somewhat long for idiom recognition. (Of course, one could add a preceding hint instruction "idiom follows: N instruction parcels" and even include more hint information such as being a "vectorizable" loop, the expected use of stores [e.g., this thread/core in near future in order of writing (a large copy/set might place a portion of the data in L1 and prepare prefetching from L2), other core (no point in allocating to L1), no expected reuse soon (send to main memory), or I/O (likely allocate to last level cache)], et al.)

Idioms that are not "guaranteed" to be recognized and optimized can become like specialized ("legacy") instructions that are almost pessimized. Compilers and library writers will choose alternatives, reducing the incentive to optimize such idioms (or specialized instructions).

(Mitch Alsup is considering whether his Virtual Vector Method eliminates the need for a separate memory copy instruction. VVM effectively provides a hint about the vectorization of a loop.)

The choice of what operations should be encoded as instructions, which should have dedicated idioms, which should be handled by an accelerator, and which should not be handled specially — this choice depends on expected use, expected benefit/cost, and marketing factors.

Obviously, lesser improvements of more commonly used (more primitive) operations can easily be more effective (generalization of Amdahl's law to fractional improvements).

I think Intel has marketing incentives to introduce instructions rather than idioms (such seems to encourage hardware upgrades) and rather than accelerators (which have less exclusivity and less compatibility guarantee). Intel emphasizes instruction set and compatibility.

> In context, Keller clearly did not intend that you could implement a literal 386/87
> CPU today that's competitve with M1, or x86 GPUs are right around the corner.

Yes, the context was specifically for "general purpose" compute targeting maximum (single-thread?) performance. Conventional ISAs are very similar in terms of what operations are performed (after decode) for "general purpose" code.

I would argue that AArch64 (and similar RISC-y ISAs) would provide a significant advantage for microarchitecture designers with no experience of any ISA's peculiarities and only basic familiarity with microarchitectural techniques. A bunch of graduate students could probably design a decent microarchitecture for AArch64 in a few years, but I think x86-64 design would benefit more from specific experience or at least very extensive knowledge.

x86-64 is also less friendly to smaller cores. This would seem mainly to be a power consumption issue for heterogeneous core designs. For microcontrollers, even AArch64 seems a bit "large".

Keller also noted that general purpose processors were doing significant work to convert serial instruction sequences into dynamic operation graphs. This hints that Explicit Data Graph Execution oriented interfaces might be better. His comments on data movement costs also hint that even EDGE is not focusing on an important factor (EDGE concentrates more on reducing control overheads than data/command communication overheads).

His comments on GPU's explicit parallelism and the advantage of further specialization for neural-inspired operations (as well as other comments) placed more emphasis on software.

I do not entirely agree with his statements that the future is in wasting computation because it is cheap and wasting computation for the sake of scaling (leading to higher total performance). While it is common wisdom that there is a trade-off between redundancy (both computation and storage) and scalability and that cost per bit/operation have decreased, I am not comfortable with reducing node peformance ten fold with the expectation of a hundred fold scaling and the value of ten fold lower time-to-solution justifying any extra costs (in large part because compute is cheap). However, I also recognize that I have excessive affection for micro-optimization and "just right" designs that ignore practical considerations (such as design effort or economics).

I did like his personel management comments that people are not all the same and that good managers exploit individual strengths (or at least develop hiring practices that consider cultural fit).
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
New Lex Fridman interview with Jim KellerJohnG2021/02/20 12:25 AM
  12:30 "[ISA] do not matter very much" (NT)Moritz2021/02/20 10:45 AM
    AAARGH, what are we going to argue about then? (NT)j2021/02/20 03:43 PM
    Blasphemy! (NT):]2021/02/21 05:49 AM
    12:30 "[ISA] do not matter very much"anon22021/02/21 11:25 PM
      12:30 "[ISA] do not matter very much"Brett2021/02/22 12:59 AM
        12:30 "[ISA] do not matter very much"Etienne Lorrain2021/02/22 02:17 AM
      12:30 "[ISA] do not matter very much"Dummond D. Slow2021/02/22 09:57 AM
        12:30 "[ISA] do not matter very much"Anon2021/02/22 11:52 AM
          12:30 "[ISA] do not matter very much"juanrga2021/02/22 12:01 PM
          12:30 "[ISA] do not matter very much"Mark Roulo2021/02/22 12:54 PM
          ARM being a good idea doesn't mean it would have worked for AMDDummond D. Slow2021/02/22 02:34 PM
            ARM being a good idea doesn't mean it would have worked for AMDAnon2021/02/22 04:25 PM
              ARM being a good idea doesn't mean it would have worked for AMDDummond D. Slow2021/02/22 05:55 PM
                ARM being a good idea doesn't mean it would have worked for AMDDoug S2021/02/23 01:03 PM
                  ARM being a good idea doesn't mean it would have worked for AMDDummond D. Slow2021/02/23 01:27 PM
                    ARM being a good idea doesn't mean it would have worked for AMDBrett2021/02/23 04:57 PM
                      3rd parties licensing ARM coresAnon2021/02/25 05:01 AM
                        3rd parties licensing ARM coresAnon2021/02/25 05:48 AM
                        3rd parties licensing ARM coresdmcq2021/02/25 07:01 AM
                          3rd parties licensing ARM coresDummond D. Slow2021/02/25 10:17 AM
                            3rd parties licensing ARM coresAnon2021/02/25 11:11 AM
                              3rd parties licensing ARM coresAnon2021/02/26 03:54 AM
                              3rd parties licensing ARM coresDummond D. Slow2021/02/26 11:01 AM
            ARM being a good idea doesn't mean it would have worked for AMDLinus Torvalds2021/02/22 06:06 PM
              ARM being a good idea doesn't mean it would have worked for AMDDummond D. Slow2021/02/22 08:19 PM
              ARM being a good idea doesn't mean it would have worked for AMDanon22021/02/22 08:28 PM
              ARM being a good idea doesn't mean it would have worked for AMDdmcq2021/02/23 06:35 AM
                ARM being a good idea doesn't mean it would have worked for AMDJukka Larja2021/02/23 08:12 AM
                  ARM being a good idea doesn't mean it would have worked for AMDSimon Farnsworth2021/02/23 09:42 AM
                    ARM being a good idea doesn't mean it would have worked for AMDJukka Larja2021/02/24 07:03 AM
                ARM may have been a threat to Intelwumpus2021/02/23 09:30 AM
      12:30 "[ISA] do not matter very much"blaine2021/02/22 10:37 AM
        12:30 "[ISA] do not matter very much"anon22021/02/22 08:17 PM
          12:30 "[ISA] do not matter very much"Anon2021/02/23 04:05 AM
            12:30 "[ISA] do not matter very much"Wilco2021/02/23 04:48 AM
              12:30 "[ISA] do not matter very much"Bigos2021/02/23 04:55 AM
                12:30 "[ISA] do not matter very much"Wilco2021/02/23 05:15 AM
                  12:30 "[ISA] do not matter very much"Bigos2021/02/23 06:16 AM
                12:30 "[ISA] do not matter very much"Travis Downs2021/02/27 12:46 AM
              12:30 "[ISA] do not matter very much"Anon2021/02/23 07:26 AM
                12:30 "[ISA] do not matter very much"anon22021/02/23 05:35 PM
                  12:30 "[ISA] do not matter very much"Anon2021/02/24 08:57 AM
                12:30 "[ISA] do not matter very much"Wilco2021/02/24 05:37 AM
                  12:30 "[ISA] do not matter very much"Etienne Lorrain2021/02/24 07:24 AM
                    12:30 "[ISA] do not matter very much"Anon2021/02/24 09:11 AM
                    12:30 "[ISA] do not matter very much"rwessel2021/02/24 09:45 AM
                      12:30 "[ISA] do not matter very much"Etienne Lorrain2021/02/25 02:02 AM
                        12:30 "[ISA] do not matter very much"rwessel2021/02/25 05:51 AM
                        12:30 "[ISA] do not matter very much"Anon2021/02/25 05:53 AM
                  12:30 "[ISA] do not matter very much"Anon2021/02/24 09:07 AM
                    12:30 "[ISA] do not matter very much"Wilco2021/02/24 12:37 PM
                      runtime selection vs. heterogenous cores?Matt Sayler2021/02/24 07:10 PM
                        runtime selection vs. heterogenous cores?Wilco2021/02/26 06:22 AM
            12:30 "[ISA] do not matter very much"anon22021/02/23 05:20 AM
              12:30 "[ISA] do not matter very much"Anon2021/02/23 07:21 AM
                12:30 "[ISA] do not matter very much"none2021/02/23 08:37 AM
                  12:30 "[ISA] do not matter very much"rwessel2021/02/23 10:44 AM
                    12:30 "[ISA] do not matter very much"anon22021/02/23 05:30 PM
                      12:30 "[ISA] do not matter very much"Anon2021/02/24 09:25 AM
                        12:30 "[ISA] do not matter very much"anon.12021/02/25 07:13 AM
                  12:30 "[ISA] do not matter very much"Anon2021/02/24 09:44 AM
                12:30 "[ISA] do not matter very much"anon22021/02/23 04:51 PM
                  12:30 "[ISA] do not matter very much"Anon2021/02/24 09:31 AM
            12:30 "[ISA] do not matter very much"vvid2021/02/23 07:41 AM
              12:30 "[ISA] do not matter very much"Michael S2021/02/23 09:52 AM
                12:30 "[ISA] do not matter very much"rwessel2021/02/23 10:33 AM
                  12:30 "[ISA] do not matter very much"Linus Torvalds2021/02/23 12:44 PM
                    12:30 "[ISA] do not matter very much"rwessel2021/02/23 01:21 PM
                      12:30 "[ISA] do not matter very much"Linus Torvalds2021/02/23 01:30 PM
                        12:30 "[ISA] do not matter very much"Andrey2021/02/25 04:06 AM
                          12:30 "[ISA] do not matter very much"Anon2021/02/25 06:04 AM
                            12:30 "[ISA] do not matter very much"Andrey2021/02/25 06:54 AM
                              12:30 "[ISA] do not matter very much"Anon2021/02/25 07:33 AM
                          12:30 "[ISA] do not matter very much"Linus Torvalds2021/02/25 11:35 AM
                            12:30 "[ISA] do not matter very much"Andrey2021/02/25 02:34 PM
                              12:30 "[ISA] do not matter very much"Etienne Lorrain2021/02/26 02:18 AM
                              12:30 "[ISA] do not matter very much"dmcq2021/02/26 04:23 PM
                12:30 "[ISA] do not matter very much"Anon2021/02/24 09:45 AM
              12:30 "[ISA] do not matter very much"Gabriele Svelto2021/02/23 10:15 AM
          Context of ISA doesn't matterPaul A. Clayton2021/02/26 01:03 PM
  Is there a text version? (NT)Foo_2021/02/20 05:33 PM
    good question (NT)Michael S2021/02/21 05:31 AM
    Is there a text version?:]2021/02/21 11:34 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?