M1 GPU has higher FP32 than A-series GPUs

By: Adrian (a.delete@this.acm.org), December 9, 2020 6:37 am
Room: Moderated Discussions
K.K (anon.delete@this.anon.com) on December 9, 2020 5:01 am wrote:
> Adrian (a.delete@this.acm.org) on December 9, 2020 4:28 am wrote:
> > Regarding "So no ALU fusion, but ALU "reuse" (if it makes
> > any sense)", in the case when you have 1:2 speed rate
> > ratios, like for Apple A14, then it is certain that you have a FP32 ALU that is split into 2 FP16 ALUs.
> >
> > For the adder parts of the ALUs, using either the words "fusion" or "splitting" is irrelevant.
> > You just have 32 1-bit adders and they are reconfigured between FP16 and FP32 just by
> > changing the ways that the carries are propagated. The increased complexity of 1 FP32
> > adder over 2 FP16 adders (e.g. due to carry lookahead) is negligible.
> >
> > However, 1 multiplier for FP32 has a larger size than 2 multipliers for FP16 (with more 1-bit adders, up to
> > double, for 24-bit FP32 fractions vs. 11/12-bit FP16 fractions).
> > So, when 2 FPU16 ALUs are fused, the smaller
> > combined FP16 multipliers would have to be used twice, to
> > generate partial products, so you would see a larger
> > speed rate ratio, e.g. of 1:4, between FP32 and FP16, because
> > you have only half as many FP32 ALUs after fusion
> > and each ALU would need at least 2 cycles to generate the result for an FMA or FMUL operation.
>
> Thanks, that is very interesting!
>
> How were AMD and Nvidia doing it before they introduced dual-rate FP16 support? That is a fairly
> recent change if I am not mistaken, but those GPUs supported FP16 data since forever.
>
> Is it possible that Apple supply went from fused FP16 (2xFP16 = 1xFP32) ALUs to just FP32 ALUs,
> and completely dropped FP16 support from the M1? That would mean that the calculation is always
> performed with 32-bit precision, but data gets converted as necessary when entering/leaving the
> ALU (i.e. a 16-bit operand is widened to 32-bit, operated upon with 32-bit precision and then
> rounded to 16-bit). Would there be a way to test this? My numerics is rather dusty, but I suppose
> that performing an operation with FP32 precision and then rounding it down to FP16 will result
> in the same accuracy as performing the operation with FP16 precision from the start...
>
>

Before dual-rate FP16 was supported, FP16 was done like it probably is done in M1.

The bits that compose an FP16 number are a subset of those from an FP32 number. Therefore they are routed towards their correct positions at the inputs, while the other bit positions are masked. At the output, the rounding and the overflow/underflow checks are modified, to account for the reduced precision and range, then the output bits are routed to the FP16 positions.


Performing a sequence of FP32 operations with a final FP16 rounding will give very different results from the same sequence of FP16 operations.

When the operations are designated as FP16, rounding down to FP16 is mandatory after each operation, because you could store it and look at it later, expecting the correctly defined result.


With the exception of extremely unlikely cases of error cancellation, the result of a sequence of FP16 operations will have a much larger error than when using intermediate FP32 operations. The final error is around the error due to the number of bits used for magnitude, multiplied by the number of operations. So you will need to do many thousands of dependent FP32 operations to accumulate the same error as the error introduced by a single FP16 operation.



Even on GPUs where FP16 operations have the same rate as FP32, there are 3 reasons that could recommend the choice of FP16:

1. Less memory needed for large arrays. However the arrays could be stored as FP16, but the computations could be done as FP32, to reduce the errors.

2. Lower power consumption. Because the FP32 ALU has many bits masked when doing FP16, many gates will not be switching and they will not consume power.

3. When testing programs intended to be deployed on devices where FP16 operations are faster.








< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
M1 GPU microbenchmarks (peak FLOPS)K.K2020/12/05 03:31 AM
  M1 GPU microbenchmarks (peak FLOPS)K.K.2020/12/05 03:57 AM
  M1 GPU microbenchmarks (peak FLOPS)Adrian2020/12/05 04:00 AM
  M1 GPU microbenchmarks (peak FLOPS)Chester2020/12/05 10:39 AM
    M1 GPU microbenchmarks (peak FLOPS)K.K2020/12/06 03:46 AM
  M1 GPU microbenchmarks (peak FLOPS)Maynard Handley2020/12/05 12:12 PM
  M1 GPU has higher FP32 than A-series GPUsAndrei F2020/12/06 03:24 AM
    M1 GPU has higher FP32 than A-series GPUsK.K2020/12/06 03:49 AM
    M1 GPU has higher FP32 than A-series GPUsJeff S.2020/12/06 11:12 PM
      M1 GPU has higher FP32 than A-series GPUsK.K.2020/12/07 02:32 AM
        M1 GPU has higher FP32 than A-series GPUsJeff S.2020/12/07 10:29 PM
          M1 GPU has higher FP32 than A-series GPUsK.K.2020/12/09 01:35 AM
            M1 GPU has higher FP32 than A-series GPUsAdrian2020/12/09 02:36 AM
              M1 GPU has higher FP32 than A-series GPUsK.K.2020/12/09 02:59 AM
                M1 GPU has higher FP32 than A-series GPUsAdrian2020/12/09 03:16 AM
                  M1 GPU has higher FP32 than A-series GPUsK.K2020/12/09 04:56 AM
                    M1 GPU has higher FP32 than A-series GPUsAdrian2020/12/09 05:28 AM
                      M1 GPU has higher FP32 than A-series GPUsK.K2020/12/09 06:01 AM
                        M1 GPU has higher FP32 than A-series GPUsAdrian2020/12/09 06:37 AM
                          and beyond just the flops...anonymou52020/12/09 02:08 PM
                            and beyond just the flops...Doug S2020/12/09 03:06 PM
                              and beyond just the flops...Maynard Handley2020/12/09 04:05 PM
                                and beyond just the flops...Adrian2020/12/09 05:36 PM
                                and beyond just the flops...anonymou52020/12/09 08:28 PM
                                  and beyond just the flops...Maynard Handley2020/12/09 08:40 PM
                                    and beyond just the flops...anonymou52020/12/10 02:18 AM
                                      and beyond just the flops...anonymou52020/12/10 02:51 AM
                                      and beyond just the flops...Maynard Handley2020/12/10 09:53 AM
                                        and beyond just the flops...Megol2020/12/10 10:57 AM
                                          and beyond just the flops...Maynard Handley2020/12/10 12:16 PM
                                            and beyond just the flops...anonymou52020/12/10 04:12 PM
                                              and beyond just the flops...anonymou52020/12/10 04:24 PM
                                                and beyond just the flops...Maynard Handley2020/12/10 06:18 PM
                                                  and beyond just the flops...Maynard Handley2020/12/10 06:23 PM
                                                    and beyond just the flops...anonymou52020/12/10 09:59 PM
                                                    and beyond just the flops...Gabriele Svelto2020/12/11 01:57 AM
                                                      and beyond just the flops...Dummond D. Slow2020/12/11 09:52 AM
                                                        and beyond just the flops...Maynard Handley2020/12/11 09:57 AM
                                                          and beyond just the flops...Dummond D. Slow2020/12/11 10:00 AM
                                                            and beyond just the flops...Maynard Handley2020/12/11 11:22 AM
                                                              and beyond just the flops...Dummond D. Slow2020/12/12 09:13 AM
                                                                and beyond just the flops...Maynard Handley2020/12/12 01:41 PM
                                                                  and beyond just the flops...Dummond D. Slow2020/12/12 03:35 PM
                                                  and beyond just the flops...Dummond D. Slow2020/12/11 09:41 AM
                                                    and beyond just the flops...Maynard Handley2020/12/11 10:01 AM
                                                      and beyond just the flops...Dummond D. Slow2020/12/11 10:23 AM
                                                        and beyond just the flops...Dummond D. Slow2020/12/11 10:24 AM
                                              and beyond just the flops...Adrian2020/12/11 01:37 AM
                                                and beyond just the flops...Adrian2020/12/11 02:59 AM
                                                and beyond just the flops...Ungo2020/12/11 03:39 AM
                                                and beyond just the flops...Maynard Handley2020/12/11 10:07 AM
                                                  and beyond just the flops...Adrian2020/12/11 11:39 AM
                                                    and beyond just the flops...anonymou52020/12/11 03:01 PM
                                                    and beyond just the flops...David Hess2020/12/12 08:17 AM
                                                      and beyond just the flops...Jukka Larja2020/12/12 11:08 AM
                                                        and beyond just the flops...David Hess2020/12/12 11:36 AM
                                                          and beyond just the flops...Jukka Larja2020/12/12 09:45 PM
                                                            and beyond just the flops...David Hess2020/12/13 06:59 PM
                                                        and beyond just the flops...Adrian2020/12/12 03:11 PM
                                                          and beyond just the flops...Adrian2020/12/12 03:21 PM
                                                            and beyond just the flops...David Hess2020/12/13 07:02 PM
                              and beyond just the flops...useruser2020/12/09 09:38 PM
                                and beyond just the flops...anonymou52020/12/10 02:31 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?