Current NVidia GPU Launch Latency: 10 microseconds

By: Mark Roulo (nothanks.delete@this.xxx.com), May 31, 2022 1:07 pm
Room: Moderated Discussions
Peter Lewis (peter.delete@this.notyahoo.com) on May 31, 2022 4:37 am wrote:
> > Just think about what happens in system in presence of coherent
> > device connected through slow link at each cache miss
> > that also missed coherence filter/cache directory. Answer: instead of 50-55 ns CPU waits 1000-1500 ns.
>
> Why do you think a PCIe 5 device will take 1000-1500ns to respond? Considering the price
> of modern GPUs, if they are really that slow, customers would demand a refund.

CUDA Kernel launch latency seems to be around 10 microseconds (so the time to run a one-thread no-op kernel will be around 10 microseconds).

One can imagine a more integrated PCIe CLX protocol doing things more in H/W and keeping the OS out of the critical path, but the current time to do a minimal something with a GPU is about 10 microseconds.

The GPUs "get away with it" because they are more throughput engines than latency engines and because even 100 frames/sec is still 10,000 microseconds so a single kernel's latency is still not a limiter.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Crazy CXL NamesPeter Lewis2022/05/30 03:54 PM
  Crazy CXL Namesanon22022/05/30 07:19 PM
    Crazy CXL NamesPeter Lewis2022/05/30 09:30 PM
      Crazy CXL Namesanon22022/05/31 02:24 AM
        Crazy CXL NamesPeter Lewis2022/05/31 02:43 AM
          Crazy CXL Namesanon22022/05/31 03:01 AM
            Crazy CXL NamesPeter Lewis2022/05/31 03:36 AM
              Crazy CXL Namesanon22022/05/31 04:32 AM
                Crazy CXL NamesPeter Lewis2022/05/31 05:21 AM
                  Crazy CXL Namesanon22022/05/31 06:03 AM
                    Crazy CXL NamesPeter Lewis2022/05/31 04:27 PM
                      Crazy CXL Namesanon22022/05/31 04:58 PM
                      Crazy CXL Namesgoose2022/05/31 05:35 PM
                        Crazy CXL NamesPeter Lewis2022/06/03 03:10 AM
                          Reason for Crazy CXL NamesGary Kopp2022/06/24 01:43 PM
                            Reason for Crazy CXL Namesdmcq2022/06/25 11:18 AM
                              Reason for Crazy CXL NamesMichael S2022/06/26 01:30 AM
                                Reason for Crazy CXL NamesDummond D. Slow2022/06/26 06:49 AM
                                  Reason for Crazy CXL NamesMichael S2022/06/26 07:32 AM
                                    Reason for Crazy CXL Namesdmcq2022/06/26 11:54 AM
                                Reason for Crazy CXL Namesanon22022/06/26 03:26 PM
                                  Reason for Crazy CXL NamesEric P2022/07/01 02:48 PM
                                    Reason for Crazy CXL Namesanon22022/07/01 07:56 PM
                                      Reason for Crazy CXL NamesGroo2022/07/02 05:13 AM
                                    Reason for Crazy CXL NamesUngo2022/07/01 11:14 PM
                                      Reason for Crazy CXL NamesEric P2022/07/05 03:21 AM
                      Crazy CXL NamesDoug S2022/06/01 12:44 AM
    Crazy CXL NamesMichael S2022/05/31 01:21 AM
      Crazy CXL Namesanon22022/05/31 02:20 AM
      Crazy CXL NamesPeter Lewis2022/05/31 02:24 AM
        Crazy CXL NamesMichael S2022/05/31 03:07 AM
          Crazy CXL NamesPeter Lewis2022/05/31 04:37 AM
            Current NVidia GPU Launch Latency: 10 microsecondsMark Roulo2022/05/31 01:07 PM
              Current NVidia GPU Launch Latency: 10 microsecondsFreddie2022/06/01 09:44 AM
      Crazy CXL Names---2022/05/31 07:56 PM
        Crazy CXL Namesaaron spink2022/05/31 08:32 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell tangerine? 🍊