Serialization delay

Article: The Common System Interface: Intel's Future Interconnect
By: Jonathan Kang (johnbk.delete@this.gmail.com), September 28, 2007 4:38 am
Room: Moderated Discussions
rwessel (robertwessel@yahoo.com) on 9/27/07 wrote:
---------------------------
>Jonathan Kang (johnbk@gmail.com) on 9/27/07 wrote:
>---------------------------
>>That's a good point. Yes, in the situation of multiple, parallel links, all but
>>one can be turned off. But for schemes such as cache coherency, bursts of large
>>(or small) chunks of data would need to be transmitted as fast as possible.
>>
>>For instance, if the link between 3 MPU's are in the idle state (only 1 link active)
>>and a snoop is made. If the response is that the cache data must be updated, then
>>that cache data must be transmitted over the link as quickly as possible to prevent
>>stalls. This isn't possible unless the link can go between idle (low bandwidth)
>>and active (full bandwidth) within a matter of a few cycle times.
>
>
>That's true, but that largely going to be a self limiting problem - if the CPU
>is idle, it may be a bit slow in responding to actual cache line updates/writebacks/whatever
>(just the coherency checks themselves should have plenty of bandwidth with a single
>link), and there will be a limited number of times a cache line will need to be transferred off the idle CPU.

AMD fixed the problem of the CPU having to wake-up for a cache snoop in Barcelona I think. I suspect Intel will do the same thing in Nehalem. So the cache can respond near-instantaneously to snoop and update requests.

The problem is that even though cache updates over the links happen with relatively low frequency, it's like any other cache-miss access, if it ever happens, it could potentially stall the CPU requesting the data for a (relative to the CPU) long time.

>Obviously if the CPU has the memory controller, there's going to be a lot of real
>traffic, even if the CPU is idle, but even in that case some of the links can probably
>be shut down with minimal impact on the system, since the traffic from the CPU is simply not going to be there.

The problem is knowing which links to shut down. The non-deterministic nature of cache-coherency means that you will almost always get times when a link has to go from idle (one link on) to active (all links on) in order to transmit data. If it doesn't, a CPU stalls for hundreds or thousands of its own clock cycles waiting.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/08/28 01:16 AM
  New article - The Common System Interface: Intel's Future InterconnectVincent Diepeveen2007/08/28 05:02 AM
  New article - The Common System Interface: Intel's Future InterconnectRichard Cownie2007/08/28 10:28 AM
    New article - The Common System Interface: Intel's Future InterconnectVincent Diepeveen2007/08/31 11:44 AM
      New article - The Common System Interface: Intel's Future InterconnectRichard Cownie2007/08/31 08:53 PM
        New article - The Common System Interface: Intel's Future InterconnectVincent Diepeveen2007/09/01 02:21 AM
          Adding layers can simplify designPaul A. Clayton2007/09/01 07:39 AM
          New article - The Common System Interface: Intel's Future InterconnectMichael S2007/09/02 02:25 AM
        New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/14 12:47 PM
    New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/09/14 08:47 PM
  New article - The Common System Interface: Intel's Future InterconnectPaul2007/08/28 11:04 AM
    New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/08/28 12:43 PM
      New article - The Common System Interface: Intel's Future InterconnectJoe Chang2007/08/28 06:17 PM
        New article - The Common System Interface: Intel's Future InterconnectJoe Chang2007/08/29 04:27 PM
  Thanks for the workWouter Tinus2007/08/28 12:33 PM
    Thanks for the workmac2007/08/29 12:44 PM
  New article - The Common System Interface: Intel's Future InterconnectHerbert Hum2007/08/28 01:22 PM
    ThanksDavid Kanter2007/08/28 04:13 PM
  Many thanks, very, very interesting! (NT)Cameron Jack2007/08/29 01:51 AM
  very nice article + memory ctrl integrationMarcin Niewiadomski2007/08/29 11:46 AM
    very nice article + memory ctrl integrationDavid Kanter2007/09/14 08:50 PM
      very nice article + memory ctrl integrationMarcin Niewiadomski2007/09/16 08:48 PM
  Coherency: Forwarding and OwnedPeter Gerdes2007/08/29 02:11 PM
    Coherency: Forwarding and OwnedDavid Kanter2007/08/29 06:29 PM
      Coherency: Forwarding and Ownednick2007/08/29 07:03 PM
        Coherency: Forwarding and OwnedDavid Kanter2007/08/29 11:08 PM
      Coherency: Forwarding and OwnedMichael S2007/08/30 01:17 AM
        Coherency: Forwarding and OwnedDavid Kanter2007/08/30 07:31 AM
      Coherency: Forwarding and OwnedPeter Gerdes2007/08/30 11:46 AM
        Coherency: Forwarding and OwnedDavid Kanter2007/08/30 01:46 PM
          Coherency: Forwarding and OwnedPeter Gerdes2007/08/30 07:03 PM
            Coherency: Forwarding and OwnedDavid Kanter2007/09/14 08:44 PM
              Node Interleaveunknown2007/09/15 03:14 AM
                Node InterleaveDavid Kanter2007/09/15 07:50 AM
                  Node InterleaveHoward Chu2007/09/16 12:14 PM
              Coherency: Forwarding and OwnedPeter Gerdes2007/09/16 12:50 PM
                Coherency: Forwarding and OwnedDavid Kanter2007/09/16 04:34 PM
                  Coherency: Forwarding and OwnedEduardoS2007/09/16 04:52 PM
                  Coherency: Forwarding and OwnedJonathan Kang2007/09/17 05:16 AM
                  Coherency: Forwarding and OwnedMatthias2007/09/17 06:59 AM
                    Coherency: Forwarding and Owned - additionMatthias2007/09/17 07:01 AM
                Coherency: Forwarding and Ownedanonymous2007/09/17 09:15 AM
                  Coherency: Forwarding and OwnedPeter Gerdes2007/09/17 12:44 PM
  New article - The Common System Interface: Intel's Future InterconnectMr. Camel2007/08/30 03:16 PM
    New article - The Common System Interface: Intel's Future InterconnectMichael S2007/08/31 01:11 AM
      New article - The Common System Interface: Intel's Future InterconnectMr. Camel2007/08/31 03:13 AM
        New article - The Common System Interface: Intel's Future InterconnectMichael S2007/08/31 03:24 AM
        New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/08/31 05:39 AM
          New article - The Common System Interface: Intel's Future InterconnectMichael S2007/08/31 06:53 AM
      New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/08/31 05:41 AM
        New article - The Common System Interface: Intel's Future InterconnectMichael S2007/08/31 06:36 AM
          New article - The Common System Interface: Intel's Future InterconnectMr. Camel2007/08/31 08:36 AM
  Thanks and excellent work!Jack A.2007/08/30 07:41 PM
  Lamport's TLAKonrad Schwarz2007/09/02 01:57 AM
    Lamport's TLADavid Kanter2007/09/02 07:55 PM
    Lamport's TLABrannon2007/09/03 07:12 AM
      Lamport's TLAKonrad Schwarz2007/09/18 10:21 AM
        Lamport's TLABrannon2007/09/18 01:58 PM
  New article - The Common System Interface: Intel's Future InterconnectJosé Javier Zarate2007/09/09 04:01 PM
  New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/16 05:42 PM
    Remote prefetchDavid Kanter2007/09/17 07:51 AM
  New article - The Common System Interface: Intel's Future InterconnectJigal2007/09/22 02:39 PM
    New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/09/22 04:35 PM
      New article - The Common System Interface: Intel's Future Interconnect8B/10B Latency2007/09/22 06:16 PM
        New article - The Common System Interface: Intel's Future Interconnectanon2007/09/22 08:05 PM
          New article - The Common System Interface: Intel's Future InterconnectDavid W. Hess2007/09/22 08:50 PM
            Clocking in CSIDavid Kanter2007/09/23 08:46 AM
              Hypertransport 3 AC CouplingDavid W. Hess2007/09/23 09:32 AM
                Hypertransport 3 AC Couplinganon2007/09/23 09:53 AM
                  Clocking lanesDavid Kanter2007/09/23 10:51 AM
        New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/09/22 08:34 PM
          New article - The Common System Interface: Intel's Future InterconnectDavid W. Hess2007/09/22 09:10 PM
            New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/25 07:15 AM
          New article - The Common System Interface: Intel's Future InterconnectMichael S2007/09/23 01:06 AM
            New article - The Common System Interface: Intel's Future InterconnectDavid W. Hess2007/09/23 03:41 AM
            Serialization delayDavid Kanter2007/09/23 08:57 AM
              Serialization delayMichael S2007/09/23 11:20 AM
                Serialization delayDavid Kanter2007/09/23 12:43 PM
                  Serialization delayMichael S2007/09/24 12:40 AM
                    Serialization delayMichael S2007/09/24 04:28 AM
                    Serialization delayAaron Spink2007/09/24 12:19 PM
                      Serialization delayMichael S2007/09/25 03:38 AM
                        Serialization delayJonathan Kang2007/09/25 08:10 AM
                          Serialization delayDavid W. Hess2007/09/26 12:22 AM
                        Serialization delayAaron Spink2007/09/25 12:13 PM
                          Thank you (NT)Michael S2007/09/25 12:53 PM
                Serialization delayJonathan Kang2007/09/25 07:26 AM
                  Serialization delayMichael S2007/09/25 01:57 PM
                    Serialization delayJonathan Kang2007/09/26 05:24 AM
                      Serialization delayDavid W. Hess2007/09/26 06:39 AM
                        Serialization delayJonathan Kang2007/09/26 09:56 AM
                          Serialization delayDavid W. Hess2007/09/27 02:21 AM
                            Serialization delayJonathan Kang2007/09/27 04:36 AM
                              Serialization delayDavid W. Hess2007/09/27 05:31 PM
                      Serialization delayrwessel2007/09/26 01:26 PM
                        Serialization delayJonathan Kang2007/09/27 07:16 AM
                          Serialization delayrwessel2007/09/27 12:20 PM
                            Serialization delayJonathan Kang2007/09/28 04:38 AM
                              Serialization delayrwessel2007/09/28 01:00 PM
                                Serialization delayJonathan Kang2007/10/01 07:07 AM
                                  Cache coherent latencyDavid Kanter2007/10/01 07:20 AM
                                    Cache coherent latencyblaine2007/10/01 10:36 AM
                                      Critical word first on coherent interconnectsDavid Kanter2007/10/01 11:10 AM
                                        Does ccHT do critical word first?blaine2007/10/02 07:10 AM
                                    Cache coherent latencyJonathan Kang2007/10/01 12:34 PM
                                      Cache coherent latencyDavid Kanter2007/10/01 01:13 PM
                      Serialization delayMichael S2007/09/28 04:32 AM
                        Serialization delayanonymous2007/09/28 10:25 AM
                          Serialization delayMichael S2007/09/29 09:06 AM
        New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/25 07:05 AM
      New article - The Common System Interface: Intel's Future Interconnectjigal2007/09/23 12:37 PM
        CSI, PCI and HTDavid Kanter2007/09/23 12:46 PM
        New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/25 07:39 AM
          New article - The Common System Interface: Intel's Future Interconnectjigal2007/09/25 02:16 PM
            New article - The Common System Interface: Intel's Future InterconnectMichael S2007/09/26 03:14 AM
              New article - The Common System Interface: Intel's Future InterconnectAnonymous2007/09/26 09:41 AM
                New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/26 09:59 AM
            New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/26 05:48 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?