New article - The Common System Interface: Intel's Future Interconnect

Article: The Common System Interface: Intel's Future Interconnect
By: jigal (jigal2.delete@this.gmail.com), September 25, 2007 3:16 pm
Room: Moderated Discussions
1st, thanx guys for the in depth explanation.

I begin to understand the major point here, that CSI
is an extension of a CPU to CPU or CPU to memory bus,
whereas PCI-e is and always will be a peripheral bus.
CSI would carry x86 instruction streams and its data,
whereas PCI-e was never designed to do that, and would
probably not be extended to take it on.

I dare ask a few more questions then:
1. Wouldn't intel need to open CSI such that vendors can
produce new north bridges to communicate with it?
After all - a bridge between PCI-e and CSI is needed, no?

2. Could CSI (partially)
replace PCI-e in the slightly longer term?
E.g., would it be used for off chip GPU's instead of
PCIe?

3. On servers - what is the leading peripheral
connectivity technology?
PCI-e? HT? FC? InfiniBand? Or even 10GbE?
(InfiniBand seemed to enter only in HPC, no?)
Or is it completely vendor determined?
(e.g., IBM's servers would go FC,
Intel x86 based servers would go PCI-e,
AMD x86 based servers would go HT.., etc.)









Jonathan Kang (johnbk@gmail.com) on 9/25/07 wrote:
---------------------------
>jigal (jigal2@gmail.com) on 9/23/07 wrote:
>---------------------------
>>Thanx for the reply.
>>
>>Pretty much overwhelmed by the the level of technical details - since I am not a vlsi guy, but merely a s/w guy.
>>The reason I am asking, is that, after all, Intel invested
>>considerable effort in adding the AS layer on top of the
>>PCI-E, which seems to turn it to a full networking layer,
>>which CSI also seems to add.
>>(although you could argue that "Intel" in this sense is a
>>bunch of engineers toying with their own standard...)
>
>The key here is in the physical implementation and limitations rather than what's
>viewed from software. Keep in mind that software will not be the only thing using
>CSI. It's used by ASICs to communicate with each without the control of software.
>This adds a requirement that PCI-e didn't necessarily have: low-latency.
>
>This, on top of the power requirements (not just max power but average power) means
>that a new link had to be made that wasn't as power hungry and had significantly
>lower wake-up latencies (since they'd be shutting the link off during idle times) than PCI-e.
>
>>This is in contrast to AMD which has a consistent on board
>>p2p link - the HT link, which seems to be PCI friendly,
>>and was invented when PCI-E wasn't relevant, I guess.
>>I remember reading an pdf on their website trying to
>>differentiate HT from PCI as on intra board links vs.
>>inter-board links, or even inter-chassis, etc...
>
>Before PCI-e, AMD systems used PCI and its variant flavors along with HT links.
>The two are not meant for the same purpose and they have different physical limitations
>and features. Intel's original vision for PCI-e wasn't just to connect peripherals
>inside of a beige box but to use it as a chassis-to-chassis link as well. This
>means that it had to be a truly serial link where line-latency was a non-issue and
>signal strength/integrity had to be maintained over very long cables. This ruled
>something like HT (which is still parallel to a degree) out. Think of PCI-e more
>as fiber-channel or ethernet as it is *very* similar to both of those.
>
>>Is the HT used for coherency protocols between AMD asics?
>>And regardless, now that we have a new guy in town,
>>what is the future of PCI-E, if they're not going to
>>add coherency protocols on top of it?
>
>PCI-e will serve the purpose it was intended for, peripheral connection and inter-chassis
>(or inter-board) connection. It was made to tolerate very long delays and to operate
>between multiple devices (no common voltage required).
>
>CSI will compete with HT for inter-chip communication where latency and power are
>limiting factors (such as multi-processor configurations).
>
>>I guess on the upside, it would provide plenty of work
>>to engineers to develop and sell hubs for CSI/PCI-E/HT :-)
>
>Considering Intel's proprietary nature in the past, I'm not sure anybody outside
>of Intel will be using CSI. Intel had to open PCI-e in order to get peripheral
>vendors to use it but the same need not be said of CSI.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/08/28 02:16 AM
  New article - The Common System Interface: Intel's Future InterconnectVincent Diepeveen2007/08/28 06:02 AM
  New article - The Common System Interface: Intel's Future InterconnectRichard Cownie2007/08/28 11:28 AM
    New article - The Common System Interface: Intel's Future InterconnectVincent Diepeveen2007/08/31 12:44 PM
      New article - The Common System Interface: Intel's Future InterconnectRichard Cownie2007/08/31 09:53 PM
        New article - The Common System Interface: Intel's Future InterconnectVincent Diepeveen2007/09/01 03:21 AM
          Adding layers can simplify designPaul A. Clayton2007/09/01 08:39 AM
          New article - The Common System Interface: Intel's Future InterconnectMichael S2007/09/02 03:25 AM
        New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/14 01:47 PM
    New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/09/14 09:47 PM
  New article - The Common System Interface: Intel's Future InterconnectPaul2007/08/28 12:04 PM
    New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/08/28 01:43 PM
      New article - The Common System Interface: Intel's Future InterconnectJoe Chang2007/08/28 07:17 PM
        New article - The Common System Interface: Intel's Future InterconnectJoe Chang2007/08/29 05:27 PM
  Thanks for the workWouter Tinus2007/08/28 01:33 PM
    Thanks for the workmac2007/08/29 01:44 PM
  New article - The Common System Interface: Intel's Future InterconnectHerbert Hum2007/08/28 02:22 PM
    ThanksDavid Kanter2007/08/28 05:13 PM
  Many thanks, very, very interesting! (NT)Cameron Jack2007/08/29 02:51 AM
  very nice article + memory ctrl integrationMarcin Niewiadomski2007/08/29 12:46 PM
    very nice article + memory ctrl integrationDavid Kanter2007/09/14 09:50 PM
      very nice article + memory ctrl integrationMarcin Niewiadomski2007/09/16 09:48 PM
  Coherency: Forwarding and OwnedPeter Gerdes2007/08/29 03:11 PM
    Coherency: Forwarding and OwnedDavid Kanter2007/08/29 07:29 PM
      Coherency: Forwarding and Ownednick2007/08/29 08:03 PM
        Coherency: Forwarding and OwnedDavid Kanter2007/08/30 12:08 AM
      Coherency: Forwarding and OwnedMichael S2007/08/30 02:17 AM
        Coherency: Forwarding and OwnedDavid Kanter2007/08/30 08:31 AM
      Coherency: Forwarding and OwnedPeter Gerdes2007/08/30 12:46 PM
        Coherency: Forwarding and OwnedDavid Kanter2007/08/30 02:46 PM
          Coherency: Forwarding and OwnedPeter Gerdes2007/08/30 08:03 PM
            Coherency: Forwarding and OwnedDavid Kanter2007/09/14 09:44 PM
              Node Interleaveunknown2007/09/15 04:14 AM
                Node InterleaveDavid Kanter2007/09/15 08:50 AM
                  Node InterleaveHoward Chu2007/09/16 01:14 PM
              Coherency: Forwarding and OwnedPeter Gerdes2007/09/16 01:50 PM
                Coherency: Forwarding and OwnedDavid Kanter2007/09/16 05:34 PM
                  Coherency: Forwarding and OwnedEduardoS2007/09/16 05:52 PM
                  Coherency: Forwarding and OwnedJonathan Kang2007/09/17 06:16 AM
                  Coherency: Forwarding and OwnedMatthias2007/09/17 07:59 AM
                    Coherency: Forwarding and Owned - additionMatthias2007/09/17 08:01 AM
                Coherency: Forwarding and Ownedanonymous2007/09/17 10:15 AM
                  Coherency: Forwarding and OwnedPeter Gerdes2007/09/17 01:44 PM
  New article - The Common System Interface: Intel's Future InterconnectMr. Camel2007/08/30 04:16 PM
    New article - The Common System Interface: Intel's Future InterconnectMichael S2007/08/31 02:11 AM
      New article - The Common System Interface: Intel's Future InterconnectMr. Camel2007/08/31 04:13 AM
        New article - The Common System Interface: Intel's Future InterconnectMichael S2007/08/31 04:24 AM
        New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/08/31 06:39 AM
          New article - The Common System Interface: Intel's Future InterconnectMichael S2007/08/31 07:53 AM
      New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/08/31 06:41 AM
        New article - The Common System Interface: Intel's Future InterconnectMichael S2007/08/31 07:36 AM
          New article - The Common System Interface: Intel's Future InterconnectMr. Camel2007/08/31 09:36 AM
  Thanks and excellent work!Jack A.2007/08/30 08:41 PM
  Lamport's TLAKonrad Schwarz2007/09/02 02:57 AM
    Lamport's TLADavid Kanter2007/09/02 08:55 PM
    Lamport's TLABrannon2007/09/03 08:12 AM
      Lamport's TLAKonrad Schwarz2007/09/18 11:21 AM
        Lamport's TLABrannon2007/09/18 02:58 PM
  New article - The Common System Interface: Intel's Future InterconnectJosé Javier Zarate2007/09/09 05:01 PM
  New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/16 06:42 PM
    Remote prefetchDavid Kanter2007/09/17 08:51 AM
  New article - The Common System Interface: Intel's Future InterconnectJigal2007/09/22 03:39 PM
    New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/09/22 05:35 PM
      New article - The Common System Interface: Intel's Future Interconnect8B/10B Latency2007/09/22 07:16 PM
        New article - The Common System Interface: Intel's Future Interconnectanon2007/09/22 09:05 PM
          New article - The Common System Interface: Intel's Future InterconnectDavid W. Hess2007/09/22 09:50 PM
            Clocking in CSIDavid Kanter2007/09/23 09:46 AM
              Hypertransport 3 AC CouplingDavid W. Hess2007/09/23 10:32 AM
                Hypertransport 3 AC Couplinganon2007/09/23 10:53 AM
                  Clocking lanesDavid Kanter2007/09/23 11:51 AM
        New article - The Common System Interface: Intel's Future InterconnectDavid Kanter2007/09/22 09:34 PM
          New article - The Common System Interface: Intel's Future InterconnectDavid W. Hess2007/09/22 10:10 PM
            New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/25 08:15 AM
          New article - The Common System Interface: Intel's Future InterconnectMichael S2007/09/23 02:06 AM
            New article - The Common System Interface: Intel's Future InterconnectDavid W. Hess2007/09/23 04:41 AM
            Serialization delayDavid Kanter2007/09/23 09:57 AM
              Serialization delayMichael S2007/09/23 12:20 PM
                Serialization delayDavid Kanter2007/09/23 01:43 PM
                  Serialization delayMichael S2007/09/24 01:40 AM
                    Serialization delayMichael S2007/09/24 05:28 AM
                    Serialization delayAaron Spink2007/09/24 01:19 PM
                      Serialization delayMichael S2007/09/25 04:38 AM
                        Serialization delayJonathan Kang2007/09/25 09:10 AM
                          Serialization delayDavid W. Hess2007/09/26 01:22 AM
                        Serialization delayAaron Spink2007/09/25 01:13 PM
                          Thank you (NT)Michael S2007/09/25 01:53 PM
                Serialization delayJonathan Kang2007/09/25 08:26 AM
                  Serialization delayMichael S2007/09/25 02:57 PM
                    Serialization delayJonathan Kang2007/09/26 06:24 AM
                      Serialization delayDavid W. Hess2007/09/26 07:39 AM
                        Serialization delayJonathan Kang2007/09/26 10:56 AM
                          Serialization delayDavid W. Hess2007/09/27 03:21 AM
                            Serialization delayJonathan Kang2007/09/27 05:36 AM
                              Serialization delayDavid W. Hess2007/09/27 06:31 PM
                      Serialization delayrwessel2007/09/26 02:26 PM
                        Serialization delayJonathan Kang2007/09/27 08:16 AM
                          Serialization delayrwessel2007/09/27 01:20 PM
                            Serialization delayJonathan Kang2007/09/28 05:38 AM
                              Serialization delayrwessel2007/09/28 02:00 PM
                                Serialization delayJonathan Kang2007/10/01 08:07 AM
                                  Cache coherent latencyDavid Kanter2007/10/01 08:20 AM
                                    Cache coherent latencyblaine2007/10/01 11:36 AM
                                      Critical word first on coherent interconnectsDavid Kanter2007/10/01 12:10 PM
                                        Does ccHT do critical word first?blaine2007/10/02 08:10 AM
                                    Cache coherent latencyJonathan Kang2007/10/01 01:34 PM
                                      Cache coherent latencyDavid Kanter2007/10/01 02:13 PM
                      Serialization delayMichael S2007/09/28 05:32 AM
                        Serialization delayanonymous2007/09/28 11:25 AM
                          Serialization delayMichael S2007/09/29 10:06 AM
        New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/25 08:05 AM
      New article - The Common System Interface: Intel's Future Interconnectjigal2007/09/23 01:37 PM
        CSI, PCI and HTDavid Kanter2007/09/23 01:46 PM
        New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/25 08:39 AM
          New article - The Common System Interface: Intel's Future Interconnectjigal2007/09/25 03:16 PM
            New article - The Common System Interface: Intel's Future InterconnectMichael S2007/09/26 04:14 AM
              New article - The Common System Interface: Intel's Future InterconnectAnonymous2007/09/26 10:41 AM
                New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/26 10:59 AM
            New article - The Common System Interface: Intel's Future InterconnectJonathan Kang2007/09/26 06:48 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?