Interconnection Structures Patrick Happ Raul Queiroz Feitosa - PowerPoint PPT Presentation

Interconnection Structures Patrick Happ Raul Queiroz Feitosa

Objective To present key issues that affect interconnection design. 2 Interconnection Structures

Outline  Introduction  Computer Busses  Bus Types  PCI  PCI Express 3 Interconnection Structures

Introduction  All the units must be connected  Different type of connection for different type of unit:  Memory  Input/Output  CPU 4 Interconnection Structures

Unit Types 5 Interconnection Structures

Computer Busses A bus is a common electrical pathway between multiple devices 6 Interconnection Structures

Functional groups of bus lines 7 Interconnection Structures

Data Bus Carries data  Remember that there is no difference between “data” and “instruction” at this level. Data bus width determines the amount of data moved in a single access.  8, 16, 32, 64 bit 8 Interconnection Structures

Address bus Identifies the source or destination of data  e.g. CPU needs to read an instruction (data) from a given location in memory. Address bus width determines maximum memory capacity of system  e.g. 8080 has 16 bit address bus giving 64k address space 9 Interconnection Structures

Control bus Carries control and timing information. Typical control lines:  Memory and I/O read/write signal  Interrupt request/acknowledge  Bus grant/request  Clock signals  Reset 10 Interconnection Structures

Physical Realization of Bus Architecture 11 Interconnection Structures

Single Bus Problems  The more devices attached to the bus,  the greater the bus length,  the larger the propagation delays, and  the lower the maximum transfer rate.  Most systems use multiple buses to overcome these problems 12 Interconnection Structures

Traditional bus architecture Example: ISA bus 13 Interconnection Structures

High Performance Bus Example: PCI Bus 14 Interconnection Structures

Bus Types Dedicated  Separate data & address lines Multiplexed  Shared lines  Address valid or data valid control line  Advantage - fewer lines  Disadvantages  More complex control  Performance 15 Interconnection Structures

Bus Arbitration There may be more than one potential bus master, e.g.  CPU and DMA controller  Multiple CPUs in a parallel shared bus system. An arbitration mechanism is required to guarantee that no more than one master controls the bus at a time. 16 Interconnection Structures

Bus Arbitration bus Why? line Vcc Vcc digital output model IN IN Q 3 Q 3 OUT OUT Q 1 Q 1 OE OE Q 4 Q 4 Q 2 Q 2 short circuit OE=low;IN=low → Q 3 off ; Q 4 on ; OUT=low OUT=low (Q 3 off - Q 4 on) OE=low;IN=high → Q 3 on ; Q 4 off ; OUT=high OUT=high (Q 3 on - Q 4 off) OE=high → Q 3 off ; Q 4 off ; OUT=highZ OUT=highZ (Q 3 off - Q 4 off) 17 Interconnection Structures

Methods of Arbitration Centralised  a single arbiter grants bus access. bus bus requests grants R 0 G 0 R 1 G 1 R 2 G 2 Arbiter . . . . . . R n-1 G n-1 18 Interconnection Structures

Methods of Arbitration Distributed  Each module may claim the bus.  Control logic on all modules. 19 Interconnection Structures

Methods of Arbitration Distributed  Bus request and Busy lines are open collector.  Requesting devices set Bus request=0  Requesting devices make Out=0 ; non requesting devices make Out=In  Requesting device with In=1 , gets the bus  It waits until Busy=1 , sets Busy=0 and takes the bus  Upon relinquishing the Bus the device sets Busy=1 20 Interconnection Structures

Timing Synchronous  Events determined by clock signals.  Control Bus includes clock line.  Usually sync on leading edge  Usually a single cycle for an event  A READY/WAIT line signals when the slave is expected to have completed the access.  Advantage: simple implementation (due to the clock signal). 21 Interconnection Structures

Synchronous Timing Diagram 22 Interconnection Structures

Synchronous Timing Diagram 23 Interconnection Structures

Timing Asynchronous  No clock signal (due to distortion)  Events determined by completion of earlier events  Status lines signal when the slave completes the access.  Advantages:  allows for “fractional” cycles  no minimal access time is imposed  longer busses are possible (clock distortion) 24 Interconnection Structures

Asynchronous Timing Diagram 25 Interconnection Structures

Data Transfer Type Time → Address Data Address Write (multiplexed) operation Data access Address Data time Write (non-multiplexed) operation Read (multiplexed) operation Address Address Data read Data write Read-modify-write operation Data Read (non-multiplexed) operation Address Data write Data read Read after write operation Address Data Data Data Block Data Transfer 26 Interconnection Structures

The PCI Bus The bus structure of a Pentium 4. 27 Interconnection Structures

The PCI Bus Characteristics  Synchronous  Parallel  32 or 64 bit transfers  Up to 528 MB/s The three 64-bit PCI slots and a single 32-bit PCI-slot 28 Interconnection Structures

PCI Bus Signals(1) Mandatory PCI bus signals. 29 Interconnection Structures

PCI Bus Signals(2) Optional PCI bus signals. 30 Interconnection Structures

PCI Bus Transactions multiplexed address and data lines bus command/bit map for bytes enable AD and C/BE are enabled read: master will accept; write: data present read: data present; write: slave will accept master Examples of 32-bit PCI bus transactions. slave both 31 Interconnection Structures

Problem with parallel busses Clock skew  a phenomenon in synchronous circuits in which the clock signal (sent from the clock circuit) arrives at different components at different times. discrepancy 32 Interconnection Structures

Problem with parallel busses Clock skew may be caused by many factors:  wire length,  variation in intermediate devices,  capacitive coupling,  material imperfections, … As the clock rate increases, less variation can be tolerated if the circuit is to function properly. It imposes a clock rate limit to the parallel bus. 33 Interconnection Structures

PCIe A typical PCI Express system. Manages multiple PCIe streams 34 Interconnection Structures

PCIe PCIe endpoint: An I/O device or controller that implements PCIe, Gigabit ethernet switch, a graphis or video controler, disk interface, or a communication controler PCIe/PCI bridge: Allows Switch: manages older PCI devices to be multiple PCIe streams connected to PCIe-base system. 35 Interconnection Structures

PCIe Characteristics  Communication flows through one or more pairs of unidirectional connections, called lanes .  Each connection of a lane consists of one wire for the signal and one for ground, which provide high noise immunity.  PCIe devices communicate through a link , which is built up from a collection of one or more lanes  links with 1, 2, 4 ,8 16 and 32 lanes are allowed. 36 Interconnection Structures

PCIe Configuration paired serial links 37 Interconnection Structures

PCIe Evolution Evolution  2004: PCIe 1.1 → 2.5 GB/s (per lane).  2007: PCIe 2.0 → 5.0 GB/s (per lane).  2010: PCIe 3.0 → 8.0 GB/s (per lane).  2017: PCIe 4.0 → 16 .0 GB/s (per lane).  2019: PCIe 5.0 → 32 .0 GB/s (per lane).  2021: PCIe 6.0 → 64 .0 GB/s (per lane). 38 Interconnection Structures

PCIe Evolution 39 Source: Wikipedia Interconnection Structures

PCIe Protocol Layers  A protocol is a set of rules governing the conversation between two parties.  A protocol stack is a hierarchy of protocols that deals with different issues at different layers.  The PCI Express protocol stack has 3 layers: 40 Interconnection Structures

PCI Express – physical layer  It deals with moving bits from a sender to a receiver.  Recall that each point-to-point connection consists of one or more pairs of simplex (unidirecional) links, called lanes.1, 2, 4, 8, 16 or 32 pairs are allowed A PCI Express x16 slot A PCI Express x1 slot  No master clock – 128b/130b enconding → 1 enssures enough clock transitions to keep synchonization. 41 Interconnection Structures

PCIe Multilane Distribution 42 Interconnection Structures

PCIe Link Layer  It deals with packet transmission  It adds to the header+payload the sequence number and error-correction code CRC  If the CRC is checked OK, the receiver sends back an acknowledgement packet, otherwise it asks for retransmission.  This greatly improves data integrity. Transaction layer header payload Link layer seq # header payload CRC Physical layer frame seq # header payload CRC frame 43 Interconnection Structures