RapidIO Overview March 2014 Barry Wood, Chair, RapidIO Technical - - PowerPoint PPT Presentation

▶
rapidio overview
SMART_READER_LITE
LIVE PREVIEW

RapidIO Overview March 2014 Barry Wood, Chair, RapidIO Technical - - PowerPoint PPT Presentation

Mar 05, 2024 409 likes •744 views

RapidIO Overview March 2014 Barry Wood, Chair, RapidIO Technical Working Group barry.wood@idt.com OR twg-chair@rapidio.org RapidIO Evolution Date Event Max Lane Speed Electricals 1997 Mercury Computer and Motorola (Freescale) begin to

slide-1
SLIDE 1

RapidIO Overview

March 2014

Barry Wood, Chair, RapidIO Technical Working Group barry.wood@idt.com OR twg-chair@rapidio.org

slide-2
SLIDE 2

RapidIO Evolution

March 2014 2

Date Event Max Lane Speed Electricals 1997 Mercury Computer and Motorola (Freescale) begin to collaborate on a replacement for the Raceway interconnect 2000, March RapidIO Trade Association formed 2000, June RapidIO 1.2 (Parallel PHY) 1 Gbps LVDS, 8/16 bit bus 2003, Oct RapidIO 1.3 (Serial PHY) Adopted as ISO standard (ISO/IEC DIS 18372) 3.125 Gbaud XAUI, 8b/10b 2008 RapidIO 2.0 (Faster) 6.25 Gbaud OIF CEI, 8b/10b 2013 RapidIO 3.0 (Faster) 10.3125 Gbaud 10G-kr, 10G-ab 64b/67b 2015(?) RapidIO 4.0 (Faster) 25 Gbaud 64b/67b?

  • Over 6 million RapidIO switches shipped
  • Over 30 million 10-20 Gbps ports shipped > 10GbE
slide-3
SLIDE 3

RapidIO Attributes

March 2014 3

Scalability Fault Tolerance Interoperable Power Efficiency Low Latency Any System Topology

slide-4
SLIDE 4

Example System: ASML & Prodrive

March 2014 4

100+ KW Power Supply Precision Optics RapidIO Control

APPLICATION: Semiconductor photolithography machines 10,000 to 50,000 Hz control loop frequencies => nanometer precision https://www.youtube.com/watch?v=B9uDMNmajgw

slide-5
SLIDE 5

Axxia Communications Processor DSP: several products In TCI64xx family DSP , PowerQUICC & QorIQ multicore XLS416 family Multicore Processor FPGA: Arria and Stratix Family FPGA: Virtex 4/5/6 families FPGA Wireless Baseband Processor DSP Oct22xx PowerPC based processors 460GT Switches, Bridges & IP CPS and Tsi Family Network Processor Octeon 2 family Network Processor WinPath3

Strong Ecosystem

March 2014 5

Used by all of the top 10 Wireless OEMs

slide-6
SLIDE 6

RapidIO Specification Structure

March 2014 6

Physical Layer (4) Parallel PHY (Deprecated) (6) Serial Transport Layer (3) Transport Logical Layer (1) Logical I/O (2) Message (9) Flow Control (10) Data Streaming (11) Multicast (5) Global Shared Memory

(8) Error Management /Hot Swap

(12) VoQ Backpressure Annex II

Encapsulation

Annex I HW API System Bringup

(7) Interop

slide-7
SLIDE 7

RapidIO Specification Mapped to Devices

March 2014 7

uProc

Switch

DSP DSP DSP DSP

Logical Transport

Switch

Physical

A B

Packet 0 CS CS CS Packet 1 CS CS Packet 2 Packet 2 CS CS

Hardware Terminated Read/Write and Messaging

slide-8
SLIDE 8

Packet Exchange Protocol

March 2014 8

Packet 2 Packet 3

Tx Rx

SOP Packet 1

Tx Rx

EOP

Tx Rx

EOP SOP EOP Packet Transfer Receiver-based Flow Control Error Recovery RFR

Reliable, In Order, Packet Delivery 31-4095 Outstanding Packets

slide-9
SLIDE 9

Control Symbols

March 2014 9

Control Symbol Identifier Type 0 Parm 0 CRC CMD

Transmit Side “Type 1” 24, 48, or 64 bits

Parm 1

Receive Side “Type 0”

Type1 Packet Accepted Packet Retry Packet Not Accepted Status VC Status Link Response Timestamp Start-of-packet Stomp End-of-packet Restart-from-retry Link Request Multicast Event No Operation/Ignore Timestamp Calibration

slide-10
SLIDE 10

Packet Format Overview

March 2014 10

Logical Transport Physical Logical Header

FType

Data Payload Data Payload

TT Dest ID Src ID

Physical Bits Early CRC Final CRC 10 2 4 8,16,32 8,16,32 8 to 256 bytes 16 16

Control Symbol

32,64

Control Symbol

32,64

Packets cannot be altered by RapidIO switches – CRC is constant. RapidIO Standard defines 9 Virtual Channels (guaranteed bandwidth) Virtual Channel 0 supports 4 or 8 priorities.

slide-11
SLIDE 11

Topology and Routing

March 2014 11

Mesh Star

Switch

End Point End Point End Point End Point End Point

Switch Switch Switch Switch End Point End Point End Point End Point

Switch End Point End Point End Point End Point End Point Switch Switch

Tree

  • RapidIO is topology agnostic
  • Packet DestinationID defines route
  • Routing is controlled through standard registers
  • DestinationID routing can be multicast or unicast

within each switch

Dual Star

End Point End Point End Point End Point End Point Switch Switch

slide-12
SLIDE 12

I/O System

March 2014 12

slide-13
SLIDE 13

I/O System Memory Mapping

March 2014 13 CPU1 Packet CPU2 RapidIO Memory Map

Logical Transport Physical Transaction FType Size SrcTID Memory Address Data Payload Data Payload TT Dest ID Src ID Physical Bits Early CRC Final CRC 10 2 4 8,16 8,16 4 4 8 32, 48, 64 8 to 256 bytes 16 16 Control Symbol 32,64 Control Symbol 32,64

CPU2 Internal Memory Map

slide-14
SLIDE 14

Message Passing and Data Streaming

March 2014 14

Operation Transactions Used Possible System Usage Description Packet Format Doorbell DOORBELL, RESPONSE Event notification independent of source and target memory map implementation, efficient hardware implementation Logical Layer Retry Part 2, Section 3.3.1 FType 10 Section 4.2.4 Data Message MESSAGE, MESSAGE RESPONSE Message transfer independent of source and target memory map implementation, limited number of connections/queues, efficient hardware implementation Logical Layer Retry Part 2, Section 3.3.2 FType 11 Section 4.2.5 Data Streaming Data Streaming Single Segment, Start/Continue /End Segments, Extended Header Flow Control Message transfer independent of source and target memory map implementation. Connection oriented, 64K connections/queues, support for XON/XOFF, Rate and Credit based flow control. No Logical Layer Retry Part 10, Section 3.2 FType 9 Part 10, Section 4.2

slide-15
SLIDE 15

Type 10 Packet - Doorbell

March 2014 15 Response Packet Request Packet

Logical Transport Physical

Rsvd FType TID TT Dest ID Src ID Physical Bits Final CRC

10 2 4 8 8 16

Info

16 Request Packet

TType FType TID TT Dest ID Src ID Physical Bits Final CRC

10 2 4 4 8 16 Response Packet

Status

4 “STATUS” is “Retry” when the receive queue is full.

slide-16
SLIDE 16

FType 9 – Data Streaming

March 2014 16

Single Segment Start Segment End Segment Start Segment End Segment Cont Segment Cont Segment Cont Segment Cont Segment

Max 64KB transfer, Byte granularity

slide-17
SLIDE 17

Data Streaming Flow Control

March 2014 17

Src 0 Src 1 Src 2 Server Sources send Transmit Queue Status to Server Src 0 Src 1 Src 2 Server Server Responds with Flow Control Messages

3 Flow Control Protocols Supported:

  • XON/XOFF
  • Rate Based
  • Credit Based

Sources communicate queue fill level 000/255 means “Empty” and 255/255 means “Full” Flow Control can be applied to

  • A single Stream
  • A group of Streams
  • All Streams for a port
slide-18
SLIDE 18

XON/XOFF Flow Control

March 2014 18

F E Src 0 Src 1 Src 2 Server Sources send Transmit Queue Status to Server Src 0 Src 1 Src 2 Server Server Responds with Flow Control Messages XON/XOFF

  • Service the fullest queue exclusively OR
  • Round robin among different queues OR
  • Use for “babbler” control

F E F E XON XOFF XON

slide-19
SLIDE 19

Rate Based Flow Control

March 2014 19

F E Src 0 Src 1 Src 2 Server Sources send Transmit Queue Status to Server Src 0 Src 1 Src 2 Server Server Responds with Flow Control Messages Rate Based

  • Increase rate by a fraction of current rate
  • Decrease rate by a fraction of current rate
  • XON
  • XOFF

F E F E Inc 255/255 NoChange Dec 64/255

slide-20
SLIDE 20

Credit Based Flow Control

March 2014 20

F E Src 0 Src 1 Src 2 Server Sources send Transmit Queue Status to Server Src 0 Src 1 Src 2 Server Server Responds with Flow Control Messages Credit Based

  • Allocate credits to one of up to 16 buckets
  • XON
  • XOFF

F E F E <All> 255 Cr 10 Credits left 10 Credits left <Low Prio> XOFF <All> 128 Cr

slide-21
SLIDE 21

RapidIO System Discovery

March 2014 21

uProc

Switch DSP 0 DSP 1 DSP 3 DSP 2

1

2 3 4 5

System discovery uses a recursive algorithm to

  • Discover location and connectivity of all switches and endpoints
  • Allocate destination IDs to DSPs
  • Configure switch routing tables

Discovery can be executed from any endpoint Standard system discovery supports two redundant hosts

slide-22
SLIDE 22

RapidIO for Fault Tolerant Systems

March 2014 22

uProc Switch

Switch Switch

DSP DSP DSP DSP

Switch

DSP DSP DSP DSP System Host Data Processing

  • Detection
  • Isolation
  • Notification
  • Diagnosis
  • Recovery

uProc Redundant System Host

“Fail-stop” Error Management Support

slide-23
SLIDE 23

RapidIO Form Factor & Connector Standards

March 2014 23

This Slide Intentionally Blank

slide-24
SLIDE 24

New RapidIO Features for Space

March 2014 24

  • It is possible to build Space systems with devices

compliant to the RapidIO 1.x or 2.x specifications.

  • RapidIO Trade Association’s NGSIS Task Group is

defining a profile of required features for Space compliant devices

  • Additional features for space operation are being

defined.

slide-25
SLIDE 25

March 2014 25

Bridge

SpaceWire/Fibre RapidIO Packets

SpaceWire/Fibre Bridge Encapsulates RapidIO Packets

Endpoint Endpoint

SpaceWire/Fibre Endpoints process RapidIO packets

  • In software
  • In hardware

RapidIO Endpoints only know about RapidIO

RapidIO  SpaceWire/Fibre Bridging

slide-26
SLIDE 26

March 2014 26

RapidIO Packets Encapsulate SpaceWire/Fibre

Endpoint Endpoint

SpaceWire/SpaceFibre Endpoints need only know about SpaceWire/SpaceFibre RapidIO Endpoints process SpaceWire/SpaceFibre packets

  • in software
  • in hardware

Bridge

SpaceWire SpaceFibre Message Packets (4K) Data Streaming (64K) Implementation Specific Read/Write (256 Bytes) Read/Write to Buffer

RapidIO  SpaceWire/Fibre Bridging

slide-27
SLIDE 27

RapidIO  SpaceWire/Fibre Bridging

March 2014 27

Bridge

SpaceWire/Fibre RapidIO

  • Bridge maps between SpaceWire and RapidIO routing values
  • SpaceWire Routing => RapidIO DestID, priority, packet type, etc.
  • RapidIO DestID => SpaceWire Routing Header, VC, priority, etc.
  • Bridge could recognize whether SpaceFibre or RapidIO component had

been plugged in based on presence/absence of K28.7

  • Electrical specification harmonization requires discussion
slide-28
SLIDE 28

Open Source Stack, Software Task Group

March 2014 28

Kernel User Space

APPLICATION NOT IN SCOPE OPEN SOURCE RTA STACK OPEN SOURCE KERNEL

Hardware

Portable/ Legacy Application Kernel Interface RapidIO Device Interfaces RIONET DMA Events Kernel Driver MPORT Drivers Sockets RDMA Events

CLI Open source command line interpreter for fabric management RapidIO Standard APIs RapidIO standard interface definitions and behavior Fabric Functions Implementation of RapidIO Standard APIs Fabric Management Endpoint APIs Universal programming model for endpoint functions Switch APIs Universal programming model for switch functions. Used by Alcatel- Lucent systems today. Messaging/Sockets/RDMA /Events Implementation of RapidIO Standard API’s Data Path RapidIO Device Interfaces Standardized hardware functions.

Fabric Functions Switch APIs Endpoint APIs Switch APIs High Performance/ Embedded Application RapidIO Standard APIs C L I

Messaging

RapidIO Device Interfaces

slide-29
SLIDE 29

Customization/Optimization

March 2014 29

APPLICATION NOT IN SCOPE OPEN SOURCE RTA STACK OPEN SOURCE KERNEL

Hardware

Portable/ Legacy Application Kernel Interface Fabric Functions Switch APIs Endpoint APIs RIONET DMA Events Kernel Driver MPORT Drivers RapidIO Standard APIs

CUSTOM COMPONENTS

Switch APIs

TOOLS, GUI, OPTIMIZATION

Extensions Messaging

Sockets RDMA Events

MPORT Drivers

RapidIO Device Interfaces TOOLS, GUI, OPTIMIZATION Value adders for debug/monitoring, system visualization, data interpretation, and topology specific functions MPORT Drivers Drivers optimized for and aware of executing hardware

  • perations in user

mode. Extensions Additional data path and/or fabric management services provided to applications High Performance/ Embedded Application C L I Fabric Functions Switch APIs Endpoint APIs Switch APIs RapidIO Standard APIs

Extensions Kernel User Space

slide-30
SLIDE 30

Next Steps

March 2014 30

  • Map SOIS Service Specifications to RapidIO

packets and/or software stack

  • Join the RapidIO Trade Association to
  • influence Space Rev 2 feature definition
  • Specify RapidIO  SpaceWire/Fibre bridging
  • Specify RapidIO  CAN, 1533 bridging?
  • Discuss further over beer?
slide-31
SLIDE 31

Backup Charts

March 2014

Barry Wood, Chair, RapidIO Technical Working Group barry.wood@idt.com OR twg-chair@rapidio.org

slide-32
SLIDE 32

Maintenance Packet

March 2014 32

  • Whenever a switch processing element that does not have as associated

device ID receives a maintenance packet it examines the hop_count field. If the received hop_count is zero, the access is for that switch. If the hop_count is not zero, it is decremented and the packet is sent out of the switch according to the destinationID field.

  • However, since maintenance response packets are always targeted at an

end point, the hop_count field shall always be assigned a value of 0xFF by the source of the packets to prevent them from being inadvertently accepted by an intervening device.

slide-33
SLIDE 33

Type 11 Packet - Message

March 2014 33

Logical Transport Physical

MsgLen FType Size TT Dest ID Src ID Physical Bits

10 2 4 8,16,32 8,16,32 4 4 4

Letter

2 Single-Segment Message Packet – 64 Mailboxes

TType FType TT Dest ID Src ID Physical Bits Final CRC

10 2 4 8,16,32 8,16,32 4 16 Single Segment Response Packet

Status

4

MBox XMBox

2 4

Letter

2

MBox XMBox

2

MsgLen FType Size TT Dest ID Src ID Physical Bits

10 2 4 8,16,32 8,16,32 4 4 4

Letter

2 Multi-Segment Message Packet – 4 Mailboxes

TType FType TT Dest ID Src ID Physical Bits Final CRC

10 2 4 8,16,32 8,16,32 4 16 Multi Segment Response Packet

Status

4

MBox MsgSeg

2 4

Letter

2

MBox MsgSeg

2