QoS QoS Aware Aware BiNoC BiNoC Architecture Architecture Shih - - PowerPoint PPT Presentation

QoS QoS Aware Aware BiNoC BiNoC Architecture Architecture

Shih Shih-

• Hsin

Hsin Lo, Ying Lo, Ying-

• Cherng

Cherng Lan Lan, , Hsin Hsin-

• Hsien

Hsien Yeh Yeh, , Wen Wen-

• Chung Tsai,

Chung Tsai, Yu Yu-

• Hen

Hen Hu Hu, and Sao , and Sao-

• Jie

Jie Chen Chen

Ying Cherng Lan Ying-Cherng Lan

CAD System Lab Graduate Institute of Electronics Engineering National Taiwan University Taipei, Taiwan, ROC

Introduction Introduction

The trend toward many-core processing chips is now a well The trend toward many-core processing chips is now a well established one Interconnect delay dominates gate g delay

– Global interconnect delay continuously increasing – Need multiple clock Need multiple clock cycles to cross chip die – Limits the performance

• f microprocessors

Page 2

• f microprocessors

Communication Centric Design Communication Centric Design

The design concept of a system is moving gradually from The design concept of a system is moving gradually from computation-centric to communication centric. Conventional bus-based architecture becomes no longer a feasible communication scheme in terms of bandwidth and scalability communication scheme in terms of bandwidth, and scalability.

Page 3

Network on Chip Network on Chip

Network on Chip (NoC) is a promising solution to mitigate the i i i ti l it d id b tt ever increasing communication complexity and provide better scalability.

W J Dally and B Towles “Route Packets Not Wires: On Chip Interconnection Networks ” in – W.J. Dally and B. Towles, Route Packets, Not Wires: On-Chip Interconnection Networks, in Proceedings of DAC, pp. 684-689, Jun. 2001. –

• L. Benini and G. DeMicheli, “Networks on Chips: a New SoC Paradigm,” IEEE Computer, vol.

35, no. 1, pp. 70-78, Jan. 2002. –

• A. Jantsch and H. Tenhunen (Eds.), Networks on Chip, Kluwer Academic Publishers, 2003.

i wire PE PE

Page 4

Quality of Service for Quality of Service for NoC NoC

Since many of the system applications have real-time requirements Since many of the system applications have real-time requirements, the system and the network have to be predictable. To proceed a practical application, there are numerous type of packets in different importance need to be transmitted packets in different importance need to be transmitted.

• GS (guaranteed service) : guaranteed in latency. (e.g., real time stream)

BE (b t ff t) t d l i t

• BE (best effort) : guaranteed only in correctness

Page 5

How to provide How to provide QoS QoS for for NoC NoC

To provide QoS in network on chip, two To provide QoS in network on chip, two communication scheme have been proposed.

– Connection-oriented mechanism (Circuit switching ) – Connection-less mechanism (Packet- switching)

Page 6

Related Works Related Works

It is proven that connection less scheme is better in a variable bit rate application application.

– M. D. Harmanci, “Quantitative modelling and comparison of communication schemes to guarantee Quality-of-Service in Networks-on-Chip” ISCAS ’05

In a typical connection-less QoS scheme, the packets with different priorities can be adapted to a virtual channel NoC router.

– E. Bolotin, “QNoC: QoS architecture and design process for network on chip”, J.

• Syst. Architecture: EUROMICRO J ’04

Page 7

Motivational Example Motivational Example

Conventional uni-directional inter-router communication channel Under the typical uni-directional NoC, only GS1 is granted while yp , y g another channel with opposite direction is used by the BE1 flow with the lower QoS requirement.

Page 8

Motivational Example Motivational Example

Enhance real-time traffic routing flexibility by dynamic reconfigurable bi-directional channel. The inter-router arbitration can be applied to further enhance the channel usage priority for the GS traffic. channel usage priority for the GS traffic.

Page 9

QoS QoS Aware Aware BiNoC BiNoC Architecture Architecture

Bidirectional channel direction control module are implemented for inter-router Bidirectional channel direction control module are implemented for inter router arbitration.

Page 10

Prioritized Virtual Channel Management and Inter Prioritized Virtual Channel Management and Inter-

• router

router Arbitration Arbitration Arbitration Arbitration

GS packets always has the higher priority to get the output bandwidth during the intra-router arbitration. g inter-router arbitration improve the channel utilization for GS packets.

Page 11

Inter Inter-

• Router Channel Direction Control Scheme

Router Channel Direction Control Scheme

Config ration of a bi directional channel is managed b a Configuration of a bi-directional channel is managed by a finite state machine in the channel control modules.

Page 12

Inter Inter-

• Router Transmission Scheme

Router Transmission Scheme

The channel state reflects whether this port can be used to t t d t tl t

• utput data currently or not.

The channel can be used to

• utput data at present.

p p

Transitional state between idle and free , used to accommodate inter- router communication delay, not for

• utput currently

Page 13

The channel direction is inward, not for

• utput currently.
• utput currently.

Starvation Avoidance Starvation Avoidance

To prevent the inter-router starvation problem, one of these two FSMs will be designated with a higher priority (HP) and the other with a will be designated with a higher priority (HP) and the other with a lower priority (LP).

Page 14

Prioritized Channel Control FSM Prioritized Channel Control FSM

High priority FSM Low priority FSM

Page 15

Prioritized Routing Restriction Prioritized Routing Restriction

A prioriti ed ro ting restriction is applied to lea e more A prioritized routing restriction is applied to leave more available communication bandwidth for GS traffic.

BE traffic : deterministic routing – BE traffic : deterministic routing – GS traffic : adaptive routing – Odd-Even Turn model is applied to prevent deadlock pp p

• G. M. Chiu, “The Odd-Even Turn Model for Adaptive

Routing,” IEEE Transactions on Parallel and Distributed Systems, vol. 11, no. 7, pp. 729-738, Jul. 2000.

The prioritized routing can help the GS traffic to exploit more channel resource for transmission

y , , , pp ,

more channel resource for transmission.

Page 16

Experimental Setup Experimental Setup

Simulation setup Simulation setup

– 8x8 2-D mesh – Cycle accurate HDL simulation – 32 flit buffer implemented in 4 virtual channels in each direction – Uniform, transpose and hotspot traffic

Page 17

Experimental Results Experimental Results

latency results between BiNoC 4VC, and BiNoC QoS. latency results between BiNoC_4VC, and BiNoC_QoS.

• GS traffic occupies 20% of the total traffic

150 200 250 300

ncy(cycle) BiNoC_4VC BiNoC_QoS(GS) BiNoC_QoS(BE)

150 200 250 300

ncy(cycle) BiNoC_4VC BiNoC_QoS(GS) BiNoC_QoS(BE)

50 100 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

late flitinjectionrate(flit/node/cycle)

50 100 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

late flitinjectionrate(flit/node/cycle)

250 300

BiNoC_4VC BiN C Q S(GS)

uniform transpose

50 100 150 200

latency(cycle) BiNoC_QoS(GS) BiNoC_QoS(BE)

Page 18

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

flitinjectionrate(flit/node/cycle)

hotspot

Experimental Results Experimental Results

latency results between NoC_QoS, and BiNoC_QoS. Inter-router arbitration can further reduce the latency of GS packets because of the doubled bandwidth utilization flexibility

150 200 250 300

ncy(cycle) NoC_QoS(GS) NoC_QoS(BE) BiNoC_QoS(GS) BiNoC_QoS(BE)

150 200 250 300

ncy(cycle) NoC_QoS(GS) NoC_QoS(BE) BiNoC_QoS(GS) BiNoC_QoS(BE)

50 100 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

late flitinjectionrate(flit/node/cycle)

50 100 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

laten flitinjectionrate(flit/node/cycle)

uniform transpose

250 300

) NoC_QoS(GS) NoC QoS(BE)

50 100 150 200

latency(cycle NoC_QoS(BE) BiNoC_QoS(GS) BiNoC_QoS(BE)

Page 19

hotspot

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

flitinjectionrate(flit/node/cycle)

Experimental Results Experimental Results

latency results between BiNoC_QoS, and BiNoC_QoS_OE. h i i i d i i i h l h k id h bl ki The prioritized routing restriction can help the GS packets to avoid the blocking nodes thus reduces the latency.

300 300 100 150 200 250 300

atency(cycle) BiNoC_QoS(GS) BiNoC_QoS(BE) BiNoC_QoS_OE(GS) BiNoC_QoS_OE(BE)

100 150 200 250 300

atency(cycle) BiNoC_QoS(GS) BiNoC_QoS(BE) BiNoC_QoS_OE(GS) BiNoC_QoS_OE(BE)

if t

50 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

la flitinjectionrate(flit/node/cycle)

50 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

la flitinjectionrate(flit/node/cycle)

uniform transpose

250 300

) BiNoC_QoS(GS)

50 100 150 200

latency(cycle) BiNoC_QoS(BE) BiNoC_QoS_OE(GS) BiNoC_QoS_OE(BE)

Page 20

hotspot

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

flitinjectionrate(flie/node/cycle)

Experimental Results Experimental Results

latency results between NoC QoS, and BiNoC QoS in various GS ratios latency results between NoC_QoS, and BiNoC_QoS in various GS ratios

• f the total traffic.

Results are obtained by running hotspot traffic.

Page 21

Experimental Results Experimental Results

latency results between BiNoC QoS, and BiNoC QoS OE in various GS latency results between BiNoC_QoS, and BiNoC_QoS_OE in various GS ratios of the total traffic. Results are obtained by running hotspot traffic.

Page 22

Experimental Results Experimental Results

Flit consumption rate of GS and BE packets under hotspot traffic with 20% GS Flit consumption rate of GS and BE packets under hotspot traffic with 20% GS ratio to the total traffic.

0.3

N C Q S(GS)

0.1 0.15 0.2 0.25

• nsumptionrate

flit/node/cycle) NoC_QoS(GS) BiNoC_QoS(GS) BiNoC_QoS_OE(GS)

0.05 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

co (f flitinjectionrate(flit/node/cycle)

0.15 0.2 0.25

mptionrate

• de/cycle)

0.05 0.1

consum (flit/no NoC_QoS(BE) BiNoC_QoS(BE) BiNoC_QoS_OE(BE)

Page 23

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

flitinjectionrate(flit/node/cycle)

Closing Remarks Closing Remarks

A connection-less QoS mechanism based on the bi- A connection-less QoS mechanism based on the bi- directional channel NoC (BiNoC) backbone is proposed. A flexible virtual channel management mechanism and a novel prioritized routing policy are integrated novel prioritized routing policy are integrated the proposed inter-router arbitration scheme can significantly improve the channel utilization for GS packets

Page 24