Overview Instruction level parallelism Dynamic Scheduling - - PDF document

1

Chapter 2

1

Instruction-Level Parallelism and Its Exploitation

Overview

• Instruction level parallelism
• Dynamic Scheduling Techniques

– Scoreboarding – Tomasulo’s Algorithm

• Reducing Branch Cost with Dynamic Hardware

2

• Reducing Branch Cost with Dynamic Hardware

Prediction

– Basic Branch Prediction and Branch-Prediction Buffers – Branch Target Buffers

• Overview of Superscalar and VLIW processors

CPI Equation

Pipeline CPI = Ideal pipeline CPI + Structural stalls + RAW stalls + WAR stalls + WAW stalls + Control stalls

Technique Reduces Loop unrolling Control stalls Basic pipeline scheduling RAW stalls

3

Dynamic scheduling with scoreboarding RAW stalls Dynamic scheduling with register renaming WAR and WAW stalls Dynamic branch prediction Control stalls Issuing multiple instructions per cycle Ideal CPI Compiler dependence analysis Ideal CPI and data stalls Software pipelining and trace scheduling Ideal CPI and data stalls Speculation All data and control stalls Dynamic memory disambiguation RAW stalls involving memory

Instruction Level Parallelism

• Potential overlap among instructions
• Few possibilities in a basic block

– Blocks are small (6-7 instructions) – Instructions are dependent

4

• Exploit ILP across multiple basic blocks

– Iterations of a loop

for (i = 1000; i > 0; i=i-1) x[i] = x[i] + s;

– Alternative to vector instructions

Basic Pipeline Scheduling

• Find sequences of unrelated instructions
• Compiler’s ability to schedule

– Amount of ILP available in the program – Latencies of the functional units

• Latency assumptions for the examples

– Standard MIPS integer pipeline

5

Standard MIPS integer pipeline – No structural hazards (fully pipelined or duplicated units – Latencies of FP operations:

Instruction producing result Instruction using result Latency FP ALU op FP ALU op 3 FP ALU op SD 2 LD FP ALU op 1 LD SD

Sample Pipeline

IF ID FP1 FP2 FP3 FP4 EX

DM

WB FP1 FP2 FP3 FP4

6

. . .

IF ID FP1 FP2 FP3 FP4

DM

WB IF ID FP1 FP2 FP3 stall stall stall

FP ALU FP ALU

IF ID FP1 FP2 FP3 FP4

DM

WB IF ID DM WB EX stall stall

FP ALU SD

2

Basic Scheduling

for (i = 1000; i > 0; i=i-1) x[i] = x[i] + s;

Sequential MIPS Assembly Code

Loop: LD F0, 0(R1) ADDD F4, F0, F2 SD 0(R1), F4 SUBI R1, R1, #8 BNEZ R1, Loop

Pipelined execution: Scheduled pipelined execution:

7

p Loop: LD F0, 0(R1) 1 stall 2 ADDD F4, F0, F2 3 stall 4 stall 5 SD 0(R1), F4 6 SUBI R1, R1, #8 7 stall 8 BNEZ R1, Loop 9 stall 10 p p Loop: LD F0, 0(R1) 1 SUBI R1, R1, #8 2 ADDD F4, F0, F2 3 stall 4 BNEZ R1, Loop 5 SD 8(R1), F4 6

Loop Unrolling

Unrolled loop (four copies):

Loop: LD F0, 0(R1) ADDD F4, F0, F2 SD 0(R1), F4 LD F6, -8(R1) ADDD F8, F6, F2 SD

• 8(R1), F8

LD F10 16(R1)

Scheduled Unrolled loop:

Loop: LD F0, 0(R1) LD F6, -8(R1) LD F10, -16(R1) LD F14, -24(R1) ADDD F4, F0, F2 ADDD F8, F6, F2 ADDD F12 F10 F2

8

LD F10, -16(R1) ADDD F12, F10, F2 SD

• 16(R1), F12

LD F14, -24(R1) ADDD F16, F14, F2 SD

• 24(R1), F16

SUBI R1, R1, #32 BNEZ R1, Loop ADDD F12, F10, F2 ADDD F16, F14, F2 SD 0(R1), F4 SD

• 8(R1), F8

SUBI R1, R1, #32 SD 16(R1), F12 BNEZ R1, Loop SD 8(R1), F16

Dynamic Scheduling

• Scheduling separates dependent instructions

– Static – performed by the compiler – Dynamic – performed by the hardware

• Advantages of dynamic scheduling

9

– Handles dependences unknown at compile time – Simplifies the compiler – Optimization is done at run time

• Disadvantages

– Can not eliminate true data dependences

Out-of-order execution (1/2)

• Central idea of dynamic scheduling

– In-order execution:

DIVD F0, F2, F4 IF ID DIV ….. ADDD F10, F0, F8 IF ID stall stall stall … SUBD F12 F8 F14 IF stall stall

10

– Out-of-order execution:

SUBD F12, F8, F14 IF stall stall ….. DIVD F0, F2, F4 IF ID DIV ….. SUBD F12, F8, F14 IF ID A1 A2 A3 A4 … ADDD F10, F0, F8 IF ID stall …..

Out-of-Order Execution (2/2)

• Separate issue process in ID:

– Issue

• decode instruction
• check structural hazards
• in-order execution

11

– Read operands

• Wait until no data hazards
• Read operands
• Out-of-order execution/completion

– Exception handling problems – WAR hazards

Dynamic Scheduling with a Scoreboard

• Details in Appendix A.7
• Allows out-of-order execution

– Sufficient resources – No data dependencies

ibl f i i d h d

12

• Responsible for issue, execution and hazards
• Functional units with long delays

– Duplicated – Fully pipelined

• CDC 6600 – 16 functional units

3

MIPS with Scoreboard

13

Scoreboard Operation

• Scoreboard centralizes hazard management

– Every instruction goes through the scoreboard – Scoreboard determines when the instruction can read its operands and begin execution

14

– Monitors changes in hardware and decides when an stalled instruction can execute – Controls when instructions can write results

• New pipeline

ID EX WB Issue

Read Regs Execution

Write

Execution Process

• Issue

– Functional unit is free (structural) – Active instructions do not have same Rd (WAW)

• Read Operands

– Checks availability of source operands R l RAW h d d i ll ( t f d

15

– Resolves RAW hazards dynamically (out-of-order execution)

• Execution

– Functional unit begins execution when operands arrive – Notifies the scoreboard when it has completed execution

• Write result

– Scoreboard checks WAR hazards – Stalls the completing instruction if necessary

Scoreboard Data Structure

• Instruction status – indicates pipeline stage
• Functional unit status

Busy – functional unit is busy or not Op – operation to perform in the unit (+, -, etc.)

16

Fi – destination register Fj, Fk – source register numbers Qj, Qk – functional unit producing Fj, Fk Rj, Rk – flags indicating when Fj, Fk are ready

• Register result status – FU that will write registers

Scoreboard Data Structure (1/3)

Instruction Issue Read operands Execution completed Write

LD F6, 34(R2)

Y Y Y Y

LD F2, 45(R3)

Y Y Y

MULTD F0, F2, F4

Y

SUBD F8, F6, F2

Y

DIVD F10, F0, F6

Y

ADDD F6, F8, F2

17

Name Busy Op Fi Fj Fk Qj Qk Rj Rk Integer Y Load F2 R3 N Mult1 Y Mult F0 F2 F4 Integer N Y Mult2 N Add Y Sub F8 F6 F2 Integer Y N Divide Y Div F10 F0 F6 Mult1 N Y F0 F2 F4 F6 F8 F10 F12 . . . F30

Functional Unit

Mult1 Int Add Div

Scoreboard Data Structure (2/3)

18

4

Scoreboard Data Structure (3/3)

19

Scoreboard Algorithm

20

Scoreboard Limitations

• Amount of available ILP
• Number of scoreboard entries

– Limited to a basic block – Extended beyond a branch

21

• Number and types of functional units

– Structural hazards can increase with DS

• Presence of anti- and output- dependences

– Lead to WAR and WAW stalls

Tomasulo Approach

• Another approach to eliminate stalls

– Combines scoreboard with – Register renaming (to avoid WAR and WAW)

• Designed for the IBM 360/91

22

g

– High FP performance for the whole 360 family – Four double precision FP registers – Long memory access and long FP delays

• Can support overlapped execution of

multiple iterations of a loop

Tomasulo Approach

23

Stages

• Issue

– Empty reservation station or buffer – Send operands to the reservation station – Use name of reservation station for operands

E t

24

• Execute

– Execute operation if operands are available – Monitor CDB for availability of operands

• Write result

– When result is available, write it to the CDB

5

Example (1/2)

25

Example (2/2)

26

Tomasulo’s Algorithm

27

An enhanced and detailed design in Fig. 2.12 of the textbook

Loop: LD F0, 0(R1) MULTD F4,F0,F2 SD 0(R1), F4 SUBI R1, R1, #8 BNEZ R1, Loop

Loop Iterations

28

Dynamic Hardware Prediction

• Importance of control dependences

– Branches and jumps are frequent – Limiting factor as ILP increases (Amdahl’s law)

• Schemes to attack control dependences

– Static

i ( ll h i li )

29

• Basic (stall the pipeline)
• Predict-not-taken and predict-taken
• Delayed branch and canceling branch

– Dynamic predictors

• Effectiveness of dynamic prediction schemes

– Accuracy – Cost

Basic Branch Prediction Buffers

IR: PC

Branch Instruction

+

Branch Target a.k.a. Branch History Table (BHT) - Small direct-mapped cache of T/NT bits

30

PC: BHT

PC + 4

T (predict taken) NT (predict not- taken)

6

N-bit Branch Prediction Buffers

Use an n-bit saturating counter Only the loop exit causes a misprediction 2-bit predictor almost as good as any general n-bit predictor

31

Prediction Accuracy of a 4K-entry 2-bit Prediction Buffer

32

Branch-Target Buffers

• Further reduce control stalls (hopefully to 0)
• Store the predicted address in the buffer
• Access the buffer during IF

33

Prediction with BTF

34

Performance Issues

• Limitations of branch prediction schemes

– Prediction accuracy (80% - 95%)

• Type of program
• Size of buffer

– Penalty of misprediction

35

y p

• Fetch from both directions to reduce penalty

– Memory system should:

• Dual-ported
• Have an interleaved cache
• Fetch from one path and then from the other

Five Primary Approaches in use for Multiple-issue Processors

36