Chapter 6 1 CSc 314 T W Bennet Mississippi College Pipelining - - PowerPoint PPT Presentation

Chapter 6

CSc 314 · T W Bennet · Mississippi College

1

Pipelining

Making the CPU faster.

• Form of parallel processing.
• New instructions are started before previous ones finish.
• At any given time, several instructions are active at

various stages of completion.

CSc 314 · T W Bennet · Mississippi College

2

Laundry

CSc 314 · T W Bennet · Mississippi College

3

CSc 314 · T W Bennet · Mississippi College

4

Efficiency

• Does not decrease the execution time of any individual

instruction.

• Increases the throughput of instructions.
• Efficient for long streams of instructions: The pipeline

must be full for maximum benefit.

CSc 314 · T W Bennet · Mississippi College

5

Pipeline Stages

Break each instruction up into five stages.

• IF: Instruction Fetch
• ID: Instruction Decode and register read.
• EX: EXecution or address calculation.
• MEM: data MEMory access.
• WB: Write Back to register.

CSc 314 · T W Bennet · Mississippi College

6

The Pipeline: An Instruction Assembly Line 1

M[0x412918] = 4981 M[0x41292c] = 79

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $16 = 41 $17 = -111 $18 = 45 $19 = 7321 $20 = 499 $21 = 10 add $18, $11, $13 lw $16, 0($13) sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $18, $11, $13

Memory Data

CSc 314 · T W Bennet · Mississippi College

7

The Pipeline: An Instruction Assembly Line 2

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $16 = 41 $17 = -111 $18 = 45 $19 = 7321 $20 = 499 $21 = 10 sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) lw $16, 0($13) add $18, $11, $13 $11 = 16 $13 = 0x412918 M[0x412918] = 4981 M[0x41292c] = 79 lw $16, 0($13) add $18, $11, $13

Memory Data

CSc 314 · T W Bennet · Mississippi College

8

The Pipeline: An Instruction Assembly Line 3

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $16 = 41 $17 = -111 $18 = 45 $19 = 7321 $20 = 499 $21 = 10

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $18, $11, $13 $11 = 16 $13 = 0x412918 $18 = 0x412928 lw $16, 0($13) $13 = 0x412918 sub $21, $10, $11 M[0x412918] = 4981 M[0x41292c] = 79 sub $21, $10, $11 lw $16, 0($13) add $18, $11, $13

Memory Data

CSc 314 · T W Bennet · Mississippi College

9

The Pipeline: An Instruction Assembly Line 4

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $16 = 41 $17 = -111 $18 = 45 $19 = 7321 $20 = 499 $21 = 10 add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $18, $11, $13 $11 = 16 $13 = 0x412918 $18 = 0x412928 lw $16, 0($13) $13 = 0x412918 $13 + 0 = 0x412918 sub $21, $10, $11

• r $19, $14, $13

$11 = 16 $10 = 47 M[0x412918] = 4981 M[0x41292c] = 79

• r $19, $14, $13

lw $16, 0($13) sub $21, $10, $11 add $18, $11, $13

Memory Data

CSc 314 · T W Bennet · Mississippi College

10

The Pipeline: An Instruction Assembly Line 5

lw $16, 0($13) $13 = 0x412918 $13 + 0 = 0x412918

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $16 = 41 $17 = -111 $18 = 45 $19 = 7321 $20 = 499 $21 = 10 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $20, $18, $12 sub $21, $10, $11 $10 = 47 $11 = 16 $13 = 0x412918 $14 = 32 $16 = 4981 $18 = 0x412928 $11 = 16 $13 = 0x412918 add $18, $11, $13 $21 = 11

• r $19, $14, $13

M[0x412918] = 4981 M[0x41292c] = 79 add $20, $18, $12

• r $19, $14, $13

sub $21, $10, $11 lw $16, 0($13) add $18, $11, $13

Memory Data

CSc 314 · T W Bennet · Mississippi College

11

The Pipeline: An Instruction Assembly Line 6

lw $16, 0($13) $13 = 0x412918 $13 + 0 = 0x412918 sub $21, $10, $11 $10 = 47 $11 = 16 $21 = 11

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $16 = 41 $17 = -111 $19 = 7321 $20 = 499 $21 = 10 xor $10, $18, $16 sw $21, 8($19) $18 = 0x412928 $16 = 4981 lw $11, 4($18) add $20, $18, $12 $12 = 29 $18 = 0x412928 $13 = 0x412918 $14 = 32

• r $19, $14, $13

$19 = 0x412938 M[0x412918] = 4981 M[0x41292c] = 79 lw $16, 0($13) sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) add $18, $11, $13

Memory Data

CSc 314 · T W Bennet · Mississippi College

12

The Pipeline: An Instruction Assembly Line 7

sub $21, $10, $11 $10 = 47 $11 = 16

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $17 = -111 $19 = 7321 $20 = 499 $21 = 10 sw $21, 8($19) $18 = 0x412928 $16 = 4981 $21 = 11 $12 = 29 $18 = 0x412928 $20 = 0x412945 lw $11, 4($18) $18 = 0x412928 xor $10, $18, $16 $13 = 0x412918 $14 = 32 $19 = 0x412938

• r $19, $14, $13

add $20, $18, $12 M[0x412918] = 4981 M[0x41292c] = 79 lw $16, 0($13) sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 add $18, $11, $13

Memory Data

CSc 314 · T W Bennet · Mississippi College

13

The Pipeline: An Instruction Assembly Line 8

$12 = 29 $18 = 0x412928 $20 = 0x412945 add $20, $18, $12

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $17 = -111 $19 = 7321 $20 = 499 $18 = 0x412928 $16 = 4981 $21 = 11 $19 = 0x412938 $13 = 0x412918 $14 = 32

• r $19, $14, $13

lw $11, 4($18) $18 = 0x412928 $18 + 4 = 0x41292c xor $10, $18, $16 $16 = 4981 $18 = 0x412928 sw $21, 8($19) M[0x412918] = 4981 M[0x41292c] = 79 lw $16, 0($13) sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $18, $11, $13

Data Memory

CSc 314 · T W Bennet · Mississippi College

14

The Pipeline: An Instruction Assembly Line 9

$12 = 29 $18 = 0x412928 add $20, $18, $12

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $17 = -111 $20 = 499 $18 = 0x412928 $16 = 4981 $21 = 11 $19 = 0x412938 $20 = 0x412945 lw $11, 4($18) $18 = 0x412928 $18 + 4 = 0x41292c $11 = 79 xor $10, $18, $16 $16 = 4981 $18 = 0x412928 $10 = 0x413a5d sw $21, 8($19) $19 = 0x412938 $21 = 11 M[0x412918] = 4981 M[0x41292c] = 79 lw $16, 0($13) sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $18, $11, $13

Data Memory

CSc 314 · T W Bennet · Mississippi College

15

The Pipeline: An Instruction Assembly Line 10

lw $11, 4($18) $18 = 0x412928 $18 + 4 = 0x41292c xor $10, $18, $16 $16 = 4981 $18 = 0x412928 $10 = 0x413a5d sw $21, 8($19) $19 = 0x412938 $19 + 8 = 0x412940

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $11 = 16 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $17 = -111 $18 = 0x412928 $16 = 4981 $21 = 11 $19 = 0x412938 $20 = 0x412945 $11 = 79 $21 = 11 M[0x412918] = 4981 M[0x41292c] = 79 lw $16, 0($13) sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $18, $11, $13

Data Memory

CSc 314 · T W Bennet · Mississippi College

16

The Pipeline: An Instruction Assembly Line 11

xor $10, $18, $16 $16 = 4981 $18 = 0x412928 sw $21, 8($19) $19 = 0x412938 $19 + 8 = 0x412940

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$10 = 47 $12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $17 = -111 $18 = 0x412928 $16 = 4981 $21 = 11 $19 = 0x412938 $20 = 0x412945 $11 = 79 $10 = 0x413a5d M[0x412940]=11 $21 = 11 M[0x412918] = 4981 M[0x41292c] = 79 lw $16, 0($13) sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $18, $11, $13

Data Memory

CSc 314 · T W Bennet · Mississippi College

17

The Pipeline: An Instruction Assembly Line 12

sw $21, 8($19) $19 = 0x412938 $19 + 8 = 0x412940

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

$12 = 29 $13 = 0x412918 $14 = 32 $15 = -5 $17 = -111 $18 = 0x412928 $16 = 4981 $21 = 11 $19 = 0x412938 $20 = 0x412945 $11 = 79 $10 = 0x413a5d $21 = 11 M[0x412940]=11 M[0x412918] = 4981 M[0x41292c] = 79 lw $16, 0($13) sub $21, $10, $11

• r $19, $14, $13

add $20, $18, $12 lw $11, 4($18) xor $10, $18, $16 sw $21, 8($19) add $18, $11, $13

Data Memory

CSc 314 · T W Bennet · Mississippi College

18

Pipelined Datapath

Instruction# memory Address 4 32 Add Add# result Shift# left 2 Instruction IF/ID EX/MEM MEM/WB M# u# x 1 Add PC Write# data M# u# x 1 Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 16 Sign# extend Write# register Write# data Read# data 1 ALU# result M# u# x ALU Zero ID/EX Data# memory Address

CSc 314 · T W Bennet · Mississippi College

19

Pipeline Registers

• Each is quite large.
• Holds the results of each stage ready for the next.
• Serves the function of the conveyer belt.

CSc 314 · T W Bennet · Mississippi College

20

Pipeline Flow 1

Instruction# memory Address 4 32 Add Add# result Shift# left 2 Instruction IF/ID EX/MEM MEM/WB M# u# x 1 Add PC Write# data M# u# x 1 Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 16 Sign# extend Write# register Write# data Read# data 1 ALU# result M# u# x ALU Zero ID/EX

Instruction fetch lw $10,

20($1)

Address Data# memory

Clock 1

CSc 314 · T W Bennet · Mississippi College

21

Pipeline Flow 2

Instruction# memory Address 4 32 Add Add# result Shift# left 2 Instruction IF/ID EX/MEM MEM/WB M# u# x 1 Add PC Write# data M# u# x 1 Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 16 Sign# extend Write# register Write# data Read# data 1 ALU# result M# u# x ALU Zero ID/EX

Instruction decode lw $10,

20($1)

Instruction fetch sub $11, $2, $3

Address Data# memory

Clock 2

CSc 314 · T W Bennet · Mississippi College

22

Pipeline Flow 3

Instruction# memory Address 4 Add Add# result Shift# left 2 Instruction IF/ID EX/MEM MEM/WB M# u# x 1 Add PC Write# data M# u# x 1 Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 Write# register Write# data Read# data 1 ALU# result M# u# x ALU Zero ID/EX

Execution lw $10,

20($1)

Instruction decode sub $11, $2, $3

32 16 Sign# extend Address Data# memory

Clock 3

CSc 314 · T W Bennet · Mississippi College

23

Pipeline Flow 4

Instruction# memory Address 4 Add Add# result Shift# left 2 Instruction IF/ID EX/MEM MEM/WB M# u# x 1 Add PC Write# data M# u# x 1 Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 32 16 Sign# extend Write# register Write# data

Memory lw $10,

20($1)

Read# data 1 ALU# result M# u# x ALU Zero ID/EX

Execution sub $11, $2, $3

Data# memory Address

Clock 4

CSc 314 · T W Bennet · Mississippi College

24

Pipeline Flow 5

Instruction# memory Address 4 32 Add Add# result 1 ALU# result Zero Shift# left 2 Instruction IF/ID EX/MEM ID/EX MEM/WB

Write back

M# u# x 1 Add PC Write# data M# u# x 1 Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 16 Sign# extend M# u# x ALU Read# data Write# register Write# data

lw $10,

20($1)

Memory sub $11, $2, $3

Address Data# memory

Clock 5

CSc 314 · T W Bennet · Mississippi College

25

Pipeline Flow 6

Instruction# memory Address 4 32 Add Add# result 1 ALU# result Zero Shift# left 2 Instruction IF/ID EX/MEM ID/EX MEM/WB

Write back

M# u# x 1 Add PC Write# data M# u# x 1 Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 16 Sign# extend M# u# x ALU Read# data Write# register Write# data

sub $11, $2, $3

Address Data# memory

Clock 6

CSc 314 · T W Bennet · Mississippi College

26

Pipeline Control: High Level

Instructions take their control signals with them.

CSc 314 · T W Bennet · Mississippi College

27

Pipeline Control: Detailed

CSc 314 · T W Bennet · Mississippi College

28

Pipeline Control Example: 1

Instruction# memory Instruction# [20-16] MemtoReg ALUOp Branch RegDst ALUSrc 4 Instruction# [15-0] M# u# x 1

Add Add# result

Registers Write# register Write# data Read# data 1 Read# data 2 Read# register 1 Read# register 2 Sign# extend M# u# x 1

ALU# result Zero

ALU# control Shift# left 2 RegWrite MemRead Control

ALU

Instruction# [15-11] EX M WB M WB WB Instruction IF/ID EX/MEM ID/EX

ID: before<1> EX: before<2> MEM: before<3> WB: before<4>

MEM/WB

IF: lw $10, 20($1)

000 00 0000 000 00 00 00 M# u# x 1

Add

PC Data# memory Address Write# data Read# data M# u# x 1 MemWrite Address

Clock 1

CSc 314 · T W Bennet · Mississippi College

29

Pipeline Control Example: 2

WB EX M Instruction# memory MemtoReg ALUOp Branch RegDst ALUSrc 4 M# u# x 1

Add Add# result

Write# register Write# data M# u# x 1

ALU# result Zero

ALU# control Shift# left 2 RegWrite

ALU

M WB WB Instruction IF/ID EX/MEM ID/EX

ID: lw $10, 20($1) EX: before<1> MEM: before<2> WB: before<3>

MEM/WB

IF: sub $11, $2, $3

010 11 0001 000 00 00 00 M# u# x 1

Add

PC Write# data Read# data M# u# x 1 lw Control Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 X 10 20 X 1 Instruction# [20-16] Instruction# [15-0] Sign# extend Instruction# [15-11] 20 $X $1 10 X MemRead MemWrite Data# memory Address Address

Clock 2

CSc 314 · T W Bennet · Mississippi College

30

Pipeline Control Example: 3

Instruction# memory Address Instruction# [20-16] MemtoReg Branch ALUSrc 4 Instruction# [15-0] 1

Add Add# result

Registers Write# register Write# data Read# data 1 Read# data 2 Read# register 1 Read# register 2

ALU# result

Shift# left 2 RegWrite MemRead Control

ALU

Instruction# [15-11] EX M WB WB Instruction IF/ID EX/MEM ID/EX

ID: sub $11, $2, $3 EX: lw $10, . . . MEM: before<1> WB: before<2>

MEM/WB

IF: and $12, $4, $5

000 10 1100 010 11 00 1 00 M# u# x 1

Add

PC Write# data Read# data M# u# x 1 MemWrite sub 11 X X 3 2 X $3 $2 X 11 $1 20 10 M# u# x M# u# x 1 ALUOp RegDst ALU# control M WB

Zero

Sign# extend Data# memory Address

Clock 3

CSc 314 · T W Bennet · Mississippi College

31

Pipeline Control Example: 4

WB EX M Instruction# memory Address MemtoReg ALUOp Branch RegDst ALUSrc 4 1

Add Add# result

Write# register Write# data 1

ALU# result

ALU# control Shift# left 2 RegWrite M WB Instruction IF/ID EX/MEM ID/EX

ID: and $12, $2, $3 EX: sub $11, . . . MEM: lw $10, . . .WB: before<1>

MEM/WB

IF: or $13, $6, $7

000 10 1100 000 10 10 1 11 1 M# u# x 1

Add

PC Write# data M# u# x 1 and Control Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 12 X X 5 4 Instruction# [20-16] Instruction# [15-0] Instruction# [15-11] X $5 $4 X 12 MemRead MemWrite $3 $2 11 M# u# x M# u# x

ALU

Address Read# data Data# memory 10 WB

Zero

Sign# extend

Clock 4

CSc 314 · T W Bennet · Mississippi College

32

Pipeline Control Example: 5

Instruction# memory Address Instruction# [20-16] Branch ALUSrc 4 Instruction# [15-0] 1

Add Add# result

Registers Write# register Write# data Read# data 1 Read# data 2 Read# register 1 Read# register 2

ALU# result

Shift# left 2 RegWrite MemRead Control

ALU

Instruction# [15-11] EX M WB Instruction IF/ID EX/MEM ID/EX

ID: or $13, $6, $7 EX: and $12, . . . MEM: sub $11, . . . WB: lw $10, . . .

MEM/WB

IF: add $14, $8, $9

000 10 1100 000 10 10 1 10 M# u# x 1

Add

PC Write# data Read# data M# u# x 1 MemWrite

• r

13 X X 7 6 X $7 $6 X 13 $4 M# u# x M# u# x 1 ALUOp RegDst ALU# control M WB 11 10 10 $5 12 WB MemtoReg 1 1

Zero

Data# memory Address Sign# extend

Clock 5

CSc 314 · T W Bennet · Mississippi College

33

Pipeline Control Example: 6

WB EX M Instruction# memory Address MemtoReg ALUOp Branch RegDst ALUSrc 4 1

Add Add# result

1

ALU# result

ALU# control Shift# left 2 RegWrite M WB Instruction IF/ID EX/MEM ID/EX

ID: add $14, $8, $9 EX: or $13, . . . MEM: and $12, . . . WB: sub $11, . . .

MEM/WB

IF: after<1>

000 10 1100 000 10 10 1 10 1 M# u# x 1

Add

PC Write# data M# u# x 1 add Control Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 14 X X 9 8 Instruction# [20-16] Instruction# [15-0] Instruction# [15-11] X $9 $8 X 14 MemRead MemWrite $7 $6 13 M# u# x M# u# x

ALU

Read# data 12 WB 11 11 Write# register Write# data

Zero

Data# memory Address Sign# extend

Clock 6

CSc 314 · T W Bennet · Mississippi College

34

Pipeline Control Example: 7

Instruction# memory Address Instruction# [20-16] Branch ALUSrc 4 Instruction# [15-0] 1

Add Add# result

Registers Write# register Write# data

ALU# result

Shift# left 2 RegWrite MemRead Control

ALU

Instruction# [15-11] Sign# extend EX M WB Instruction IF/ID EX/MEM ID/EX

ID: after<1> EX: add $14, . . . MEM: or $13, . . .WB: and $12, . . .

MEM/WB

IF: after<2>

000 00 0000 000 10 10 1 10 M# u# x 1

Add

PC Write# data Read# data M# u# x 1 MemWrite $8 M# u# x M# u# x 1 ALUOp RegDst ALU# control M WB 13 12 12 $9 14 WB MemtoReg 1 Read# data 1 Read# data 2 Read# register 1 Read# register 2

Zero

Data# memory Address

Clock 7

CSc 314 · T W Bennet · Mississippi College

35

Pipeline Control Example: 8

WB EX M Instruction# memory Address MemtoReg ALUOp Branch RegDst ALUSrc 4 1

Add Add# result

1

ALU# result Zero

ALU# control Shift# left 2 RegWrite M WB Instruction IF/ID EX/MEM ID/EX

ID: after<2> EX: after<1> MEM: add $14, . . . WB: or $13, . . .

MEM/WB

IF: after<3>

000 00 0000 000 00 00 10 1 M# u# x 1

Add

PC Write# data M# u# x 1 Control Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 Instruction# [20-16] Instruction# [15-0] Sign# extend Instruction# [15-11] MemRead MemWrite M# u# x M# u# x

ALU

Read# data 14 WB 13 13 Write# register Write# data Data# memory Address

Clock 8

CSc 314 · T W Bennet · Mississippi College

36

Pipeline Control Example: 9

WB EX M Instruction# memory Address MemtoReg ALUOp Branch RegDst ALUSrc 4 1

Add Add# result

1

ALU# result Zero

ALU# control Shift# left 2 RegWrite M WB Instruction IF/ID EX/MEM ID/EX

ID: after<3> EX: after<2> MEM: after<1> WB: add $14, . . .

MEM/WB

IF: after<4>

000 00 0000 000 00 00 00 1 M# u# x 1

Add

PC Write# data M# u# x 1 Control Registers Read# data 1 Read# data 2 Read# register 1 Read# register 2 Instruction# [20-16] Instruction# [15-0] Sign# extend Instruction# [15-11] MemRead MemWrite M# u# x M# u# x

ALU

Read# data WB 14 14 Write# register Write# data Data# memory Address

Clock 9

CSc 314 · T W Bennet · Mississippi College

37

CSc 314 · T W Bennet · Mississippi College

38

Data Hazards

An instruction which uses a value may fetch it before the instruction which computes it has stored the value. add $s0, $t0, $t1 sub $t2, $s0, $t3

CSc 314 · T W Bennet · Mississippi College

39

Data Hazards in the Pipeline 1

addi $10, $10, 4 $10 = 0x401208 $10 = 0x40120c

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 sw $11, 0($10) $10 = 0x401208 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $11 = 781 $10 = 0x401208 $10 + 0 = 0x401208 M[0x401208] = 34

Data Memory

CSc 314 · T W Bennet · Mississippi College

40

Data Hazards in the Pipeline 2

addi $10, $10, 4 $10 = 0x401208

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $10 = 0x401208 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 $10 = 0x40120c lw $11, 0($10) $10 = 0x401208 M[0x401208] = 34 $10 + 0 = 0x401208 $11 = 34 add $11, $11, $11 $11 = 781 sub $12, $11, $10 $11 = 781 $12 = 4981 sw $11, 0($10) sw $11, 0($10) $11 = 1562

Memory Data

CSc 314 · T W Bennet · Mississippi College

41

Data Hazards in the Pipeline 3

lw $11, 0($10) $10 = 0x401208 $10 + 0 = 0x401208

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 sw $11, 0($10) $10 = 0x40120c $11 = 34 add $11, $11, $11 $11 = 781 $11 = 1562 sub $12, $11, $10 $11 = 781 $10 = 0x401208 $12 = 0x400efb sw $11, 0($10) $10 = 0x40120c $11 = 781

Memory Data

CSc 314 · T W Bennet · Mississippi College

42

Data Hazards in the Pipeline 4

add $11, $11, $11 $11 = 781

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 sw $11, 0($10) $10 = 0x40120c $11 = 34 $11 = 1562 sub $12, $11, $10 $11 = 781 $10 = 0x401208 $12 = 0x400efb sw $11, 0($10) $10 = 0x40120c $11 = 781 $10 + 0 = 0x40120c

Memory Data

CSc 314 · T W Bennet · Mississippi College

43

Data Hazards in the Pipeline 5

sub $12, $11, $10 $11 = 781 $10 = 0x401208 sw $11, 0($10) $10 = 0x40120c $11 = 781

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 sw $11, 0($10) $10 = 0x40120c $11 = 1562 $12 = 0x400efb M[0x40120c] = 781 $10 + 0 = 0x40120c

Memory Data

CSc 314 · T W Bennet · Mississippi College

44

Data Hazards in the Pipeline 6

Write-Back Fetch Decode Execute Memory Instruction

Memory Register File

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 sw $11, 0($10) $10 = 0x40120c $11 = 1562 $12 = 0x400efb M[0x40120c] = 781 sw $11, 0($10) $10 = 0x40120c $11 = 781 $ 1 + = x 4 1 2 c

Memory Data

CSc 314 · T W Bennet · Mississippi College

45

Dealing With Data Hazards

• Stall: Wait for the data.
• Forward: Pass new data backwards along the pipeline.
• Delayed Loads: Say that when a register is set, its value

is undefined for some number of steps. Let the assembler or compiler worry about it.

CSc 314 · T W Bennet · Mississippi College

46

MIPS Example Solution

• Values computed by the ALU are forwarded.
• Stall one cycle for values fetched from memory.

The computed values are available in the pipeline. The memory contents are not.

CSc 314 · T W Bennet · Mississippi College

47

Data Forwarding And Stalling 1

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 sw $11, 0($10) $10 = 0x401208 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 addi $10, $10, 4 M[0x40120c] = 75

Memory Data Register File

CSc 314 · T W Bennet · Mississippi College

48

Data Forwarding And Stalling 2

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 sw $11, 0($10) $10 = 0x401208 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 addi $10, $10, 4 lw $11, 0($10) $10 = 0x401208 M[0x40120c] = 75

Memory Data Register File

CSc 314 · T W Bennet · Mississippi College

49

Data Forwarding And Stalling 3

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 sw $11, 0($10) $10 = 0x401208 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 addi $10, $10, 4 $10 = 0x401208 lw $11, 0($10) $10 = 0x40120c add $11, $11, $11 M[0x40120c] = 75 $10 = 0x401208

Data Memory Register File

CSc 314 · T W Bennet · Mississippi College

50

Data Forwarding And Stalling 4

addi $10, $10, 4 $10 = 0x401208 $10 = 0x40120c

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 sw $11, 0($10) $10 = 0x401208 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 add $11, $11, $11 sub $12, $11, $10 M[0x40120c] = 75 $11 = 781

Memory Data Register File

$10 + 0 = 0x40120c lw $11, 0($10) $10 = 0x401208 0x40120c CSc 314 · T W Bennet · Mississippi College

51

Data Forwarding And Stalling 5

addi $10, $10, 4 $10 = 0x401208 $10 + 0 = 0x40120c lw $11, 0($10) $10 = 0x40120c

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $10 = 0x401208 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 $10 = 0x40120c M[0x40120c] = 75 $11 = 75 sub $12, $11, $10 sw $11, 0($10) sw $11, 0($10) $10 = 0x401208 $11 = 781 add $11, $11, $11

Register File

$11 = 781

Data Memory

wait! CSc 314 · T W Bennet · Mississippi College

52

Data Forwarding And Stalling 6

$10 + 0 = 0x40120c lw $11, 0($10) $10 = 0x40120c

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $11 = 781 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 M[0x40120c] = 75 $10 = 0x40120c $11 = 75 add $11, $11, $11

Register File

$11 = 150 sub $12, $11, $10 $10 = 0x40120c sw $11, 0($10) sw $11, 0($10) $11 = 781

Memory Data

$11 = 75 $11 = 781 75 CSc 314 · T W Bennet · Mississippi College

53

Data Forwarding And Stalling 7

$10 + 0 = 0x40120c lw $11, 0($10) $10 = 0x40120c

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 M[0x40120c] = 75 $10 = 0x40120c sw $11, 0($10) $11 = 75 $10 = 0x40120c sub $12, $11, $10 $12 = -0x401176 sw $11, 0($10) $10 = 0x40120c add $11, $11, $11 $11 = 75 $11 = 150 $11 = 75

Register File Memory Data

$11 = 150 $11 = 781 150 CSc 314 · T W Bennet · Mississippi College

54

Data Forwarding And Stalling 8

add $11, $11, $11 $11 = 75

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 M[0x40120c] = 75 $10 = 0x40120c sw $11, 0($10) $11 = 75 $11 = 150 $11 = 150 $10 = 0x40120c

Register File

sub $12, $11, $10 $12 = -0x401176 sw $11, 0($10) $10 = 0x40120c

Memory Data

$11 = 150 $11 = 75 150 $10 + 0 = 0x40120c CSc 314 · T W Bennet · Mississippi College

55

Data Forwarding And Stalling 9

$11 = 150 $10 = 0x40120c sub $12, $11, $10 sw $11, 0($10) $10 = 0x40120c

Register File

$10 + 0 = 0x40120c

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $12 = 4981 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 M[0x40120c] = 75 $10 = 0x40120c sw $11, 0($10) $11 = 150 $12 = -0x401176 $11 = 150 M[0x40120c] = 150

Data Memory

CSc 314 · T W Bennet · Mississippi College

56

Data Forwarding And Stalling 10

$11 = 150 sw $11, 0($10) $10 = 0x40120c

Write-Back Fetch Decode Execute Memory Instruction

Memory

addi $10, $10, 4 lw $11, 0($10) add $11, $11, $11 sub $12, $11, $10 $13 = 2 $14 = 1000 $15 = 783 $16 = 459 $17 = 10 $18 = 4981 $19 = 0 $20 = 7 $21 = 0x44812 M[0x401208] = 34 $10 = 0x40120c sw $11, 0($10) $11 = 150 $12 = -0x401176 M[0x40120c] = 150

Data Memory Register File

CSc 314 · T W Bennet · Mississippi College

57

Control Hazards

Instructions are fetched in address order. After an unconditional jump, the following instruction is fetched before the jump is decoded and the target computed. For a conditional branch, we also don’t know yet if we’re actually going to branch.

CSc 314 · T W Bennet · Mississippi College

58

Dealing With Control Hazards

• Stall: Don’t start a new instruction until everything is

known. May have to discard instructions fetched after a branch.

• Predict: Guess a result and cancel the calculation if it

turns out wrong.

• Delayed Branch: Just say that branch is not effective

immediately. Instruction(s) after branch executed unconditionally. Let the assembler or compiler worry about it.

CSc 314 · T W Bennet · Mississippi College

59

Delayed Branch

• Can be used with the others to reduce cost of stall or

incorrect prediction.

• Usually just one instruction is executed unconditionally

after the branch. This is called the delay slot.

• Compiler optimizer tries to put something useful there.

CSc 314 · T W Bennet · Mississippi College

60

Filling the Delay Slot

CSc 314 · T W Bennet · Mississippi College

61

MIPS Example Solution

• Move target computation to decode station.
• Add a comparator at decode to decide if branches are

taken.

• One cycle is lost if the branch is taken.

Alternative: Define a delay slot. Then no cycle is lost if the delay slot can do useful work.

CSc 314 · T W Bennet · Mississippi College

62

Hardware

CSc 314 · T W Bennet · Mississippi College

63

Branch Prediction

• Guess which way the branch will go and act accordingly.
• If wrong, discard partial results.

CSc 314 · T W Bennet · Mississippi College

64

Static Branch Prediction

• Predict all branches the same way all the time.
• Usually, predict not taken.

CSc 314 · T W Bennet · Mississippi College

65

Dynamic Branch Prediction

• Guess based on past behavior.
• Must record the history of each branch.
• Use a small memory as a hash.
• The hash address is the low part of the branch

instruction’s memory address.

• Ignore collistions. They reduce performance, not

correctness.

CSc 314 · T W Bennet · Mississippi College

66

Dynamic Branch Prediction Types

• One-bit: Predict it will branch the same as last time.

Wrong twice for a simple loop.

• Two-bit: Retain prediction until wrong twice. Wrong once

for a simple loop.

CSc 314 · T W Bennet · Mississippi College

67

Two-Bit Dynamic Branch Prediction

Taken Taken Taken Taken Not taken Not taken Not taken Not taken Predict taken Predict taken Predict not taken Predict not taken

CSc 314 · T W Bennet · Mississippi College

68

Exceptions

• Must jump to O/S.
• Can occur occur at any stage.
• Must re-construct a state after the faulting instruction.

CSc 314 · T W Bennet · Mississippi College

69

The Whole Thing

CSc 314 · T W Bennet · Mississippi College

70

Faster Still: Superscalar

• Multiple Pipelines
• More than one instruction per cycle.
• All kinds of terrible ordering problems.

CSc 314 · T W Bennet · Mississippi College

71

Faster Still: Dynamic Pipeline Scheduling

• Dispatch instructions out of order.

CSc 314 · T W Bennet · Mississippi College

72