A Memory System Design Framework: Creating Smart Memories Amin - - PowerPoint PPT Presentation

http://www.c2s2.org

Amin Firoozshahian, Alex Solomatnikov

Hicamp Systems Inc.

Ofer Shacham, Zain Asgar, Stephen Richardson, Christos Kozyrakis, Mark Horowitz

Stanford University

A Memory System Design Framework: Creating Smart Memories

An Era of Chip-Multiprocessors…

• Single-thread performance scaling has stopped
• More processor cores on the same die
• Claim:

Scale performance Keep design complexity constant

IBM Cell Intel Nehalem Sun Rock 2 Amin Firoozshahian

Looking a Little More Closely

Sun Rock

Reality…

• Replicated cores
• Incredibly complicated memory system

Large amounts of logic

• Innovation is in the memory system

Transactions, streaming, fast synchronization, security, etc.

• Never exactly the same
• Where all the bugs are!

4 Amin Firoozshahian

ISA for Memory Systems

• Can we regularize the memory system hardware?
• “Program” it rather than “Design” it?
• Benefits:

Reduce design time Patch errors Run-time tuning

• How can we do this?

Amin Firoozshahian 5

Amin Firoozshahian

Shared Memory System

• Resources:

Local memory

Data, state bits

Interconnect Controllers

• Operations:

Probing state bits Track requests Communication Data movements (spill / refill)

$ Cache Controller Proc Proc $ Cache Controller Interconnect Memory miss

Msg

6

Amin Firoozshahian

Streaming Memory System

• Resources:

Local memory Interconnect Controllers

• Operations:

Communication Data movements Track outstanding transfers

Local Mem Interconnect

…

Memory Proc Local Mem DMA DMA Proc 7

Transactional Memory System

• Resources

Local memory

More state bits

Interconnect Controllers

• Operations

Data movements State checks / updates Communication

$ Commit Controller $ Commit Controller Interconnect Memory Addr. FIFO Addr. FIFO Proc Proc 8 Amin Firoozshahian

Commonalities

• Same resources and operations
• Different in:

How the operations are sequenced Interpretation of state bits

• We need:

Flexible local storage and interconnect Programmable controllers

Amin Firoozshahian 9

Local Memories

• Programmable memory mat

Data array State bits PLA logic Comparator

• Accessed by

Address, Opcode

• Returns

data, state, compare result

[K. Mai et.al., “Architecture and Circuit Techniques for a Reconfigurable Memory Block,” IEEE International Solid-State Circuits Conference, February 2004 10 Data State Cmp Update Address Opcode

Programmable Controllers

• Use an off-the-shelf processor?

FLASH, Typhoon, etc.

• Too slow

All the way to the L1 cache interface

• Our approach:

Micro-coded engines (functional units) Each class of operations in a separate engine

Amin Firoozshahian 11

Programming

• A set of subroutines

A set of basic operations Executed in a functional unit

• Each one calls next

Link subroutines to each other

Amin Firoozshahian 12

Msg

Unit 1 Unit 3

Msg

Unit 2

Microarchitecture

• A small pipeline
• Configuration (“program”) memories

Horizontal micro-code Decide what to do Decide how to proceed

13 Amin Firoozshahian

Organization

Tracking State Update Data Movement

USHR MSHR Line Buffers

Processor Interface Network Interface

Interrupt

DMA DMA DMA

To/From Processors To/From Network To/From local storages

14 Amin Firoozshahian

Read Miss Example

Amin Firoozshahian 15 Tracking State Update Data Movement

USHR MSHR Line Buffers

Processor Interface Network Interface

Interrupt

DMA DMA DMA Miss

Read Miss Evict

Access Tags

Line Read

Access Data

WB / Miss

Spill Read Miss

Read Miss

Read Miss

Programming Complexity

• Cache Coherence

Message types received by controller: 6

From processor: Cache miss, Upgrade miss, Prefetch From network: Coherence request, Refill, Upgrade

Subroutine types in Tracking unit: 11

• Streaming

Message types: 5

Direct access, Gather, Scatter, Gather reply, Scatter ack.

Subroutine types in Tracking unit: 9

16 Amin Firoozshahian

Smart Memories

8-core CMP system ST 90nm-GP CMOS technology 5.5 ns cycle time (181MHz) 2.9M gates, 55M transistors

7.77mm 7.77mm 17

Status

• System bring-up……………...…..
• System configuration……….…...
• JTAG tests…………………….…...
• Coherent shared memory tests…
• Transactional tests (TCC)……….
• Streaming tests……………………
• More testing in progress
• Planning for a 32-processor system

Test Chip 18 Amin Firoozshahian

Evaluation

• Comparison with a hardwired controller

But which one? You would claim I am cheating!

• Compare with an “ideal” controller

Assume controller actions occur in zero time Account for external actions

Data read/write Message send/receive

• Gives an upper bound

Amin Firoozshahian 19

Average Read Latency

Amin Firoozshahian 20 Coherent Shared Memory Streaming Transactions

1 2 3 4 5 6 7 8 9 FFT MPEG2 Enc Barnes FMM 179.art Bitonic Sort MPEG2 Enc Barnes MP3D Cycles

Average Read Latency - 32 processor system

Real Controllers Ideal controllers

Execution Time

• Total average overhead: 15%

Amin Firoozshahian 21 Coherent Shared Memory Streaming Transactions

10.64 14.51 24.29 6.93 7.58 1.88 14.14 8.33 20.03 5 10 15 20 25 30

Overhead (%)

Average Overhead (%)

Conclusion

• Strong similarity between memory systems

Common resources and operations

• A framework for memory systems design

Generate specific “instances”

• Modest performance overhead

Compared to ideal systems

Amin Firoozshahian 22