FAST: A Functionally Accurate Simulation Toolset for the Cyclops-64 - - PowerPoint PPT Presentation

FAST: A Functionally Accurate Simulation Toolset for the Cyclops-64 Cellular Architecture

Juan del Cuvillo, Weirong Zhu, Ziang Hu, Guang Gao Department of Electrical and Computer Engineering University of Delaware June 4, 2005 First Workshop on Modeling, Benchmarking and Simulation (MoBS) at the 32nd International Symposium on Computer Architecture

Outline

• Cyclops-64 architecture:
• C64 supercomputer.
• C64 node.
• C64 architecture details.
• FAST design and implementation:
• Pipeline.
• Segmented memory.
• Memory and interconnect contention.
• Experience:
• Architecture design verification.
• Early system software development.
• Application development.

Cyclops-64 Supercomputer

Cyclops-64 Node Logical View

Cyclops-64 Node Main Features

• Clock frequency 500MHz.
• 75 processors:
• 2 thread units (in-order issue, out-of-order

completion).

• 2 32KB SRAM banks.
• FPU and integer multiply-accumulate unit.
• 32KB I-cache shared by 5 processors.
• A-switch
• 4 ports, 4GB/s per port.
• 1GB off-chip SDDRAM.
• Crossbar network:
• 96 ports, 4GB/s per port.
• Connects processors(80), I-cache(4), DRAM

(4), A-switch(6), host-interface(2).

Cyclops-64 Architecture Relevant Features

• Integrates proc., memory and comm.
• 150 thread units, 4.7MB on-chip SRAM,

crossbar switch and A-switch device.

• No resource virtualization
• Non-preemptive execution.
• OS will not interrupt user program.
• No HW virtual memory manager.
• Memory hierarchy visible by programmer.
• ISA provides:
• Support for thread level execution.
• In-memory atomic operations.

Simulation Requirements

• Architecture team:
• Multicore technology.
• Design verification.
• Space exploration.
• System software group:
• Toolchain development & testing.
• Runtime system design.
• Users:
• Application development.
• Performance estimation.

How does FAST meet the re- quirements?

• Multichip, multithreaded C64 system.
• Functionally accurate (not cycle accu-

rate).

• Timing sensitive.
• Binary compatible.
• Instrumentability.
• Execution driven.

How does FAST meet the re- quirements?

• Instruction execution.
• Exception handling.
• Segmented memory space.
• Execution trace and statistics.
• Memory and interconnect contention.
• A-switch device.
• Debugger.

Instruction Pipeline Fetch and Decode

Instn fetch: account for delay if PIB and I-cache miss, not for branch prediction

Instruction Pipeline Read Registers

1 cycle for all 1 and 2-operand instn, 2 cycles for 3-operand instn (FMAx). In-order-issue.

Instruction Pipeline Execution

RISC-like instn execution model based on eXecution/Delay pairs Independent instructions executed in parallel

Instruction Pipeline Instruction Timing

Instruction type x d Branch 2 Count pop. 1 1

• Int. multiply

1 5

• Int. divide, reminder

1 33

• Float. add, multiply, convert

1 5

• Float. multiply-add

1 10

• Float. divide

1 33 Float square root 1 56 Memory op (local SPM) 1 2 Memory op (global SRAM) 1 20 Memory op (off-chip DRAM) 1 36 Others 1

Instruction Pipeline Commit

Put results away Out-of-order completion Don't account for write in reg. file conflicts.

Commit (cont)

• Out-of-order execution: x/d model.
• But threads run synchronously; single

clock signal.

• Special instn. imply inter-thread syn-

chronization.

• FAST synchronizes thread execution at

commit stage.

• If thread's ahead wait. Slowest thread

updates clock.

T1 T2 1 commit 1 wait 2 commit 3 commit 3 wait

Segmented Memory Space

AULI AULS AULD 0x00000000 0x40000000 0x80000000 Global/Interleaved Memory Local/Scratc hpad Memories Off- chip/DRAM Memory

SPM149 SPM1 SPM0

Non-uniform share address space

Memory and Interconnect Contention

Contention in the Crossbar

STORE X LOAD X DATUM X

Memory Bandwidth Limita- tion

STORE X

FULL

Interconnect Contention

LOAD X

FULL

A-Switch Device

• Under testing.
• Reads from / writes to memory.
• Not connected to the

crossbar/memory module.

• Overhead not accounted for.
• Estimation for multi-chip programs

less accurate.

Debugger

• Embedded assembly-level debugger.
• GDB-based source level debugger:
• Handles single-threaded programs.
• Work in progress for multi-chip mul-

ti-threaded programs.

Toolchain Verification

C64 toolchain: supports C64 ISA, segmented memory space, etc.

System Software Toolchain Verification

Methodology:

• Source: C/Fortran/assembly
• Generate executable.
• Run binary in FAST simulator.
• If expected result then toolchain OK.

2 Phases:

• Thorough ISA testing: manually inspect

trace file.

• Toolchain coverage: large # test cases.

From compiler regression testsuite to sizable applications.

Architecture Design Verifi- cation

• FAST generates execution trace.
• Trace from VHDL simulator.
• Compare program execution, instruc-

tion by instruction.

• Help validate VHDL simulator, i.e.,

chip design.

• Continue testing once hardware plat-

form (emulator) is available.

• C32 DIMES – CYSIM (SC03, SC04)
• C64 Mrs. Cyclops – FAST (SC05)

System Software Develop- ment

• Early evaluation of multithreaded run-

time system library:

• Test functionality of the TNT* library.
• Estimation of thread creation, termi-

nation, reuse times.

• Study of spin lock algorithms.
• CNET communication library devel-
• pment.

(*) In Workshop of Massive Parallel Processing (IPDPS05)

Application Development

• FAST verification:

Do the results match what the archi- tecture is capable of?

• Microbenchmarks:
• TableToy
• Matrix-matrix-multiply

Experience

• TableToy: Measure Giga Updates Per

Second (GUPS) 1 tmp1 = stable[j] (load) 2 tmp2 = table[i] (load) 3 val = tmp1 xor tmp2 (xor) 4 table[i] = val; (store)

GUPS with TableToy

Little throughput Pseudo random # generation Bank conflicts

GUPS with NewToy (S- RAM)

75% maximum throughput

MUPS with NewToy (DRAM)

DRAM maximum bandwidth 250 MUPS

MFLOPS Matrix-matrix Multiply

Baseline: 17MLOPS Optimized: 215MFLOPS/thread

• approx. 500MFLOPS/proc

~ 50% maximum

ACKNOLEDGMENTS

• Monty Denneau (IBM)
• Henry Warren (IBM)
• José Castaños (IBM)
• Christos Georgiou (IBM)
• ET International, Inc.
• Our sponsors
• Alban Douillet
• Brice Dobry
• Ge Gan
• Geoff Gerfin
• John Tully
• Weirong Zhu
• Wesley Toland
• Ziang Hu
• Fei Chen
• Hirofumi Sakane
• Yuhei
• Vishal Karna