A Look at Computer Architecture Methodologies Mario Badr and - - PowerPoint PPT Presentation

▶
a look at computer
SMART_READER_LITE
LIVE PREVIEW

A Look at Computer Architecture Methodologies Mario Badr and - - PowerPoint PPT Presentation

Jan 24, 2023 471 likes •661 views

A Look at Computer Architecture Methodologies Mario Badr and Natalie Enright Jerger Why evaluation methodologies? 1. Is computer architecture an art or a science? Experimental Data Reproducibility 2. How have evaluated metrics changed

slide-1
SLIDE 1

A Look at Computer Architecture Methodologies

Mario Badr and Natalie Enright Jerger

slide-2
SLIDE 2

Why evaluation methodologies?

  • 1. Is computer architecture an art or a science?
  • Experimental Data
  • Reproducibility
  • 2. How have evaluated metrics changed over the years?

2

slide-3
SLIDE 3

Scope of the Survey

  • 44 ISCA Proceedings
  • 1973-2017
  • Too many papers (over 1600)
  • Select papers from each proceeding across topics
  • Bias selection to impactful papers
  • 4-7 papers per proceeding
  • 222 papers total

3

slide-4
SLIDE 4

Paper Topics

Axis #1 Description Single Core A conventional general purpose processor with one core Multiple Core More than one conventional processor Specialized Architecture An unconventional processor (e.g., accelerator, GPU) Axis #2 Description or Examples Microarchitecture e.g., branch prediction, simultaneous multithreading Memory e.g., cache replacement, phase change memory, cache coherence, memory consistency Networks e.g., bus, crossbar, network-on-chip, network interface Organization The overall design of multiple components Coordination The management of multiple components to achieve a goal

4

slide-5
SLIDE 5

Surveyed Papers Along Both Axes

20 40 60 80 100 120 140 Multiple Cores Single Core Specialized Architecture Paper Count Memory Microarchitecture Networks Organization Coordination

5

slide-6
SLIDE 6

Types of Evaluations

  • None
  • Qualitative
  • Theoretical
  • Quantitative
  • Experimental data

6

slide-7
SLIDE 7

We Focus on Quantitative Evaluations

  • None
  • Qualitative
  • Theoretical
  • Quantitative
  • Experimental data
  • Analytical Model
  • Prototype
  • Simulation
  • Architectural
  • Circuit-level
  • Other

7

slide-8
SLIDE 8

The 1970s – 27 papers

  • Quantitative Evaluations: 40%
  • Evaluated Metrics
  • Performance
  • Proxies for area
  • Analytical Models
  • e.g., assume ideal parallelism
  • e.g., performance projections

1 2 3 4 5 6 7 8 Analytical Model Architectural Simulation Other Paper Count Memory Microarchitecture Networks Organization

8

slide-9
SLIDE 9

The 1980s – 46 papers

  • Quantitative Evaluations: 60%
  • Reduced costs of memory and

CPU

  • Single core processors
  • Prototyping
  • Trace-driven simulation

9

2 4 6 8 10 12 14 16 18 Analytical Model Architectural Simulation Prototyping Paper Count Memory Microarchitecture Networks Organization Coordination

slide-10
SLIDE 10

The 1990s – 47 papers

  • Quantitative Evaluations: 85%
  • Introduction of many simulators
  • SimpleScalar
  • Introduction of CACTI
  • Catches on in the next decade
  • Power/energy is considered

5 10 15 20 25 30 35 Analytical Model Architectural Simulation Prototyping Paper Count Memory Microarchitecture Networks Organization

10

slide-11
SLIDE 11

A Brief Interlude: Evaluated Metrics

1973 – 1995 1996 – 2017

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percentage of Papers Performance Power Energy Area 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Performance Power Energy Area

11

slide-12
SLIDE 12

The 2000s – 50 papers

  • Quantitative Evaluations: 98%
  • Models for power, energy, thermal
  • Wattch, HotSpot, Orion, McPAT
  • CACTI gains popularity
  • More simulator options
  • Pin, Simics
  • Tools to reduce simulation time
  • SimPoint, PinPoint, SMARTS

12

5 10 15 20 25 30 35 40 45 50 Analytical Model Architectural Simulation Prototyping Paper Count Memory Microarchitecture Networks Organization Coordination

slide-13
SLIDE 13

The 2010s – 52 papers

  • Models and prototypes used

more

  • More tools
  • Raised levels of abstraction
  • Design space exploration

13

5 10 15 20 25 30 35 40 45 50 Analytical Model Architectural Simulation Prototyping Paper Count Memory Microarchitecture Networks Organization Coordination

slide-14
SLIDE 14

Summarizing Tool Use – 1973 - 2017

14

5 10 15 20 25 30 35 40 45 50 1970s 1980s 1990s 2000s 2010s 1970s 1980s 1990s 2000s 2010s 1980s 1990s 2000s 2010s Analytical Model Architectural Simulation Prototyping Paper Count

slide-15
SLIDE 15

Computer Architecture: Art or Science?

  • Strong push to quantitative evaluations
  • Designs are evaluated with more metrics
  • Many tools developed to generate data
  • Reproducibility?

15

slide-16
SLIDE 16

The Increasingly Complex “Methodology”

  • Methodology section prominent in mid-to-late 90s
  • Methodologies grow very complex
  • More tools are used
  • Page real estate
  • Less used for methodology
  • More used for experimental data
  • Methodologies do not provide enough information

16

slide-17
SLIDE 17

Conclusion: Towards a Scientific Method

Architects

  • Better methodology section
  • Relevant experimental data
  • Release your evaluation
  • Docker
  • GitHub
  • Other technologies

Tools Developers

  • Caution against limitations
  • Output ‘artifacts’ that
  • Can be redistributed
  • Can be re-used as inputs
  • Can be analyzed

17

slide-18
SLIDE 18

Our Data is Open Source

https://github.com/mariobadr/survey-wp3 License: Apache 2.0 Mario Badr and Natalie Enright Jerger

18