Inside the Machine: An Introduction to Architecture, Parallelism - - PowerPoint PPT Presentation

Inside the Machine: An Introduction to Architecture, Parallelism & its Applications

Soumyabrata Dev

The ADAPT SFI Research Centre

https://soumyabrata.github.io/

UCD - Beijing-Dublin International College (BDIC) 7-May-2019

The ADAPT Centre is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.

www.adaptcentre.ie

Course Overview

Course website is available here: https://www.scss.tcd.ie/~devs/bdic.html

www.adaptcentre.ie

(1/3) Computer Architecture

www.adaptcentre.ie

Computer Components

At the most basic level, a computer is a device consisting of three pieces:

• A processor to interpret and execute programs
• A memory to store both data and programs
• A mechanism for transferring data to and from

the outside world.

www.adaptcentre.ie

How computer understands our code?

www.adaptcentre.ie

(2/3) Pipelining

www.adaptcentre.ie

Laundry Example

• Ann, Brian, Cathy, Dave each have
• ne load of clothes to wash, dry, and

fold.

• Washer takes 30 minutes.
• Dryer takes 40 minutes.
• Folder takes 20 minutes.

www.adaptcentre.ie

Sequential Processing

• Sequential laundry takes 6 hours for 4 loads.
• If they learned pipelining, how long would

laundry take?

www.adaptcentre.ie

Pipelined Processing

• Pipelined laundry takes 3.5 hours for 4 loads.

www.adaptcentre.ie

Thoughts on pipelining

• Pipelining doesn't help latency of single task, it

helps throughput of entire workload.

• Pipeline rate limited by slowest pipeline stage.
• Multiple tasks operating simultaneously using

different resources.

• Potential speedup = Number pipe stages.

www.adaptcentre.ie

Use of Pipelining in computer

• Instruction-level parallelism (ILP) is a measure of

how many operations in a computer program can be performed. “in parallel” at the same time.

www.adaptcentre.ie

Activity Time

Consider the following program: e = a + b f = c + d m = e*f

• Suppose, each instruction takes 1 unit of time to

complete.

• Sequential processing = 3 units of time
• How much time needed, if I employ ILP here?

www.adaptcentre.ie

Types of parallelism

Three major categories to exploit parallelism:

• Instruction-level parallelism

Multiple instructions from one instruction stream are executed simultaneously.

• Thread-level parallelism

Multiple instruction streams are executed simultaneously.

• Vector data parallelism

The same operation is performed simultaneously

• n arrays of elements.

www.adaptcentre.ie

(3/3) High Performance Computing

www.adaptcentre.ie

HPC: Powering Deep Learning

1“Are AI/Machine Learning/Deep Learning in Your Company’s Future?”, insideBigData + NVIDIA 2EMC Digital Universe with Research & Analysis by IDC

www.adaptcentre.ie

HPC Application

HPC technology is implemented in multidisciplinary areas including:

• Biosciences
• Geoscience and remote sensing data
• Oil and gas industry modeling
• Electronic design automation
• Climate modeling
• Media and entertainment

www.adaptcentre.ie

HPC Application

HPC technology is implemented in multidisciplinary areas including:

• Biosciences
• Geoscience and remote sensing
• Oil and gas industry modeling
• Electronic design automation
• Climate modeling
• Media and entertainment

Thank You

The ADAPT Centre is funded under the SFI Research Centres Programme (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.