Advance Computing for Electrical Engineers An Introduction - - PowerPoint PPT Presentation

Advance Computing for Electrical Engineers

An Introduction Virendra Singh

Associate Professor Computer Architecture and Dependable Systems Lab Department of Electrical Engineering Indian Institute of Technology Bombay

http://www.ee.iitb.ac.in/~viren/ E-mail: viren@ee.iitb.ac.in

EE-717/453:Advance Computing for Electrical Engineers

Lecture0

EE 717/453

• Pre-requisite
• Knowledge of programming (Any language C/C+

+/Java, ..)

• Attendance

– Not mandatory (except first week) – Submission of assignments (firm deadlines)

• Timing

– M Th (Slot 13): 6:30 pm to 8:00 pm – Office Hours: Wed 4 pm to 6 pm

18 July 2013 EE-717/453@IITB 2

Course Outline

• Data Structures and Algorithm

– About 6 - 8 classes – Cover basic data structures (Stack, Queue,

Linked List, Tree, Graph, Hashing

– Basic Algorithm Design

• Computer Architecture and Compiler

Design

– About 2-4 lectures – Basic architectures and compiler design

phases

18 July 2013 EE-717/453@IITB 3

Course Outline

• Operating System

– About 6-8 classes – Basics of operating system – PintOS – Implementation of basic OS functions

(Threads, user program, virtual memory, file system)

Parallel Programming

– About 6 – 8 lectures – Basics of parallel programming – CUDA

18 July 2013 EE-717/453@IITB 4

Course Outline

• Miscellaneous

– About 1-2 lectures

• Summary

18 July 2013 EE-717/453@IITB 5

Books

• Data Structures, Algorithms, and its applications in

C++

– Sartaj Sahni

• Introduction to Algorithms

– Cormen et al.

• Modern Operating Systems

– Tanenbaum

• Introduction to parallel programming

– Thomas Rauber

18 July 2013 EE-717/453@IITB 6

Evaluation

• Mid semester Examination

– 15 Marks

• Final Examination

– 30 Marks

• Assignments

– 40 Marks

• Continuous Evaluation

– 15 Marks

18 July 2013 EE-717/453@IITB 7

Grades

Absolute

• AA: > 94
• AB : 85 - 94
• BB : 75 - 84
• BC : 65 – 74
• CC : 55 – 74
• CD : 45 – 54
• DD : 40 – 45
• FR: < 40

18 July 2013 EE-717/453@IITB 8

Problem Solving: Main Steps

• 1. Problem definition
• 2. Algorithm design/ Algorithm specification
• 3. Algorithm analysis
• 4. Implementation
• 5. Testing
• 6. [Maintenance]

18 July 2013 EE-717/453@IITB 9

Running Program on Processor

Processor Performance = --------------- Time Program

Architecture --> Implementation --> Realization

mpiler Designer Processor Designer Chip Designer

Instruction s Cycles Program Instructio n Tim e Cycl e (code size)

= X X

(CPI) (cycle time)

EE-717/453@IITB

18 July

10

From Source to Executable

Compiler main() sub1() data source program foo.c main sub1 data

• bject

modules foo.o printf scanf gets fopen exit data ... static library libc.a Linkage Editor main sub1 data printf exit data load module a.out

• ther

programs ... main sub1 data printf exit data

• ther

... ... kernel

Machine memory ?

(system calls) Loader (Run Time) Dynamic library case not shown “Load time”

18 July 2013 EE-717/453@IITB 11

Problem Definition

• What is the task to be accomplished?

– Calculate the average of the grades for a

given student

– Understand the talks given out by politicians

and translate them in Chinese

What are the time / space / speed performance requirements ?

18 July 2013 EE-717/453@IITB 12

Algorithm Design/Specifications

• Algorithm: Finite set of instructions that, if followed,

accomplishes a particular task.

• Describe: in natural language / pseudo-code / diagrams /

etc

• Criteria to follow:

– Input: Zero or more quantities (externally produced) – Output: One or more quantities – Definiteness: Clarity, precision of each instruction – Finiteness: The algorithm has to stop after a finite (may

be very large) number of steps

– Effectiveness: Each instruction has to be basic enough

and feasible

• Understand speech
• Translate to Chinese

18 July 2013 EE-717/453@IITB 13

Algorithm Design/Specifications

• Algorithm: Finite set of instructions that, if followed,

accomplishes a particular task.

• Describe: in natural language / pseudo-code / diagrams /

etc

• Criteria to follow:

– Input: Zero or more quantities (externally produced) – Output: One or more quantities – Definiteness: Clarity, precision of each instruction – Finiteness: The algorithm has to stop after a finite (may

be very large) number of steps

– Effectiveness: Each instruction has to be basic enough

and feasible

• Understand speech
• Translate to Chinese

18 July 2013 EE-717/453@IITB 14

Implementation, Testing, and Maintenance

• Implementation

– Decide on the programming language to use

• C,C++,Lisp,Java,Perl,Prolog,Assembly etc.

– Write clean, well documented code

• Test, test, test
• Integrate feedback from users, fix bugs,

ensure compatibility across different versions  Maintenance

18 July 2013 EE-717/453@IITB 15

Algorithm Analysis

• Space complexity

– How much space is required

Time complexity

– How much time does it take to run the

algorithm

Often, we deal with estimates!

18 July 2013 EE-717/453@IITB 16

Space Complexity (1/3)

• Space complexity = The amount of

memory required by an algorithm to run to completion

– [Core dumps = the most often encountered cause is

“memory leaks” – the amount of memory required larger than the memory available on a given system]

Some algorithms may be more efficient if data completely loaded into memory

– Need to look also at system limitations – E.g. Classify 10 GB of text in various categories

[politics, tourism, sport, natural disasters, etc.] – can I afford to load the entire collection?

18 July 2013 EE-717/453@IITB 17

Space complexity (2/3 )

• Fixed part: The size required to store

certain data/variables, that is independent

• f the size of the problem:

– e.g. name of the data collection – same size for classifying 2GB or 1MB of texts

• Variable part: Space needed by

variables, whose size is dependent on the size of the problem:

– e.g. actual text – load 2GB of text VS. load 1MB of text

18 July 2013 EE-717/453@IITB 18

Space Complexity (3/3)

• S(P) = c + S(instance characteristics)
• Example:

void float sum (float* a, int n){ float s = 0; for (int i = 0; I < n; i++){ s += a[i] } return s; }

• Space? One word for n, one for a [passed by

reference], one for i  Constant space

18 July 2013 EE-717/453@IITB 19

Time Complexity

• Often more important than space

complexity

– space available (for computer programs!) tends to be

larger and larger

– time is still a problem for all of us

3-4 GHz multi-core processors are in the market

– still researchers estimate that the computation of

various transformations for 1 single DNA chain for one single protein on 10 GHz computer would take about 1 year to run to completion

Algorithms running time is an important issue

18 July 2013 EE-717/453@IITB 20

Running Time

• Problem: Prefix average

– Given an array X – Compute the array A such that A[i] is the average of

elements X[0] ... X[i], for i=0..n-1

Solution 1:

– At each step i, compute the element X[i] by

traversing the array A and determining the sum of its elements, respectively the average

Solution 2:

– At each step i update a sum of the elements in the

array A

– Compute the element X[i] as sum/I

Big question: Which solution to choose?

18 July 2013 EE-717/453@IITB 21

Running Time

• Suppose the program includes an if-then

statement that may execute or not  variable running time

• Typically algorithms are measured by their worst

case

18 July 2013 EE-717/453@IITB 22

Worst case Best case Average case

Experimental Approach

• Write a program that implements the

algorithm

• Run the program with data sets of varying

size.

• Determine the actual running time using a

system call to measure time (e.g. system (date) );

• Problems?

18 July 2013 EE-717/453@IITB 23

Experimental Approach

• It is necessary to implement and test the

algorithm in order to determine its running time

• Experiments can be done only on a limited

set of inputs, and may not be indicative of the running time of the other inputs

• The same hardware and software should

be used in order to compare two algorithms – condition very hard to achieve!

18 July 2013 EE-717/453@IITB 24

Theoretical Approach

• Based on high-level description of the

algorithms, rather than language dependent implementation

• Makes possible an evaluation of the

algorithms that is independent of the hardware and software environment  Generality

18 July 2013 EE-717/453@IITB 25

Algorithm Description

• How to describe algorithms independent of a programming

language

• Pseudo-Code = a description of an algorithm that

–

more structured than usual prose but

–

less formal than a programming language

• (Or diagram)
• Example: find the maximum element of an array

Algorithm ArrayMax (A, n): Input: An array A storing n intergers Output: The maximum element in A currentMax  A[0] for I  1 to n-1 do if currentMax < A[i] then currentMax  A[i] return currentMax

18 July 2013 EE-717/453@IITB 26

Pseudo Code

• Expression: use standard mathematical symbols

–

use  for assignment

–

use = for the equality relationship

• Method declarations: Algorithm name (parameters)
• Programming constructs:

–

Decision structures: if … then … [else ..]

–

While-loop: while .. Do

–

Repeat-loops: repeat … until …

–

For-loop: for … do

–

Array indexing: A[i]

• Methods

–

Calls: object methods (arguments) ; - return: return value

• Use comments
• Instructions have to be basic enough and feasible

18 July 2013 EE-717/453@IITB 27

Low Level Algorithm Analysis

• Based on primitive operations (low-level

computations independent from the programming language)

• For example:

– Make an addition = 1 operation – Calling a method or returning from a method = 1

• peration

– Index in an array = 1 operation – Comparison = 1 operation etc.

• Method: Inspect the pseudo-code and count the

number of primitive operations executed by the algorithm

18 July 2013 EE-717/453@IITB 28

Example

• Algorithm ArrayMax (A, n):

Input: An array A storing n intergers Output: The maximum element in A currentMax  A[0] for I  1 to n-1 do if currentMax < A[i] then currentMax  A[i] return currentMax

• How many operations ?

18 July 2013 EE-717/453@IITB 29

Asymptotic Notation

• Need to abstract further
• Give an idea of how the algorithm

performs

• n steps vs. n+5 steps
• n steps vs. n2 steps

18 July 2013 EE-717/453@IITB 30