Why Algorithms? K.Viswanathan Iyer, Ph.D. Professor Dept. of - - PDF document

K.V.Iyer Algorithms/B.Tech.

Why Algorithms?

K.Viswanathan Iyer, Ph.D. Professor

• Dept. of Computer Science and Engineering

National Institute of Technology Tiruchirapalli - 620 015.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

The origin of algorithms! Algos means “pain” in Greek. Algor in Latin means “to be cold”. – neither is the root for the word “algorithm”!. Algorithm stems from al-Khwarizmi, the name

• f the 19th-century Arab scholar who stressed

the importance of methodical procedures for solving problems.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

The pre-requisites The following pre-requisites are essential for an introductory course on algorithms:

• Some programming experience:

notion

• f types, use of arrays, linked lists, pro-

cedures/functions.

• Basics in (discrete) maths: combinatorial

ideas, probability, algebraic structures, el- ementary calculus.

• Problem-solving skills: the ability to take

up and solve problems related to the above. Quiz: How do you estimate Hn quickly?

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

How to go about?

• get the pre-requisites clear
• for each concept/idea, work out related

exercises/problems on your own

• understand the crisp technical/mathematical

idea(s) that makes an algorithm work (Quiz: give an example that you know)

• for each lecture, consult the books/reference

materials Remember: A typical course cannot be en- cyclopedic!

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Algorithms – Why? What? What is an Algorithm? – A step-by-step pro- cedure for solving well-posed problems. Ex. Euclid’s algorithm to compute the gcd

• f two positive integers.

Algorithms constitute a core technology in Computer Science and Engineering. Finds use in: DBMS, Compilers, GUIs, H/W design, web-technology etc. Techniques in Algorithms – Important in the practical realization of other technologies.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Algorithms – Why? What? An algorithm has the following properties (Don Knuth):

• 1. Finiteness: Termination after a finite num-

ber of steps.

• 2. Definiteness: Rigorous and unambiguous

specification of each step.

• 3. Input: Zero or more inputs, given at the

start or during intermediate steps.

• 4. Output: At least one output, which is a

consequence of the inputs.

• 5. Effectiveness:

Apart from definetiness, the steps are to be basic enough so as to be carried out on a paper in finite time.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Examples you know

• K and R vs. K (chess end-game)
• String reversal (think of LISP-style algo-

rithm)

• Palindrome checking (easy!)
• Bisection method to locate zeros of a

univariate polynomial (what is an error bound after n steps?)

• Gaussian elimination (how easy/difficult

it is?)

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Problem areas Problems come from diverse areas of science and engineering occuring in real life:

• organizing and storing large amounts of

structured data.

• collecting and organizing data from the

web.

• signals to be processed/analyzed.
• images to be processed/analyzed.
• shop-floor activities to be optimized.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Problem categories The following are the basic problem cate- gories for the study of algorithms in a typical course:

• Numerical computing – roots of f(x) =

0, solution of Ax = b, interpolation, dif- ferentiation, integration.

• Data structures – binary tree traversals,

manipulations of linked lists, BST opera- tions, AVL-tree rotations, B-Tree manip- ulations.

• Combinatorial Optimization – TSP, 0/1-

Knapsack, Bin-packing.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Problem categories (cont’d....)

• Graph theory : Easy ones – shortest path,

BFS, DFS, MST.

• Graph Theory : Hard ones – TSP, clique,

vertex cover, bandwidth.

• Computational Geometry – convex hull,

closest pair of points in a plane.

• String and set problems – subset sum,

pattern matching.

• Cryptography – primality testing, factor-

ing.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Efficency consideration In many practical problems the input size is large: then inefficiencies in programs become apparent. Programs are made to run faster by:

• designing clever data structures and com-

plicated algorithms. Quiz: find examples!

• efficient coding.

Quiz: find examples! In theory, it is interesting to understand the performance of an algorithm by analyzing its resource requirements. Typical resources: time taken and memory used.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Analyses of algorithms Analyses of algorithms are done with an over- all view to make ”improvements”. Typical analyses are:

• Correctness of non-obvious algorithms.
• Time taken in the worst-case, average-

case, best-case.

• Amount of memory space needed in the

worst-case.

• Mathematical simplifications or clarity oth-

erwise.

• Optimality.

Complexity theory is a separate area in itself!

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Design paradigms Many algorithms can also be fitted into cer- tain paradigms. The following are popular:

• divide-conquer-combine
• greedy
• dynamic programming
• backtracking
• branch-and-bound
• plane sweep
• randomization (in a broad sense)

Quiz: For each paradigm, give one problem- algorithm.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Some old all-time greats

• 1947: George Dantzig - Simplex method

for LP.

• 1951: Alston Householder - Decomposi-

tional approach to matrix computation.

• 1957:

John Backus - FORTRAN opti- mizing compiler.

• 1962: C.A.R.Hoare - Quicksort.
• 1965:

James Cooley and John Tukey - Fast Fourier Transform.

• 1969: Volker Strassen - Fast matrix mul-

tiplication.

CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech.

Some references 1. A.V.Aho, J.E.Hopcroft and J.D.Ullman, The Design and Analysis of Computer Algo- rithms, Addison-Wesley, 1974.

• 2. S.Baase and A.v. Gelder, Computer Algo-

rithms - Introduction to Design and Analysis, 3/e, Pearson-Education, 2000.

• 3. G.Brassard and P.Bratley, Fundamentals
• f Algorithmics, Prentice-Hall, 1996.

4. T.H.Cormen, C.E.Lieserson, R.L.Rivest and C.Stein, Introduction to Algorithms, 2/e, MIT, 2001. 5. E.Horowitz and S.Sahni, Fundamentals

• f Computer Algorithms, Comp. Sci. Press.

1998.

• 6. H.Wilf, Algorithms and Complexity, Prentice-

Hall, 1986.

CSED/NIT, Trichy May 2008