Why Algorithms? K.Viswanathan Iyer, Ph.D. Professor Dept. of - PDF document

K.V.Iyer Algorithms/B.Tech. Why Algorithms? K.Viswanathan Iyer, Ph.D. Professor Dept. of Computer Science and Engineering National Institute of Technology Tiruchirapalli - 620 015. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. The origin of algorithms! Algos means “pain” in Greek. Algor in Latin means “to be cold”. – neither is the root for the word “algorithm”!. Algorithm stems from al-Khwarizmi , the name of the 19 th -century Arab scholar who stressed the importance of methodical procedures for solving problems. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. The pre-requisites The following pre-requisites are essential for an introductory course on algorithms: • Some programming experience: notion of types, use of arrays, linked lists, pro- cedures/functions. • Basics in (discrete) maths: combinatorial ideas, probability, algebraic structures, el- ementary calculus. • Problem-solving skills: the ability to take up and solve problems related to the above. Quiz: How do you estimate H n quickly? CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. How to go about? • get the pre-requisites clear • for each concept/idea, work out related exercises/problems on your own • understand the crisp technical/mathematical idea(s) that makes an algorithm work (Quiz: give an example that you know) • for each lecture, consult the books/reference materials Remember: A typical course cannot be en- cyclopedic! CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Algorithms – Why? What? What is an Algorithm? – A step-by-step pro- cedure for solving well-posed problems. Ex. Euclid’s algorithm to compute the gcd of two positive integers. Algorithms constitute a core technology in Computer Science and Engineering. Finds use in: DBMS, Compilers, GUIs, H/W design, web-technology etc. Techniques in Algorithms – Important in the practical realization of other technologies. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Algorithms – Why? What? An algorithm has the following properties (Don Knuth): 1. Finiteness: Termination after a finite num- ber of steps. 2. Definiteness: Rigorous and unambiguous specification of each step. 3. Input: Zero or more inputs, given at the start or during intermediate steps. 4. Output: At least one output, which is a consequence of the inputs. 5. Effectiveness: Apart from definetiness, the steps are to be basic enough so as to be carried out on a paper in finite time. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Examples you know • K and R vs. K (chess end-game) • String reversal (think of LISP -style algo- rithm) • Palindrome checking (easy!) • Bisection method to locate zeros of a univariate polynomial (what is an error bound after n steps?) • Gaussian elimination (how easy/difficult it is?) CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Problem areas Problems come from diverse areas of science and engineering occuring in real life: • organizing and storing large amounts of structured data. • collecting and organizing data from the web. • signals to be processed/analyzed. • images to be processed/analyzed. • shop-floor activities to be optimized. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Problem categories The following are the basic problem cate- gories for the study of algorithms in a typical course: • Numerical computing – roots of f ( x ) = 0, solution of Ax = b , interpolation, dif- ferentiation, integration. • Data structures – binary tree traversals, manipulations of linked lists, BST opera- tions, AVL-tree rotations, B -Tree manip- ulations. • Combinatorial Optimization – TSP, 0 / 1- Knapsack, Bin-packing. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Problem categories (cont’d....) • Graph theory : Easy ones – shortest path, BFS, DFS, MST. • Graph Theory : Hard ones – TSP, clique, vertex cover, bandwidth. • Computational Geometry – convex hull, closest pair of points in a plane. • String and set problems – subset sum, pattern matching. • Cryptography – primality testing, factor- ing. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Efficency consideration In many practical problems the input size is large: then inefficiencies in programs become apparent. Programs are made to run faster by: • designing clever data structures and com- plicated algorithms. Quiz: find examples! • efficient coding. Quiz: find examples! In theory, it is interesting to understand the performance of an algorithm by analyzing its resource requirements. Typical resources: time taken and memory used. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Analyses of algorithms Analyses of algorithms are done with an over- all view to make ”improvements”. Typical analyses are: • Correctness of non-obvious algorithms. • Time taken in the worst-case, average- case, best-case. • Amount of memory space needed in the worst-case. • Mathematical simplifications or clarity oth- erwise. • Optimality. Complexity theory is a separate area in itself! CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Design paradigms Many algorithms can also be fitted into cer- tain paradigms . The following are popular: • divide-conquer-combine • greedy • dynamic programming • backtracking • branch-and-bound • plane sweep • randomization (in a broad sense) Quiz: For each paradigm, give one problem- algorithm. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Some old all-time greats • 1947: George Dantzig - Simplex method for LP. • 1951: Alston Householder - Decomposi- tional approach to matrix computation. • 1957: John Backus - FORTRAN opti- mizing compiler. • 1962: C.A.R.Hoare - Quicksort. • 1965: James Cooley and John Tukey - Fast Fourier Transform. • 1969: Volker Strassen - Fast matrix mul- tiplication. CSED/NIT, Trichy May 2008

K.V.Iyer Algorithms/B.Tech. Some references 1. A.V.Aho, J.E.Hopcroft and J.D.Ullman, The Design and Analysis of Computer Algo- rithms, Addison-Wesley, 1974. 2. S.Baase and A.v. Gelder, Computer Algo- rithms - Introduction to Design and Analysis, 3/e, Pearson-Education, 2000. 3. G.Brassard and P.Bratley, Fundamentals of Algorithmics, Prentice-Hall, 1996. 4. T.H.Cormen, C.E.Lieserson, R.L.Rivest and C.Stein, Introduction to Algorithms, 2/e, MIT, 2001. 5. E.Horowitz and S.Sahni, Fundamentals of Computer Algorithms, Comp. Sci. Press. 1998. 6. H.Wilf, Algorithms and Complexity, Prentice- Hall, 1986. CSED/NIT, Trichy May 2008