CS 401 Integer Multiplication / Matrix Multiplication Xiaorui Sun - PowerPoint PPT Presentation

CS 401 Integer Multiplication / Matrix Multiplication Xiaorui Sun 1

Integer Multiplication

Integer Arithmetic 1 1 1 1 1 1 0 1 Add: Given two ! -bit integers 1 1 0 1 0 1 0 1 " and # , compute " + # . Add + 0 1 1 1 1 1 0 1 1 0 1 0 1 0 0 1 0 &(!) bit operations. 1 1 0 1 0 1 0 1 * 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 Multiply: Given two ! -bit 0 0 0 0 0 0 0 0 Multiply integers " and # , compute "×# . 1 1 0 1 0 1 0 1 The “grade school” method: 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 &(! ) ) bit operations. 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 3

Divide and Conquer Let !, # be two $ -bit integers Write ! = 2 '/) ! * + ! , and # = 2 '/) # * + # , where ! , , ! * , # , , # * are all $/2 -bit integers. ! = 2 '/) ⋅ ! * + ! , # = 2 '/) ⋅ # * + # , !# = 2 '/) ⋅ ! * +! , 2 '/) ⋅ # * + # , = 2 ' ⋅ ! * # * + 2 ⁄ ' ) ⋅ ! * # , + ! , # * + ! , # , Therefore, We only need 3 values - $ = 4- $ ! * # * , ! , # , , ! * # , + ! , # * 2 + Θ($) Can we find all 3 by only 3 multiplication? So, - $ = Θ $ ) .

Key Trick: 4 multiplies at the price of 3 ! = 2 $/& ⋅ ! ( + ! * + = 2 $/& ⋅ + ( + + * !+ = 2 $/& ⋅ ! ( +! * 2 $/& ⋅ + ( + + * = 2 $ ⋅ ! ( + ( + 2 ⁄ $ & ⋅ ! ( + * + ! * + ( + ! * + * - = ! ( + ! * . = + ( + + * -. = ! ( + ! * + ( + + * = ! ( + ( + ! ( + * + ! * + ( + ! * + * ! ( + * + ! * + ( = -. − ! ( + ( − ! * + *

Key Trick: 4 multiplies at the price of 3 Theorem [Karatsuba-Ofman, 1962] Can multiply two n-digit integers in O(n 1.585… ) bit operations. ) = 2 */+ ⋅ ) - + ) . ⇒ 0 = ) - + ) . 1 = 2 */+ ⋅ 1 - + 1 . ⇒ 2 = 1 - + 1 . )1 = 2 */+ ⋅ ) - +) . 2 */+ ⋅ 1 - + 1 . = 2 * ⋅ ) - 1 - + 2 ⁄ * + ⋅ ) - 1 . + ) . 1 - + ) . 1 . 02 − 5 − 6 B A To multiply two n-bit integers: Add two !/2 bit integers. Multiply three !/2 -bit integers. Add, subtract, and shift !/2 -bit integers to obtain result. $ ! = 3$ ! 2 + ( !

Quiz Consider the following recurrence. O 1 if " = 1 ! " = 4 3!("/2) + O(") if " > 1 • A. ! " = O " %&' ( ) = *(" ,../. ) • B. ! " = O "log " • C. ! " = O " • D. None of above

Integer Multiplication (Summary) • Amusing exercise: generalize Karatsuba to do 5 size !/3 subproblems This gives Θ ! %.'(… time algorithm Still open problem.

Matrix Multiplication

Multiplying Matrices Let ! be an "×$ matrix, % be an $×& matrix. Then, ' = !% is an "×& matrix such that 10

Multiplying Matrices é ù é ù a a a a b b b b 11 12 13 14 11 12 13 14 ê ú ê ú a a a a b b b b ê ú ê ú 21 22 23 24 • 21 22 23 24 ê ú ê ú a a a a b b b b 31 32 33 34 31 32 33 34 ê ú ê ú a a a a b b b b ë û ë û 41 42 43 44 41 42 43 44 + + + + + + + + + é ! ù a b a b a b a b a b a b a b a b a b a b a b a b 11 11 12 21 13 31 14 41 11 12 12 22 13 32 14 42 11 14 12 24 13 34 14 44 ê ú + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b ê ú 21 11 22 21 23 31 24 41 21 12 22 22 23 32 24 42 21 14 22 24 23 34 24 44 = ê ú + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b 31 11 32 21 33 31 34 41 31 12 32 22 33 32 34 42 31 14 32 24 33 34 34 44 ê ú + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b ë û 41 11 42 21 43 31 44 41 41 12 42 22 43 32 44 42 41 14 42 24 43 34 44 44 11

Multiplying Matrices é ù é ù a a a a b b b b 11 12 13 14 11 12 13 14 ê ú ê ú a a a a b b b b ê ú ê ú 21 22 23 24 • 21 22 23 24 ê ú ê ú a a a a b b b b 31 32 33 34 31 32 33 34 ê ú ê ú a a a a b b b b ë û ë û 41 42 43 44 41 42 43 44 + + + + + + + + + é ! ù a b a b a b a b a b a b a b a b a b a b a b a b 11 11 12 21 13 31 14 41 11 12 12 22 13 32 14 42 11 14 12 24 13 34 14 44 ê ú + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b ê ú 21 11 22 21 23 31 24 41 21 12 22 22 23 32 24 42 21 14 22 24 23 34 24 44 = ê ú + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b 31 11 32 21 33 31 34 41 31 12 32 22 33 32 34 42 31 14 32 24 33 34 34 44 ê ú + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b ë û 41 11 42 21 43 31 44 41 41 12 42 22 43 32 44 42 41 14 42 24 43 34 44 44 12

Multiplying Matrices é ù é ù a a a a b b b b 11 12 13 14 11 12 13 14 A 11 A 12 B 11 B 12 ê ú ê ú a a a a b b b b ê ú ê ú 21 22 23 24 • 21 22 23 24 ê ú ê ú a a a a b b b b 31 32 33 34 31 32 33 34 A 21 B 21 B 22 ê ú ê ú A 22 a a a a b b b b ë û ë û 41 42 43 44 41 42 43 44 + + + + + + + + + é ! ù a b a b a b a b a b a b a b a b a b a b a b a b 11 11 12 21 13 31 14 41 11 12 12 22 13 32 14 42 11 14 12 24 13 34 14 44 A 11 B 11 +A 12 B 21 A 11 B 12 +A 12 B 22 ê ú + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b ê ú 21 11 22 21 23 31 24 41 21 12 22 22 23 32 24 42 21 14 22 24 23 34 24 44 = ê ú + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b 31 11 32 21 33 31 34 41 31 12 32 22 33 32 34 42 31 14 32 24 33 34 34 44 A 21 B 11 +A 22 B 21 ê ú A 21 B 12 +A 22 B 22 + + + + + + + + + ! a b a b a b a b a b a b a b a b a b a b a b a b ë û 41 11 42 21 43 31 44 41 41 12 42 22 43 32 44 42 41 14 42 24 43 34 44 44 13

Simple Divide and Conquer A 11 A 12 B 11 B 12 A 21 B 21 B 22 A 22 A 11 B 11 +A 12 B 21 A 11 B 12 +A 12 B 22 = A 21 B 11 +A 22 B 21 A 21 B 12 +A 22 B 22 14

Strassen’s Divide and Conquer Algorithm Naive Strassen 15

Strassen’s Divide and Conquer Algorithm • Strassen’s algorithm Multiply !×! matrices using # instead of $ multiplications (and 18 additions) ' ( = 7' ( 2 + 18( - Hence, we have ' ( = . ( /01 2 3 . Useful when (~500 . One of the most important open problem: Solve matrix multiplication in O(( - log = > () time I am curious how large ( need? 16

Summary of Divide and Conquer Divide: We reduce a problem to several subproblems. each sub-problem is at most a constant n fraction of the size of the original problem Log n levels n/2 n/2 Conquer: Recursively solve each subproblem n/4 Combine: Merge the solutions Examples: • Mergesort, Binary Search, Closest Point Pair, Integer Multiplication, Matrix Multiplication

How to solve problems by D&C Define subproblems • Sometimes, subproblems are same to the original problem (just input size is smaller), e.g. Mergesort • Sometimes, need to redefine the subproblems to allow solving subproblem recursively, e.g. Counting Inversions Algorithm design: how to obtain solution of a subproblem from solutions of smaller subproblems? 18

Master Theorem Write down recurrence relation ! " = $ ! " % + Θ(" ) ) + , + Θ(" ) ) for all Master theorem: Suppose ! " = $ ! " > %. Then, • If $ < % ) then ! " = Θ " ) • If $ = % ) then ! " = Θ " ) log " • If $ > % ) then ! " = Θ " 345 6 7