x 2 + 1 = 0 ( ) has no solutions at all if we restrict our - PowerPoint PPT Presentation

Complex numbers Any polynomial p ( x ) of of degree d ought to have d roots. (I.e., p ( x ) = 0 should have d solutions.) But the equation x 2 + 1 = 0 ( ∗ ) has no solutions at all if we restrict our attention to real numbers. Introduce a special symbol i to stand for a solution to ( ∗ ) . Then i 2 = − 1 and ( ∗ ) has the required two solutions, i and − i . Adding i allows all polynomial equations to be solved! Indeed a polynomial of degree d has d roots (taking account of multiplicities). This is the Fundamental Theorem of Algebra . 1 A&DS Lecture 4 Mary Cryan

Roots of unity In particular, x n = 1 has n solutions in the complex numbers. They may be written 1, ω n , ω 2 n , . . . , ω n − 1 n where ω n is the principal n th root of unity : ω n = cos ( 2π/n ) + i sin ( 2π/n ) , ( † ) . Convention: from now on ω n denotes the principal n th root of unity given by ( † ) . Note: e iu = cos u + i sin u so ω n = e 2πi/n . 2 A&DS Lecture 4 Mary Cryan

8th Roots of Unity 2 w = i 8 = ei*2pi/8 w 8 = (cos (2 pi/8), i*sin(2 pi/8)) = (1+i)/sqrt(2) 2pi/8 w0 = 1 8 “Wheel” representation of 8th roots-of-unity (complex plane)). Same wheel structure for any n (then ω n found at angle 2π/n ). 3 A&DS Lecture 4 Mary Cryan

The Discrete Fourier Transform (DFT) Instance A sequence of n complex numbers a 0 , a 1 , a 2 , . . . , a n − 1 . Output The sequence of n complex numbers n ) , . . . , A ( ω n − 1 A ( 1 ) , A ( ω n ) , A ( ω 2 ) n obtained by evaluating the polynomial A ( x ) = a 0 + a 1 x + a 2 x 2 + · · · + a n − 1 x n − 1 at the n th roots of unity. The DFT is a fingerprint of size n of a polynomial. It’s not the only fingerprint (why?) 4 A&DS Lecture 4 Mary Cryan

Motivation for algorithms for DFT/Inverse DFT Direct. Signal processing: mapping between time and frequency domains. Indirect. Subroutine in numerous applications, e.g., multiplying polynomials or large integers, cyclic string matching, etc. It is important, therefore to find the fastest method. There is an obvious Θ ( n 2 ) algorithm. Can we do better? YES! Really cool algorithm (Fast Fourier Transform (FFT)) runs in O ( n lg n ) time. Published by Cooley & Tukey in 1965 - basics known by Gauss in 1805! Used in *every* Digital Signal Processing application. Probably the most Important algorithm of today. In Lecture 5, we’ll show how to apply FFT to get polynomial mult. in O ( n lg n ) (not the main application, but cute!). 5 A&DS Lecture 4 Mary Cryan

Divide and conquer: attempt 1 We are interested in evaluating: A ( x ) = a 0 + a 1 x + a 2 x 2 + · · · + a n − 1 x n − 1 . Assume n even. Put a 0 + a 1 y + · · · + a n/2 − 1 y n/2 − 1 , A small ( y ) = a n/2 + a n/2 + 1 y + · · · + a n − 1 y n/2 − 1 A big ( y ) = so that A ( x ) = A small ( x ) + x n/2 A big ( x ) . To evaluate A ( x ) at the n th roots of unity, we need to evaluate A small ( y ) and A big ( y ) at the n th roots of unity. But these are not DFTs! Try again . . . 6 A&DS Lecture 4 Mary Cryan

Divide and conquer: attempt 2 We are interested in evaluating: A ( x ) = a 0 + a 1 x + a 2 x 2 + · · · + a n − 1 x n − 1 . Assume n even. Put a 0 + a 2 y + · · · + a n − 2 y n/2 − 1 , A even ( y ) = a 1 + a 3 y + · · · + a n − 1 y n/2 − 1 , A odd ( y ) = so that A ( x ) = A even ( x 2 ) + x A odd ( x 2 ) . To evaluate A ( x ) at the n th roots of unity, we need to evaluate n , . . . , ω 2 ( n − 1 ) A even ( y ) and A odd ( y ) at the points 1, ω 2 n , ω 4 . n We’ll show now that these are DFTs. 7 A&DS Lecture 4 Mary Cryan

Key facts Assuming n is even: n ) 2 = e 2πi 2πi n/2 = ω n/2 , and • ω 2 n = ( e n ) n/2 = e πi = − 1 . • ω n/2 2πi = ( e n Thus we have the following relationships between ω n and ω n/2 : ω 2 ( n − 1 ) ω 2 ω n − 2 ω n ω n + 2 . . . . . . 1 n n n n n � � � � � � . . . . . . ω n/2 − 1 ω n/2 − 1 ω n/2 . . . 1 ω n/2 . . . 1 n/2 n/2 8 A&DS Lecture 4 Mary Cryan

Key Facts (cont’d) A ( 1 ) = A even ( 1 ) + 1 · A odd ( 1 ) A ( ω n ) = A even ( ω n/2 ) + ω n A odd ( ω n/2 ) A ( ω 2 n ) = A even ( ω 2 n/2 ) + ω 2 n A odd ( ω 2 n/2 ) . . . ) = A even ( ω n/2 − 1 A odd ( ω n/2 − 1 A ( ω n/2 − 1 ) + ω n/2 − 1 ) n n n/2 n/2 9 A&DS Lecture 4 Mary Cryan

Key Facts (cont’d) A ( ω n/2 ) = A even ( 1 ) − 1 · A odd ( 1 ) n A ( ω n/2 + 1 ) = A even ( ω n/2 ) − ω n A odd ( ω n/2 ) n . . . ) = A even ( ω n/2 − 1 A odd ( ω n/2 − 1 A ( ω n − 1 ) − ω n/2 − 1 ) n n n/2 n/2 10 A&DS Lecture 4 Mary Cryan

The Fast Fourier Transform (FFT) A ( x ) = a 0 + a 1 x + a 2 x 2 + · · · + a n − 1 x n − 1 , assume n is a power of 2 . Compute A ( 1 ) , A ( ω n ) , A ( ω 2 n ) , . . . , A ( ω n − 1 ) , ( ∗ ) n as follows: 1. If n = 1 then A ( x ) is a constant so task is trivial. Otherwise split A into A even and A odd . 2. By making two recursive calls compute the values of A even ( y ) and n/2 , . . . , ω n/2 − 1 A odd ( y ) at the ( n/2 ) points 1, ω n/2 , ω 2 . n/2 3. Compute the values ( ∗ ) by using the equation A ( x ) = A even ( x 2 ) + xA odd ( x 2 ) . 11 A&DS Lecture 4 Mary Cryan

Implementation Algorithm FFT ( � a 0 , . . . , a n − 1 � ) 1. if n = 1 then return � a 0 � 2. ω n ← e 2πi/n 3. ω ← 1 4. � y even 0 , . . . , y even n/2 − 1 � ← FFT ( � a 0 , a 2 , . . . , a n − 2 � ) 5. � y odd 0 , . . . , y odd n/2 − 1 � ← FFT ( � a 1 , a 3 , . . . , a n − 1 � ) 6. for k ← 0 to n/2 − 1 do y k ← y even + ωy odd 7. k k y k + n/2 ← y even − ωy odd 8. k k 9. ω ← ωω n 10. return � y 0 , . . . , y n − 1 � Remark 4.1 Algorithm assumes that n is a power of 2 . If it is not, input sequence is padded by 0 s. 12 A&DS Lecture 4 Mary Cryan

Analysis T ( n ) worst-case running time of FFT. • Lines 1–3: Θ ( 1 ) • Lines 4–5: Θ ( 1 ) + 2T ( n/2 ) • Loop in lines 6–9: Θ ( n ) • Line 10: Θ ( 1 ) Yields the following recurrence: T ( n ) = 2T ( n/2 ) + Θ ( n ) . Solution: T ( n ) = Θ ( n · lg ( n )) . 13 A&DS Lecture 4 Mary Cryan

Reading Assignment Fast Fourier Transform , by M. Cryan, notes handed out today. [CLRS] Section 30.2 (pp. 830-838). Section 32.2 (pp. 783-791) in [CLR]. Optional Reading on DSP applications (see course webpage for these). Problems 1. Exercise 30.2-2, p. 838 of [CLRS]. Ex. 32.2-2, p. 790 of [CLR]. 2. Let f ( x ) = 3 cos ( 2x ) . For 0 ≤ k ≤ 3 , let a k = f ( 2πk/4 ) . Compute the DFT of � a 0 , . . . , a 3 � . Do the same for f ( x ) = 5 sin ( x ) . 14 A&DS Lecture 4 Mary Cryan