Stochastic Search for Signal Processing Algorithm Optimization - PowerPoint PPT Presentation

Stochastic Search for Signal Processing Algorithm Optimization Bryan Singer and Manuela Veloso Computer Science Department Carnegie Mellon University, Pittsburgh Liat Ben-Haim, November 16th 2011

Overview  Genetic algorithm  Signal processing  Walsh-Hadamard Transform  STEER: Split Tree Evolution for Efficient Runtimes  For Walsh-Hadamard Transform  Results Walsh-Hadamard Transform  For Arbitrary Transform  Results Arbitrary Transform  Conclusion: Strengths and Weaknesses

Genetic Algorithm Base Population Complete Search space (very large)

Genetic Algorithm Base Population Repeat Crossover Survival of the fittest (only parts of the population) Mutation (only parts of the population)

General ↔ STEER 5 5 4 1 2 3 3 1 1 1 1 2 2 1 5 ↔ 1 1 1 1 4 1 1 3 5 1 2 1 4 5 1 1 3 1 2 3 1 1 1 1 1 1 1 1  Population consists of algorithms  Algorithms modeled as trees  Fitness is measured in runtime on given device

 Genetic algorithm  Signal processing  Walsh-Hadamard Transform  STEER: Split Tree Evolution for Efficient Runtimes  For Walsh-Hadamard Transform  Results Walsh-Hadamard Transform  For Arbitrary Transform  Results Arbitrary Transform  Conclusion: Strengths and Weaknesses

Signal Processing: y = Ax Ax Pictures (JPEG) Audio (MP3) http://commons.wikimedia.org/wiki/File:ALC_orig.png http://de.wikipedia.org/w/index.php?title=Datei:Mp3.svg&filetimestamp=20091118142210 http://de.wikipedia.org/w/index.php?title=Datei:Phalaenopsis_JPEG.png&filetimestamp=20110430130839

Walsh-Hadamard Transform (WHT) 𝑋𝐼𝑈 2 𝑜 = 1 −1 ⨂ ⋯ ⨂ 1 1 1 𝑧 = 𝑋𝐼𝑈 2 𝑜 ⋅ 𝑦 −1 1 1 n = 1: n factors 𝑧 = 1 1 −1 ⋅ 𝑦 1 n = 2: 1 1 1 1 𝑧 = 1 −1 ⨂ 1 1 1 1 −1 1 −1 −1 ⋅ 𝑦 = ⋅ 𝑦 1 1 1 1 −1 −1 1 −1 −1 1 n = 3: 1 1 1 1 1 1 1 1 1 −1 1 −1 1 −1 1 −1 1 1 −1 −1 1 1 −1 −1 𝑧 = 1 −1 ⨂ 1 1 −1 ⨂ 1 1 1 1 −1 −1 1 1 −1 −1 1 −1 ⋅ 𝑦 = ⋅ 𝑦 1 1 1 1 1 1 1 −1 −1 −1 −1 1 −1 1 −1 −1 1 −1 1 1 1 −1 −1 −1 −1 1 1 1 −1 −1 1 −1 1 1 −1

Fast Walsh-Hadamard Transform n = 2: Opcount: 12 𝑧 = 𝑋𝐼𝑈 2 2 ∙ 𝑦 = 𝐽 2 1 ⨂ 𝑋𝐼𝑈(2 1 ) ∙ 𝑋𝐼𝑈 2 1 ⨂ 𝐽 2 1 ∙ 𝑦 = 1 1 0 0 1 0 1 0 1 −1 0 0 0 1 0 1 = ∙ ∙ 𝑦 0 0 1 1 1 0 −1 0 0 0 1 −1 0 1 0 −1 2 Opcount: 4 1 1 Opcount: 4+4 = 8

Fast Walsh-Hadamard Transform n = 3: 3 3 3 2 1 1 2 1 1 1 1 1 1 1 Opcount: 24 24 24 Opcount WHT(2 3 ): 56

5 5 5 5 4 1 1 4 4 1 1 4 3 1 1 3 1 3 3 1 1 1 1 1 1 1 1 1 1 1 1 1 5 5 5 5 5 1 1 4 4 3 2 2 2 3 3 2 2 2 2 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 5 5 5 5 5 2 2 2 3 3 3 4 1 4 1 1 1 1 1 2 1 1 1 1 2 2 1 1 3 3 1 1 1 1 1 5 1 1 5 1 2 2 1 4 1 1 1 1 1 4 1 5 5 1 3 3 1 5 4 1 1 4 1 2 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1

General Break Down Rule 𝑢 𝑋𝐼𝑈 2 𝑜 = (𝐽 2 𝑜1+⋯+𝑜𝑗−1 ⨂ 𝑋𝐼𝑈 2 𝑜 𝑗 ⨂ 𝐽 2 𝑜𝑗+1+⋯+𝑜𝑢 ) 𝑗=1 𝑜 = 𝑜 1 + ⋯ + 𝑜 𝑢 (𝑜 𝑘 : positive integers) n n 1 ... n t ... n i

WHT Example 𝑢 𝑋𝐼𝑈 2 𝑜 = (𝐽 2 𝑜1+⋯+𝑜𝑗−1 ⨂ 𝑋𝐼𝑈 2 𝑜 𝑗 ⨂ 𝐽 2 0 ⨂ 𝐵 = A 𝐽 2 𝑜𝑗+1+⋯+𝑜𝑢 ) 𝐵 ⨂ 𝐽 2 0 = A 𝑗=1 Stochastic Search for Signal Processing Algorithm Optimization

 Genetic algorithm  Signal processing  Walsh-Hadamard Transform  STEER: Split Tree Evolution for Efficient Runtimes  For Walsh-Hadamard Transform  Results Walsh-Hadamard Transform  For Arbitrary Transform  Results Arbitrary Transform  Conclusion: Strengths and Weaknesses

Goal Given input signal x of size 2 n and specific device, find fastest program for this signal size and device 5 x 3 2 5 2 1 1 1 1 1 http://de.wikipedia.org/w/index.php?title=Datei:IBM_PC_5150.jpg&filetimestamp=20060811115558

Search Techniques  Exhaustive Search  Does not scale  Dynamic Programming  Assumption: «combination of optimal solutions for subproblems leads to optimal solution»  K-Best DP  Search space restriction: Binary trees  Bad choices lead to inferior solution  Split Tree Evolution for Efficient Runtimes: STEER

STEER  Split Tree Evolution for Efficient Runtimes  Genetic Algorithm  Part of the SPIRAL research group  «Can we teach computers to write fast libraries?»  Adapted to the system used by the research group http://www.spiral.net

STEER: Genetic Algorithm Base Population Repeat Survival of the fittest Crossover (runtime) Mutation

WHT: Crossover Stochastic Search for Signal Processing Algorithm Optimization

WHT: Mutation Stochastic Search for Signal Processing Algorithm Optimization

Results: WHT  Pentium III 450 MHz (32-Bit Architecture)  Linux 2.2.5-15  WHT Package from Johnson and Püschel  «In search of the optimal Walsh-Hadamard transform», 2000  Leaves of sizes 2 1 to 2 8  Unrolled straight-line code

Stochastic Search for Signal Processing Algorithm Optimization

Arbitrary Signal Transform  Random tree generation:  Randomly choose an applicable break down rule  Apply to node, to generate a random set of children  Recursively apply to each child  Crossover  Equivalent nodes: same size and transform  New mutations

Arbitrary Transform: Mutation Stochastic Search for Signal Processing Algorithm Optimization

Results: Arbitrary Transform Stochastic Search for Signal Processing Algorithm Optimization

Stochastic Search for Signal Processing Algorithm Optimization

Strengths & Weaknesses  STEER can be used for  Missing parameters of arbitrary transforms the evolutionary using SPIRAL algorithm  Finds good if not  No guarantee for a necessarily optimal «good» solution solutions  Times more formulas  Found solutions are than DP generally better than DP  No mention of how long  Times significantly less STEER usually runs formulas than exhaustive  How much better than search Ax?

Questions?