Stochastic Search for Signal Processing Algorithm Optimization - - PowerPoint PPT Presentation

β–Ά
stochastic search for signal
SMART_READER_LITE
LIVE PREVIEW

Stochastic Search for Signal Processing Algorithm Optimization - - PowerPoint PPT Presentation

Feb 22, 2024 268 likes β€’558 views

Stochastic Search for Signal Processing Algorithm Optimization Bryan Singer and Manuela Veloso Computer Science Department Carnegie Mellon University, Pittsburgh Liat Ben-Haim, November 16th 2011 Overview Genetic algorithm Signal

slide-1
SLIDE 1

Stochastic Search for Signal Processing Algorithm Optimization

Bryan Singer and Manuela Veloso Computer Science Department Carnegie Mellon University, Pittsburgh Liat Ben-Haim, November 16th 2011

slide-2
SLIDE 2

Overview

  • Genetic algorithm
  • Signal processing
  • Walsh-Hadamard Transform
  • STEER: Split Tree Evolution for Efficient Runtimes
  • For Walsh-Hadamard Transform
  • Results Walsh-Hadamard Transform
  • For Arbitrary Transform
  • Results Arbitrary Transform
  • Conclusion: Strengths and Weaknesses
slide-3
SLIDE 3

Genetic Algorithm

Complete Search space (very large) Base Population

slide-4
SLIDE 4

Genetic Algorithm

Crossover (only parts of the population) Mutation (only parts of the population)

Survival of the fittest Repeat Base Population

slide-5
SLIDE 5

General ↔ STEER

4 3 1 1 1 1 5 1 3 1 1 1 5 2 1 1 5 2 1 1 3 1 2 1 1 3 1 2 1 1 4 1 5 1 3 1 2 1 1 4 1 5 1

↔

  • Population consists of algorithms
  • Algorithms modeled as trees
  • Fitness is measured in runtime on given device
slide-6
SLIDE 6
  • Genetic algorithm
  • Signal processing
  • Walsh-Hadamard Transform
  • STEER: Split Tree Evolution for Efficient Runtimes
  • For Walsh-Hadamard Transform
  • Results Walsh-Hadamard Transform
  • For Arbitrary Transform
  • Results Arbitrary Transform
  • Conclusion: Strengths and Weaknesses
slide-7
SLIDE 7

Signal Processing: y = Ax

Ax

Pictures (JPEG) Audio (MP3)

http://commons.wikimedia.org/wiki/File:ALC_orig.png http://de.wikipedia.org/w/index.php?title=Datei:Mp3.svg&filetimestamp=20091118142210 http://de.wikipedia.org/w/index.php?title=Datei:Phalaenopsis_JPEG.png&filetimestamp=20110430130839

slide-8
SLIDE 8

Walsh-Hadamard Transform (WHT)

𝑧 = π‘‹πΌπ‘ˆ 2π‘œ β‹… 𝑦 π‘‹πΌπ‘ˆ 2π‘œ = 1 1 1 βˆ’1 ⨂ β‹― ⨂ 1 1 1 βˆ’1 n factors

𝑧 = 1 1 1 βˆ’1 β‹… 𝑦 𝑧 = 1 1 1 βˆ’1 ⨂ 1 1 1 βˆ’1 ⨂ 1 1 1 βˆ’1 β‹… 𝑦 = 1 1 1 1 1 1 1 1 1 βˆ’1 1 βˆ’1 1 βˆ’1 1 βˆ’1 1 1 βˆ’1 βˆ’1 1 1 βˆ’1 βˆ’1 1 βˆ’1 βˆ’1 1 1 βˆ’1 βˆ’1 1 1 1 1 1 βˆ’1 βˆ’1 βˆ’1 βˆ’1 1 βˆ’1 1 βˆ’1 βˆ’1 1 βˆ’1 1 1 1 βˆ’1 βˆ’1 βˆ’1 βˆ’1 1 1 1 βˆ’1 βˆ’1 1 βˆ’1 1 1 βˆ’1 β‹… 𝑦 𝑧 = 1 1 1 βˆ’1 ⨂ 1 1 1 βˆ’1 β‹… 𝑦 = 1 1 1 1 1 βˆ’1 1 βˆ’1 1 1 βˆ’1 βˆ’1 1 βˆ’1 βˆ’1 1 β‹… 𝑦

n = 1: n = 2: n = 3:

slide-9
SLIDE 9

Fast Walsh-Hadamard Transform

n = 2: 𝑧 = π‘‹πΌπ‘ˆ 22 βˆ™ 𝑦 = 𝐽21 ⨂ π‘‹πΌπ‘ˆ(21) βˆ™ π‘‹πΌπ‘ˆ 21 ⨂ 𝐽21 βˆ™ 𝑦 = = 1 1 1 βˆ’1 1 1 1 βˆ’1 βˆ™ 1 1 1 1 1 βˆ’1 1 βˆ’1 βˆ™ 𝑦 Opcount: 4 Opcount: 4+4 = 8 Opcount: 12 2 1 1

slide-10
SLIDE 10

Fast Walsh-Hadamard Transform

3 1 2 1 1 3 2 1 1 1 3 1 1 1

Opcount: 24 24 24 Opcount WHT(23): 56

n = 3:

slide-11
SLIDE 11

4

3

1 1 1 1 5 1 4

3

1 1 1 1 5 1 4

3

1 1 1 1 5 1 4

3

1 1 1 1 5 1

2

1 1 2 1 1 4 5 1 3

1 1 1 5 2 1 1

3

1 1 1 5 2 1 1

1 1 2 1 1 4 5 1 2 5 2 1 1 3 1 2 1 1 5 2 1 1 3 1 2 1 1 5 2 1 1 3 1 2 1 1 5 2 1 1 3 1 2 1 1

3 1 2 1 1 4 1

5

1

3 1 2 1 1 4 1

5

1

3 1 2 1 1 4 1

5

1

3 1 2 1 1 4 1

5

1 4 5 1 1 1 1 1 4 5 1 1 1 1 1 5 1 1 1 1 1

slide-12
SLIDE 12

General Break Down Rule

π‘‹πΌπ‘ˆ 2π‘œ = (𝐽2π‘œ1+β‹―+π‘œπ‘—βˆ’1 ⨂ π‘‹πΌπ‘ˆ 2π‘œπ‘— ⨂

𝑒 𝑗=1

𝐽2π‘œπ‘—+1+β‹―+π‘œπ‘’) π‘œ = π‘œ1 + β‹― + π‘œπ‘’ (π‘œπ‘˜: positive integers) n nt n1 ... ... ni

slide-13
SLIDE 13

WHT Example

π‘‹πΌπ‘ˆ 2π‘œ = (𝐽2π‘œ1+β‹―+π‘œπ‘—βˆ’1 ⨂ π‘‹πΌπ‘ˆ 2π‘œπ‘— ⨂

𝑒 𝑗=1

𝐽2π‘œπ‘—+1+β‹―+π‘œπ‘’) 𝐽20 ⨂ 𝐡 = A 𝐡 ⨂ 𝐽20 = A

Stochastic Search for Signal Processing Algorithm Optimization

slide-14
SLIDE 14
  • Genetic algorithm
  • Signal processing
  • Walsh-Hadamard Transform
  • STEER: Split Tree Evolution for Efficient Runtimes
  • For Walsh-Hadamard Transform
  • Results Walsh-Hadamard Transform
  • For Arbitrary Transform
  • Results Arbitrary Transform
  • Conclusion: Strengths and Weaknesses
slide-15
SLIDE 15

Given input signal x of size 2n and specific device, find fastest program for this signal size and device

Goal x

25

3 1 1 1 5 2 1 1

http://de.wikipedia.org/w/index.php?title=Datei:IBM_PC_5150.jpg&filetimestamp=20060811115558

slide-16
SLIDE 16
  • Exhaustive Search
  • Does not scale
  • Dynamic Programming
  • Assumption: Β«combination of optimal solutions for

subproblems leads to optimal solutionΒ»

  • K-Best DP
  • Search space restriction: Binary trees
  • Bad choices lead to inferior solution
  • Split Tree Evolution for Efficient Runtimes: STEER

Search Techniques

slide-17
SLIDE 17
  • Split Tree Evolution for Efficient Runtimes
  • Genetic Algorithm
  • Part of the SPIRAL research group
  • Β«Can we teach computers to write fast libraries?Β»
  • Adapted to the system used by the research group

STEER

http://www.spiral.net

slide-18
SLIDE 18

STEER: Genetic Algorithm

Crossover Mutation Survival of the fittest (runtime) Repeat Base Population

slide-19
SLIDE 19

WHT: Crossover

Stochastic Search for Signal Processing Algorithm Optimization

slide-20
SLIDE 20

WHT: Mutation

Stochastic Search for Signal Processing Algorithm Optimization

slide-21
SLIDE 21
  • Pentium III 450 MHz (32-Bit Architecture)
  • Linux 2.2.5-15
  • WHT Package from Johnson and PΓΌschel
  • Β«In search of the optimal Walsh-Hadamard transformΒ»,

2000

  • Leaves of sizes 21 to 28
  • Unrolled straight-line code

Results: WHT

slide-22
SLIDE 22

Stochastic Search for Signal Processing Algorithm Optimization

slide-23
SLIDE 23
  • Random tree generation:
  • Randomly choose an applicable break down rule
  • Apply to node, to generate a random set of children
  • Recursively apply to each child
  • Crossover
  • Equivalent nodes: same size and transform
  • New mutations

Arbitrary Signal Transform

slide-24
SLIDE 24

Arbitrary Transform: Mutation

Stochastic Search for Signal Processing Algorithm Optimization

slide-25
SLIDE 25

Stochastic Search for Signal Processing Algorithm Optimization

Results: Arbitrary Transform

slide-26
SLIDE 26

Stochastic Search for Signal Processing Algorithm Optimization

slide-27
SLIDE 27

Strengths & Weaknesses

  • STEER can be used for

arbitrary transforms using SPIRAL

  • Finds good if not

necessarily optimal solutions

  • Found solutions are

generally better than DP

  • Times significantly less

formulas than exhaustive search

  • Missing parameters of

the evolutionary algorithm

  • No guarantee for a

Β«goodΒ» solution

  • Times more formulas

than DP

  • No mention of how long

STEER usually runs

  • How much better than

Ax?

slide-28
SLIDE 28

Questions?