Sharpness Search Algorithms for Automatic Focusing in the Scanning - - PowerPoint PPT Presentation

Sharpness Search Algorithms for Automatic Focusing in the Scanning Electron Microscope

C.F. Batten, D.M. Holburn, B.C. Breton, N.H.M. Caldwell Scientific Imaging Group Cambridge University Scanning May 7th, 2001

May 7th, 2001 Scanning 2

Motivation

• There is a growing need for instrument automation

– New applications have led to an increase in the number of novice SEM operators – Remote microscopy requires simple commands which perform more work

• Focusing is an ideal candidate for automation

– Simplifies a common and tedious operation – Helps make remote microscopy practical

May 7th, 2001 Scanning 3

Previous Work

• Scanning Electron Microscopy

– Software solution using image gradient [Tee79] – Hardware solution using image covariance [Erasmus82] – Software solution using frequency domain [Ong98, Ogasawara99] – Use of a general imaging model to predict best focus [Nicolls95]

• Optical Microscopy

– Survey of sharpness measures [Groen85, Firestone91] – Use of a Fibonacci search to find the best focus [Yeo93]

May 7th, 2001 Scanning 4

Our Approach

• Traditional autofocusing approaches

– Try to integrate additional functionality such as astigmatism correction or topological mapping – Use a fixed stepsize or iterative search and avoid more sophisticated search algorithms due to low SNR and hysteresis concerns

• Our approach

– Make a dedicated autofocusing search algorithm Increase the SNR for each image Use a more sophisticated search algorithm Decrease the number of required image captures

May 7th, 2001 Scanning 5

Outline

• Sharpness Measures

– Gradient measure – Frequency domain measure – Autocorrelation measures – Variance measure

• Sharpness Search Algorithms

– Fixed stepsize search – Fixed stepsize search with interpolation – Iterative search – Variable stepsize search – Fibonacci search

• Conclusions

May 7th, 2001 Scanning 6

Evaluating Sharpness Measures

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 10 20 30 40 50 60 Focus Step (0.01mm) Variance Sharpness Measure Forward Sweep 1 Forward Sweep 2 Forward Sweep 3

Strictly Unimodal Property Sharpness measure should have one peak at the best focus and strictly decrease away from this maximum

May 7th, 2001 Scanning 7

Gradient Measure

• Sum of the difference between every nth pixel in

both the X and Y directions

• As image comes into focus, edges become sharper

increasing the image gradient

• Sharpness Measure Properties

– Relatively easy to calculate (one of the first sharpness measures) – Very susceptible to noise – The parameter n acts a low-pass filter in the spatial domain

• (n = 1) Traditional image gradient
• (n = 2) Brenner method
• (n > 2) As long as n < feature size, can increase noise robustness

May 7th, 2001 Scanning 8

Frequency Domain Measure

• Perform Fourier transform and then sum the frequency

components below threshold frequency (Ω)

• As image comes into focus, edges become sharper which

increases the magnitude of medium frequency components

• Sharpness Measure Properties

– Allows easy integration of astigmatism correction – Fourier transform in software is computationally expensive – The parameter Ω acts as a low-pass filter in frequency domain

• Varying Ω produces similar results as varying n
• For this work, Ω chosen to be 50

May 7th, 2001 Scanning 9

Auto-correlation Measures

• Auto-correlation function is the image convolved with itself

and indicates how well neighboring pixels are correlated

• Tested two measures using the image auto-correlation

– ACFdiff Height of the central ACF peak – ACFsum Area under the central ACF peak

• Focused images contain small highly correlated regions that

result in a tall sharp central ACF peak

• Sharpness Measure Properties

– Can calculate ACF efficiently in the frequency domain – Do not need to calculate entire ACF for ACFdiff measure – Correlated noise due to limited bandwidth distortion made using the ACF more difficult

May 7th, 2001 Scanning 10

Variance Measure

• Sum the square of the difference between each pixel

and the mean image intensity

• Focused images have greater intensity variation then

blurred defocused images

• Sharpness Measure Properties

– Simple and efficient implementation – Very robust to noise – Strong adherence to the strict unimodality property

For these reasons the variance measure was selected as the primary sharpness measure for this work

May 7th, 2001 Scanning 11

Comparison of Sharpness Measures

10 20 30 0.4 0.6 0.8 1 (a) fgrad 10 20 30 −0.5 0.5 1 (c) fACFdiff 10 20 30 0.2 0.4 0.6 0.8 1 (b) ffreq 10 20 30 0.2 0.4 0.6 0.8 1 (e) fvar Pix Avg: 1 FAvg: 3 Pix Avg: 1 FAvg: 5 Pix Avg: 2 FAvg: 3 Pix Avg: 2 FAvg: 10 Pix Avg: 4 FAvg: 15 Pix Avg: 8 FAvg: 20 Pix Avg: 16 FAvg: 25 10 20 30 0.2 0.4 0.6 0.8 1 (d) fACFsum

Sharpness Measures at Various Noise Levels

May 7th, 2001 Scanning 12

Sharpness Search Algorithms

• Investigated five sharpness search algorithms

– Fixed stepsize search – Fixed stepsize search with interpolation – Iterative search – Variable stepsize search – Fibonacci search

• Notation

– l Search interval – α Desired accuracy (How close to optimum is acceptable?) – N Number of required image captures

• Goal is to find a search algorithm which minimizes N

but still achieves the desired accuracy α

May 7th, 2001 Scanning 13

Fixed Stepsize Search

• Single sweep over search interval with stepsize = 2α
• Theoretical N given by
• Peak finding reduces N
• Developed a novel method

to adjust for hysteresis effects based on relative sharpness when returning to best focus

4 4.5 5 5.5 6 6.5 400 500 600 700 800 900 1000 1100 Focal Length (mm) Variance Sharpness Measure Variance Moving Average

      + = 1 2 α l N

Fixed Stepsize Search with Peak Finding

May 7th, 2001 Scanning 14

Fixed Stepsize with Interpolation

• Interpolation can help reduce the number of image

captures while maintaining the desired accuracy

• Quadratic and linear interpolation do not perform well
• n typical variance curves
• A New Interpolation Approach

– Erasmus and Smith provide a derivation for image variance as a function of defocus [Erasmus82] – Use non-linear regression to curve fit the derived function with the collected data – This allows us to significantly reduce the required number of image captures, but is computationally expensive

May 7th, 2001 Scanning 15

Fixed Stepsize with Interpolation

8.5 8.61 8.72 8.83 8.94 0.2 0.4 0.6 0.8 1 1.2 1.4 Focal Length (mm) Normalized Variance (a) Gold on Carbon − 22,200x 8.56 8.6 8.65 8.69 8.74 0.2 0.4 0.6 0.8 1 1.2 Focal Length (mm) Normalized Variance (b) Gold on Carbon − 75,500x 8 8.66 9.33 10 10.66 0.6 0.8 1 1.2 Focal Length (mm) Normalized Variance (c) IC Tracks − 3,400x 6 7.55 9.11 10.66 12.22 0.6 0.7 0.8 0.9 1 1.1 1.2 Focal Length (mm) Normalized Variance (d) IC Tracks − 3,400x 8.69 8.98 9.27 9.56 9.85 0.5 0.6 0.7 0.8 0.9 1 1.1 Focal Length (mm) Normalized Variance (e) IC Tracks − 9,900x 7 8.33 9.66 11 12.33 0.4 0.6 0.8 1 1.2 1.4 Focal Length (mm) Normalized Variance (f) Sublimated Titanium − 1,500x

Derived Variance Function Fitted to Data from Various Specimens

May 7th, 2001 Scanning 16

Iterative Search

• Several sweeps with gradually smaller stepsizes

and search intervals

• Theoretical N given by

where η is the number

• f image captures per

iteration

      = 1))

• η

/( 2 log( ) / α log( η l N

6 6.5 7 7.5 8 8.5 9 9.5 300 350 400 450 500 550 600 Focal Length (mm) Variance Sharpness Measure Iteration 1 Iteration 2 Iteration 3 Iteration 4

Online Iterative Focus Sweep (η=8)

May 7th, 2001 Scanning 17

Variable Stepsize Search

• Reduce the stepsize as the sharpness increases
• A common technique in other maximum search problems,

but not used in SEM autofocusing due to low image SNR

• We adapt the algorithm as follows

– Reduce stepsize based on moving average of variance – Set 2α as a lower bound on the stepsize – Use peak finding – Perform final fixed stepsize search if stepsize is greater than 2α once the peak is found

• Actual number of image captures varies based on initial

stepsize and specific variance curve

May 7th, 2001 Scanning 18

Variable Stepsize Search

4 4.5 5 5.5 6 6.5 400 500 600 700 800 900 1000 1100 (a) Sharpness During Variable Stepsize Sweep Focal Length (mm) Variance Sharpness Measure Variance Moving Average 4 4.5 5 5.5 6 6.5 0.1 0.15 0.2 0.25 0.3 0.35 (b) Stepsize During Variable Stepsize Search Focal Length (mm) Stepsize (mm)

Example of Variable Stepsize Search

May 7th, 2001 Scanning 19

Fibonacci Search

• An iterative search where η = 1
• Use previous measurements and one new measurement to

narrow search interval

• To avoid adverse hysteresis

effects, must set instrument to small focal length before each image capture (~200ms)

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 400 500 600 700 800 900 1000 1100 Focal Length (arbitrary units) Sharpness Measure

• 1

2 3 4 5 6 7 a b

Fibonacci Search Image Captures

May 7th, 2001 Scanning 20

Fibonacci Search

7.1 7.15 7.2 7.25 7.3 7.35 7.4 200 400 600 800 1000 1200 1400 1600 1800 Focal Length (mm) Variance Sharpness Measure Fixed Stepsize Search Fibonacci Search

Example of Fibonacci Search

May 7th, 2001 Scanning 21

Results: Number of Image Captures

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 10 20 30 40 50 60 70 80 90 100 Relative Accuracy (accuracy/search interval) Number of Required Image Captures Fixed Stepsize Search Iterative Search (η = 8) Variable Stepsize Search (Test #1) Variable Stepsize Search (Test #2) Fibonacci Search

Total Search Time vs. Relative Accuracy

May 7th, 2001 Scanning 22

Results: Total Search Time

0.014 0.016 0.018 0.02 0.022 0.024 0.026 0.028 0.03 0.032 20 40 60 80 100 120 140 160 Relative Accuracy (accuracy/search interval) Time (seconds) Fixed Stepsize Search with Interpolation Iterative Search Variable Stepsize Search Fibonacci Search

Total Search Time vs. Relative Accuracy

May 7th, 2001 Scanning 23

Results: Relative Sharpness

• Hysteresis effects prevent us from just comparing the best

focus produced by each sharpness search algorithm

• Use relative sharpness as a more accurate metric

(a) Gold on Carbon 25,800x (b) Integrated Circuit 970x (c) Sublimated Titanium 1,350x (d) Etched Silicon 410x

Specimens Used for Relative Sharpness Tests

May 7th, 2001 Scanning 24

Results: Relative Sharpness

FIX ITP ITR VAR FIB 440 460 480 500 520 540 (b) Integrated Circuit Best Focus Variance FIX ITP ITR VAR FIB 1000 1100 1200 1300 1400 (c) Sublimated Titanium Best Focus Variance FIX ITP ITR VAR FIB 380 400 420 440 460 (d) Etched Silicon Best Focus Variance FIX ITP ITR VAR FIB 1400 1600 1800 2000 2200 2400 2600 (a) Gold on Carbon Best Focus Variance

Best Focus Sharpness for Various Search Methods

May 7th, 2001 Scanning 25

Conclusions

• 1. The variance measure is an effective sharpness

measure that is well suited for autofocusing in the scanning electron microscope.

• 2. Autofocusing research has traditionally concentrated
• n fixed stepsize and iterative searches, but more

sophisticated search algorithms can successfully reduce the total search time.