FUNDAMENTAL PROBLEMS IN IMAGE DEBLURRING Dr Slavoljub Mijovi - - PowerPoint PPT Presentation

7/10/2019 SSIP 2019 Timisora , Romania 1

FUNDAMENTAL PROBLEMS IN IMAGE DEBLURRING

Dr Slavoljub Mijović

University of Montenegro Faculty of Natural Sciences and Mathematics Podgorica; MONTENEGRO smijovic@yahoo.com

7/10/2019 CEEPUS Lecture 2

Content Before all…; Image formation and blurring process; Modeling of Image Formation Process; Inverse Problem and Naïve Solution;  General imaging problems; Inverse Ill- posed Problems;  Image restoration by regularization; Case study-mammograms;

7/10/2019 3

“In the beginning God created the heavens and the earth. The earth was formless and void, and darkness was over the surface of the deep, and the Spirit of God was moving over the surface of the waters....

The Bible

7/10/2019 4

…Then God said, “Let there be light”; and there was light. God saw that the light was good; and God separated the light from the darkness….

Image Formation and Blurring Processes

7/10/2019 CEEPUS Lecture 5

The existence of light-or other forms of electromagnetic radiation is an essential requirement for an image to be created, captured, and perceived.

Types of Images:

• Reflection Images;
• Emission Images;
• Absorption Images;

The Greek word Optike-”Theory of vision”

7/10/2019 CEEPUS Lecture 6

IMAGE FORMATION-spatial variation of

some physical quantity

Radiograph of a disk-shaped object Micro-calcification in the breast glandular tissue

• X-ray fluence
• Optical density of film
• Grey-scale value on the

monitor “Structures of interest” “Structures of interest” “Background” “Background”

Task: Find Signal: (the difference between structures of interest and background!)

Contrast Sharpness noise

Image Aquizition, Formation, and digitization

7/10/2019 CEEPUS Lecture 7

An image as a visual two dimensional (2D) representation

• f an object produced by an imaging system.

7/10/2019 CEEPUS Lecture 8



Modeling of Image Formation process AP(P)ROXIMATION!

“Make everything as simple as possible, but not simpler.” Albert Einstein

A MODEL Spherical Cow? How we usually think?

Examples of images and blurring processes

7/10/2019 CEEPUS Lecture 10

• There is not an ideal imaging system;
• Optical system in the camera is out of focus;
• Lens are not perfect;
• Motion blur;
• Turbulence blur;
• Detecting systems is not perfect;

Our Task-Image Restoration

7/10/2019 CEEPUS Lecture 11

Image restoration is based on the attempt to improve the quality of an image through knowledge of the physical processes which led to its formation ... ...i.e. to find object function o, or the

• riginal information f(x,y) from the image

function g(x,y)

Do DECONVOLUTION

• r DEBLURING or

INVERSE or just make UNDO! IIt is scientific approach to find original image by using mathematical model

• f the blurring process

RECOVERING AS MUCH INFORMATION AS POSSIBLE FROM THE GIVEN IMPERFECT DATA!!!

7/10/2019 CEEPUS Lecture 12

Remember: We are mainly interested in the characteristics of the object by deriving information from the image! Objective versus subjective information “You cannot depend on your eyes when your imagination is out of focus-Mark Twain

7/10/2019 13

A legitimate question to ask: “ When faced with a practical imaging processing problem, which techniques should I use and in which sequence?” Naturally, there is no universal answer to this

• question. Most image processing solutions are

problem specific and usually involve applica several algorithms – in a meaningful sequenc to achieve the desired goal .

My recommendation is at the first step - “Clean” the image from the influence of the imaging system and any undesired process!!!

How is mathematically described an image formation ?

7/10/2019 CEEPUS Lecture 14

n f h g   

• PSF-a characteristic of the imaging device and is a

deterministic function;

• Object function-describes object surface or its internal

structure;

• Noise-a stochastic function which is a consequence of all the

unwanted external disturbances

• - Convolution operator which ‘smears’ (convolves) one

function with another

Linear Imaging System

7/10/2019 CEEPUS Lecture 15

Linear imaging systems-cont’d

7/10/2019 CEEPUS Lecture 16

 

   

 

     



' ' ' ' ' '

, , ; , , dy dx y x f y x y x h y x g

Linear superposition integral

The Point-Spread Function

7/10/2019 CEEPUS Lecture 17

   

 

   

   

' ' ' ' ' ' ' ' ' '

, ; , , , ; , , , , , y x y x h y x g dy dx y x y x h y y x x y x g y y x x y x f       





The Point Spread Function (PSF) describes the response of an imaging system to a point source or point object

Linear Shift-Invariant (LSI) systems and the convolution integral

7/10/2019 CEEPUS Lecture 18

   

 

   

     

D)

• (2

, , , , , , , ; ,

' ' ' ' ' ' ' ' ' '

y x h y x f y x g dy dx y x f y y x x h y x g y y x x h y x y x h         

 

       

A very large number of image formation process are well described by the process of convolution. If a system is Linear Shift-Invariant then the image formation is necessarily described by convolution

An example of the LSI system and the convolution integral

The scan was aquired with uniform speed over the patient. The derived signal is proportional To the gamma activity emanating from that region of the body beneath the aperture.

Inverse problems and Naïve Solution

• Complex links among measured image quantities and object

parameters:

• the cause-effect connection of investigated phenomenon is inverse;
• a characteristic of the object plays a role of “cause”, and the observed

data of the image, such as brightness -“effect”

• Our Task: Find
• from integral equation
• r in matrix formulation

 

D

• 2

) , ( ) , ( ) , ( y x f y x h y x q   

 

y x f ,

Data → Model parameters

g Hf 

Frequency space and Fourier transforms “a big picture”

7/10/2019 CEEPUS Lecture 21

The harmonic content of signals: The fundamental idea of Fourier analysis is that any signal, be it a function of time, space or any other variables, may be expressed as a weighted linear combination of harmonic (i.e. sine and cosine) functions having different periods or frequencies.

• The Fourier representation

is a complete alternative ;

• The space domain and

Fourier domain are reciprocal

7/10/2019 CEEPUS Lecture 22

Image Transformation Analysis &Processing

Distance Inten

FOURIER TRANSFORM Quite generally, we can transform the

information with any scan line signal into a series of sinusoidal functions

• f the appropriate amplitude and spatial frequency (the spatial frequency

spectrum) and vice-versa, we can synthesize any spatial signal by summing its harmonic components

Spatial period Spatial frequency=1/spatial period

7/10/2019 CEEPUS Lecture 23

Fourier Transform examples con’d

• 2
• 1.5
• 1
• 0.5

• 3
• 2
• 1

Different signals and its Fourier transform pairs

Filtering

7/10/2019 CEEPUS Lecture 24

Original image and its Fourier transform Low-pass filtered image High-pass filtered image

Linear systems and Fourier transforms

7/10/2019 CEEPUS Lecture 25

An imaging system operates on the constituent input harmonics and its quality can be assessed by its ability to transmit the input harmonics to the output

   

   

y x y x

k k H k k F y x h y x f , , } , , {    

The convolution theorem The Fourier transform of the convolution of the two functions is equal to the product of the individual transforms

The optical transfer function (OTF)

7/10/2019 CEEPUS Lecture 26

           

       

OTF the called is , , , , } , , { } , { , , ,

y x y x y x y x

k k H k k H k k F k k G y x h y x f y x g y x h y x f y x g         

The OTF is the frequency –domain equivalent of the PSF i.e. OTF derives its name from the fact that it determines how the individual spatial frequency pairs are transferred from input to output

 

k ,

 

.

• bject

the

• f

specter frequency

• spatial

image the

• f

specter frequency

• spatial

  MTF

The Naïve Solution

7/10/2019 CEEPUS Lecture 27

         

filter Inverse ) , ( 1 ) , ( } , ) , ( { ) , ( , ) , ( , , ,

1 y x y x y x y x y x y x y x y x y x

k k H k k Y k k G k k Y y x f k k G k k Y k k H k k G k k F     



and the end of my presentation ...but

7/10/2019 CEEPUS Lecture 28

Well-Posedness

Definition due to Hadamard, 1915: Given mapping A: X Y, equation

Ax=y

is well-posed provided

• (Existence)

(Uniqueness) ; and (Stability) Equation is ill-posed if it is not well-posed.

Ax=y

Object Image

7/10/2019 CEEPUS Lecture 29

Ill-posed problems

System of two linear equations

True solution Wrong solution Wrong solution

• An ill-posed problem means

that the large data sets may contain a surprisingly small amount of information about the object x y

Ill-posed problems-Differentiation- (Edge detection)

7/10/2019 CEEPUS Lecture 31

Regularization Remedy for ill-posedness (or ill-conditioning, in discrete case). Informal Definition: “Imposes stability on an ill-posed problem in a manner that yields accurate approximate solutions, often by incorporating prior information”.

Key idea is to introduce a’priori information (size of noise e.g) and assumptions about size and smoothness of desired solution!!!

REGULARIZATION OR BUYING AN EXPENSIVE EQUIPMENT? ANSWER: BOTH!

• Regularization theory provides a sound

mathematical basis for solving the problem

Back to our case:

7/10/2019 CEEPUS Lecture 32

 

   

) , ( , , ,

' ' ' ' ' '

y x n dy dx y x f y y x x h y x g      

     

           

       

) , ( ) , ( ) , ( ) , ( ) , ( ) , ( ) , ( ) , ( ) , ( , ) , ( , , , )} , ( , , { } , { ) , ( , , ,

k k H k k N k k F k k H k k N k k H k k G k k G k k Y k k k k N k k H k k F k k G y x n y x h y x f y x g y x n y x h y x f y x g

F

                

7/10/2019 CEEPUS Lecture 33

DEMONSTRATION OF NOISE INFLUENCE

Blured original noiseless

PSF MTF Restored

• riginal

Blured original with white noise

PSF MTF Restored

• riginal

Possible solutions:

• Inverse Filter;
• The Wiener Filter;
• Constrained

deconvolution;

• Blind deconvolution;
• Iterative

deconvolution and Lucy-Richardson algorithm

• Matlab functions:

deconvwnr; deconvreg; deconvblind; deconvlucy;

7/10/2019 CEEPUS Lecture 34

Acceptable solutions;

7/10/2019 CEEPUS Lecture 35

   

) , ( , , ) , ( ) , (

^ ^

y x n y x h y x y x g y x

f n

    

7/10/2019 CEEPUS Lecture 36

THE PROJECT-Optimization Dose-Image Quality in Mammography

Why Mammography image?

The goal of mammography is the early detection of breast cancer. A great challenge because of small signals and different shapes.

A Compromise

7/10/2019 CEEPUS Lecture 38 Image quality Radiation exposure

Early experiments in radiation

A L A R A

e.g. diagnostic information vs.. radiation dose

Phantom Mammo AT

7/10/2019 CEEPUS Lecture 39

• r even simpler test bar pattern

7/10/2019 CEEPUS Lecture 40

7/10/2019 CEEPUS Lecture 41

IMAGES FOR NOISE , PSF , LSF AND MTF ESTIMATION

7/10/2019 CEEPUS Lecture 42

The test image



+ noise

Blurred and Noisy Image

Some results

7/10/2019 CEEPUS Lecture 43

The Deblurred image NSR The Deblurred image using autocorr.

7/10/2019 CEEPUS Lecture 44