Image Operator Learning and Applications Igor S. Montagner Nina S. - PowerPoint PPT Presentation

Basic operator: dilation δ B ( S ) = { p ∈ E : ˇ B p ∩ S � = ∅ } δ B ( S ) S SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 33

Basic operator: dilation δ B ( S ) = { p ∈ E : ˇ B p ∩ S � = ∅ } How does the dilation work ? SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 34

Basic operator: dilation δ B ( S ) = { p ∈ E : ˇ B p ∩ S � = ∅ } SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 35

Basic operator: dilation δ B ( S ) = { p ∈ E : ˇ B p ∩ S � = ∅ } δ B ( S ) S SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 45

Basic operator: erosion and dilation SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 46

Another basic operator: hit-miss H ( A , B c ) ( S ) = { p ∈ E : A p ⊆ S and B c p ⊆ S c } = ε A ( S ) ∩ ε B c ( S c ) places where A hits S places where B c hits the background (or, equiva- lently, misses S ) B c A places detected by the hit- miss operator SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 47

Short break for an exercise Exercise 1 SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 48

Solution to exercise 1 – structuring element A B c A In red, positions x at which A x fits in the foreground SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 49

Solution to exercise 1 – structuring element B c B c A In red, positions x at which B c x fits in the background SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 50

Solution to exercise 1 – Intersection B c A Result: the holes SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 51

Many useful operators can be built by composing these and other simple operators SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 52

Example of an operator: Contour detection f − ε B ( f ) ε B ( S ) S − ε B ( S ) S SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 53

Example of an operator: Contour detection f − ε B ( f ) Which properties of this operator are interesting ? ε B ( S ) S − ε B ( S ) S SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 54

Example of an operator: Contour detection f − ε B ( f ) Which properties of this operator are interesting ? • Translation invariance • Local definition ε B ( S ) S − ε B ( S ) S SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 55

Translation invariance [ Ψ ( f )] p = Ψ ( f p ) Ψ( S ) SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 56

Translation invariance [ Ψ ( f )] p = Ψ ( f p ) Ψ( S ) [Ψ( S )] p SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 57

Translation invariance [ Ψ ( f )] p = Ψ ( f p ) Ψ( S ) [Ψ( S )] p S p SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 58

Translation invariance [ Ψ ( f )] p = Ψ ( f p ) Ψ( S ) [Ψ( S )] p S p Ψ( S p ) SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 59

Local definition Ψ is locally defined if there is a window W such that: p ∈ Ψ ( S ) ⇐ ⇒ p ∈ Ψ ( S ∩ W ′ ) for every p ∈ E , S ∈ K E , and W ′ ⊇ W The red pixel is a contour point SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 60

Window operator or W -operator W -operator: translation invariance + local definition There is a local function ψ that uniquely characterizes Ψ S Ψ( S ) z z � � Ψ ( S )( z ) = ψ SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 64

Back to contour detection Contour detection operator : S − ε B ( S ) – translation-invariant – locally defined Local function : To decide if a pixel is a(n internal) contour point or not, it suffices to check if it is in the foreground and if there is at least one pixel adjacent to it in the background . Considering 4-adjacency, a cross- window is sufficient SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 65

Local function of the contour detection operator X ψ ( X ) 00000 0 x 1 00001 0 x 2 x 3 x 4 x 1 x 2 x 3 x 4 x 5 00010 0 x 5 00011 0 00100 1 array window 00101 1 00110 1 00111 1 · · · · · · 01001 0 · · · · · · 01101 1 · · · · · · 11011 0 Input Output · · · · · · 11110 1 11111 0 SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 66

Local function of the contour detection operator X ψ ( X ) 00000 0 x 1 00001 0 x 2 x 3 x 4 x 1 x 2 x 3 x 4 x 5 00010 0 x 5 00011 0 00100 1 array window 00101 1 00110 1 00111 1 · · · · · · 01001 0 00000 · · · · · · 01101 1 · · · · · · 11011 0 Input Output · · · · · · 11110 1 11111 0 SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 67

Local function of the contour detection operator X ψ ( X ) 00000 0 x 1 00001 0 x 2 x 3 x 4 x 1 x 2 x 3 x 4 x 5 00010 0 x 5 00011 0 00100 1 array window 00101 1 00110 1 00111 1 · · · · · · 01001 0 · · · · · · 00011 01101 1 · · · · · · 11011 0 Input Output · · · · · · 11110 1 11111 0 SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 77

Local function of the contour detection operator X ψ ( X ) 00000 0 x 1 00001 0 x 2 x 3 x 4 x 1 x 2 x 3 x 4 x 5 00010 0 x 5 00011 0 00100 1 array window 00101 1 00110 1 00111 1 · · · · · · 01001 0 · · · · · · 01001 01101 1 · · · · · · 11011 0 Input Output · · · · · · 11110 1 and so on 11111 0 SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 86

Local function of the contour detection operator X ψ ( X ) 00000 0 x 1 00001 0 x 2 x 3 x 4 x 1 x 2 x 3 x 4 x 5 00010 0 x 5 00011 0 00100 1 array window 00101 1 00110 1 00111 1 · · · · · · 01001 0 · · · · · · 01101 1 · · · · · · 11011 0 Input Output · · · · · · 11110 1 11111 0 SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 87

Short break for another exercise Exercise 2 SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 88

Solution to exercise 2 – structuring element A B c A SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 89

Solution to exercise 2 – structuring element B c B c A SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 90

Solution to exercise 2 – Intersection B c A SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 91

Representation of an operator How to represent an image operator ? If Ψ is a W -operator it suffices to know ψ : { 0, 1 } W → { 0, 1 } . Kernel of Ψ K ( Ψ ) = { X ⊆ W : ψ ( X ) = 1 } • What is the importance of the kernel ? • Do we need to explicitly represent the kernel ? SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 92

Representation of an operator How to represent an image operator ? If Ψ is a W -operator it suffices to know ψ : { 0, 1 } W → { 0, 1 } . Kernel of Ψ K ( Ψ ) = { X ⊆ W : ψ ( X ) = 1 } Kernels are related to a canonical representation of W -operators • What is the importance of the kernel ? • Do we need to explicitly represent the kernel ? SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 93

Representation of operators Hit-miss in an important piece in the representation of W -operators A hit-miss operator is equivalent to an interval ⇒ X ∈ [ A , B ] X is detected by hit-miss operator H ( A , B c ) ⇐ B c W A B The window images detected by H ( A , B c ) or in [ A , B ] are the same: (all templates that correspond top contour points) SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 94

Decomposition theorem Example : intervals for the contour detection operator { } � , � � , � � , � � , � , , , Interval operator: Λ ( A , B ) ( X ) = 1 ⇐ ⇒ X ∈ [ A , B ] Basis of Ψ : B ( Ψ ) = maximal intervals in K ( Ψ ) Minimal decomposition theorem max ψ ( X ) = [ A , B ] ∈ B ( Ψ ) Λ ( A , B ) ( X ) SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 95

What would be the basis for the extreme point case ? SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 96

What would be the basis for the extreme point case ? left endpoint right endpoint bottom endpoint top endpoint SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 97

Summary of the fundamentals You acquired knowledge on – image operators – W -operators, a broad class of image operators – characterization by local functions – basis representation: supremum of interval operators – geometrical interpretation We should also mention : – binary case: equivalent to Boolean functions – Similar results hold for gray-scale image operators SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 98

Learning image operators SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 99

Goal of image operator learning Given observed images f and respective expected transformation g , we would like to find Ψ such that Ψ ( f ) is a good approximation of g MAE ( mean absolute error ) � � MAE � Ψ � = E | [ Ψ ( f )]( p ) − g ( p ) | Empirical MAE Err = 1 � � � � [ Ψ ( f )]( p ) − g ( p ) � � | E | � p ∈ E Average of pixel-wise absolute difference Binary images: pixel error rate SIBGRAPI 2016 Tutorials — Learning Image Operators and Applications (I.S. Montagner, N.S.T. Hirata, R. Hirata Jr.) 100

Image Operator Learning and Applications Igor S. Montagner Nina S. - PowerPoint PPT Presentation

SIBGRAPI 2016 TUTORIAL Image Operator Learning and Applications Igor S. Montagner Nina S. T. Hirata Roberto Hirata Jr. Department of Computer Science Institute of Mathematics and Statistics University of So Paulo (USP) October / 2016

BBM 413 Image filtering: computes a function of a local neighborhood at Fundamentals of each

Deep Learning in Image Processing Topics: Image Filtering 101 CNNs 101 Image

Electricity System Operator Incentives BSUoS Seminar Place your chosen image here. The four

Decomposition by operator-splitting methods and applications in image deblurring Daniel

0555R Review of the Market Operator (OCM) Provision Workgroup 3 Place your chosen image

Electricity System Operator Forward Plan Consultation Event Place your chosen image here. The

Image Captioning Image Captioning Image Captioning A survey of recent deep-learning approaches

Illustrating Agnostic Learning We want a classifier to distinguish between cats and dogs Image 1

WHAT DOES LEARNING LOOK LIKE? Image sources:

TOWARDS CREATING A KNOWLEDGE GAP FOR DEEP LEARNING BASED MEDICAL IMAGE ANALYSIS Dr. S.

Learning-Based Image/Video Coding Lu Yu Zhejiang University Outlines System architecture of

Learning-based Methods for Single Image Restoration and Translation He Zhang Adobe September

Image Processing Todays Class Image Representations: Matrices Image Representations: RGB,

Learning 3D Part Assembly from A Single Image Yichen Li* 1 , Kaichun Mo* 1 , Lin Shao 1 ,

2 2 f f + = 0 2 2 x y Laplacian operator is discretized version

Operator overloading (1 of 9) Operator overloading (2 of 9) Let us assume a class to handle

Comparison of orientated and spatially variant morphological filters vs mean/median filters for

Embeddings of the Heisenberg group, uniform rectifiability, and the Sparsest Cut problem Robert

Logistics and Program WIFI: salao-atos; password: salaoatos Restrooms Coffee and Lunch

P N umber T heory Olivier Ramar A dditive C ombinatorics

Spectral Clustering Diego Jimnez-Badillo diego_jimenez@inah.gob.mx National Institute of

ALGANT Michele Serra and Rosa Winter An introduction to the program June 26th, 2018 The ALGANT

Continuous solutions to a balance law L. Caravenna, OxPDE F. Bigolin, F. Serra Cassano

Power Grid Analysis Challenges for Large Microprocessor Designs Alexander Korobkov Contents