Interactive Image Mining Annie Morin 1 , Nguyen-Khang Pham 1,2 1 - - PowerPoint PPT Presentation

▶
interactive image mining
SMART_READER_LITE
LIVE PREVIEW

Interactive Image Mining Annie Morin 1 , Nguyen-Khang Pham 1,2 1 - - PowerPoint PPT Presentation

Apr 11, 2023 193 likes •581 views

Interactive Image Mining Annie Morin 1 , Nguyen-Khang Pham 1,2 1 TEXMEX/IRISA 2 Cantho University, Vietnam Outline Image retrieval and motivation Examples Image topic discovery FCA and adaptation on images Image retrieval

slide-1
SLIDE 1

Interactive Image Mining Annie Morin1 , Nguyen-Khang Pham1,2

1 TEXMEX/IRISA 2 Cantho University, Vietnam

slide-2
SLIDE 2

CARME 2011 2

Outline

Image retrieval and motivation Examples Image topic discovery

FCA and adaptation on images

Image retrieval Image indexing using indicators of FCA

Numerical results

Conclusion and future works

slide-3
SLIDE 3

CARME 2011 3

Image retrieval

Image database Sorted list of images Image query Decreasing similarity Image retrieval System

slide-4
SLIDE 4

CARME 2011 4

Starting Point: Video Google

  • Interest point

detection

  • SIFT

computation

K-means clustering visual words SIFT SIFT ∈ R128

Quantification

Hessian - Affine detector [Mikolaczyk & Schmid, 2004] SIFT computation [Lowe, 2004] Video Google [Sivic & Zisserman, 2003] Bag of “visual words”

slide-5
SLIDE 5

CARME 2011 5

SIFT descriptor

SIFT = Scalable Invariant Feature Transform 4 x 4 x 8 (directions) = 128 (dimensions) ∈ R128

slide-6
SLIDE 6

CARME 2011 6

Visual words

6

…

R128 R128 SIFT

Visual words

Clustering

(for inst.., k-means)

slide-7
SLIDE 7

CARME 2011 7

Contingency table

word 1 word 2 word 3 … word N image 1 image 2 … image M

slide-8
SLIDE 8

CARME 2011 8

Data Analysis Techniques Used

Video Google

tf*idf weighting Probabilistic Latent Semantic Analysis (Sivic et al.,

2005)

Our goal

Replace these techniques

by Factorial Correspondence Analysis (FCA)

slide-9
SLIDE 9

CARME 2011 9

Motivation

Success of application of Factorial Correspondence Analysis on textual data

slide-10
SLIDE 10

CARME 2011 10

Some examples : the Caltech4 database

Caltech4 database

4090 images divided into 5 categories

faces :

435

motorbikes :

800

airplanes :

800

cars (rear):

1155

backgrounds :

900

Vocabulary : 2224 visual words (Sivic et al., 2005)

slide-11
SLIDE 11

CARME 2011 11

Caltech4 dataset

slide-12
SLIDE 12

CARME 2011 12

Factorial Correspondence Analysis

12

slide-13
SLIDE 13

CARME 2011 13

Factorial Correspondence analysis

Image with all its visual words Characteristic visual words for this image in this plane visual words images

slide-14
SLIDE 14

CARME 2011 14

Cars

slide-15
SLIDE 15

CARME 2011 15

Motorbikes

slide-16
SLIDE 16

CARME 2011 16

Faces

slide-17
SLIDE 17

CARME 2011 17

Airplanes

slide-18
SLIDE 18

CARME 2011 18

Backgrounds

slide-19
SLIDE 19

CARME 2011 19

Image information extraction

Red:neg Blue:pos

slide-20
SLIDE 20

CARME 2011 20

Example : Categorization of images Alogic database (961 images, 1000 visual words)

slide-21
SLIDE 21

CARME 2011 21

Categorization of images

slide-22
SLIDE 22

CARME 2011 22

Numerical results: databases

Caltech4 database

4090 images divided into 5 categories Vocabulary : 2224 visual words

Nister database

2250 groups of 4 images 10200 images in total 5000 visual words

slide-23
SLIDE 23

CARME 2011 23

Nistér dataset

slide-24
SLIDE 24

CARME 2011 24

Experimental results : methods comparison

slide-25
SLIDE 25

CARME 2011 25

Experimental results

Nistér-Stewénius

Exhaustive search

slide-26
SLIDE 26

CARME 2011 26

Speeding up the retrieval by using image categorization

One factorial axis two different topics One image may belong to several topics Topics determination:

Contribution to the inertia of an axis Quality on an axis

slide-27
SLIDE 27

CARME 2011 27

Image Indexing using FCA

Hypothesis:

Two similar images share some common

properties

Indicators of FCA have relevant properties

Representation quality of images on axes

An image is well represented on some axes

Contribution to the inertia of axes

An image highly contributes to the inertia of some

axes

slide-28
SLIDE 28

CARME 2011 28

Image Indexing using FCA

Inverted file property

Contains images which possess this property

For an axis we construct two inverted files:

Negative part/Positive part Each file contains images well represented on this axis (or

images which highly contribute to the inertia of this axis)

slide-29
SLIDE 29

CARME 2011 29

Image Indexing using FCA

Retrieval schema

For a given image query

Sort axes by their representation quality (contribution to the

inertia)

Determine relevant properties of the query (i.e. some first

axes) and take the correspondent inverted files

Merge the inverted files by majority vote candidate list Search in the candidate list

slide-30
SLIDE 30

CARME 2011 30

Inverted files

1 1 1 1 1 image 1 image 2 image 4 image 5 1 image 3 F1

+

F1

  • F2

+

F2

  • Topics

determination

…

Inverted files

Coordinates of the images in the factorial space 1 4 5 4 1

…

3 2 2 Topics axis α

α i

z

1 image i 1 1 5

slide-31
SLIDE 31

CARME 2011 31

Experimental results

Approximative search with inverted files

Method

1:

Inverted files based on the contribution

Method

2:

Inverted files based on the quality of representation

Exhaustive search

slide-32
SLIDE 32

CARME 2011 32

FCA with indexing technique

Precision and time results

Methods #images 5 img 10 img 50 img 100 img Times (ms) FCA, exhausted search 4090 95.92 94.61 91.35 89.64 0.50 Representation quality-based indexing 984 95.97 94.63 91.23 89.43 0.15 Contribution- based indexing 791 95.85 94.47 91.01 88.93 0.13 Tf*IDF, exhausted search 4090 88.24 84.81 77.52 73.72 2.94

Caltech4 dataset

  • Number of axes kept: 15
  • #images: size of candidate list
  • 3.5 times faster than FCA without indexing
  • 20 times faster than TF*IDF
slide-33
SLIDE 33

CARME 2011 33

FCA with indexing technique

Methods #images precision Times (ms) FCA, exhausted search 10200 79.82 12.01 FCA with indexing nf= 21, nf_thres = 11 650 79.75 1.04 FCA with indexing nf: auto, nf_thres: auto 397 79.96 1.33 TF*IDF, exhausted search 10200 73.04 36.45

Nistér dataset

#image: size of candidate size precision: precision at 4 first returned images

  • 10 times faster than FCA without indexing
  • 30 times faster than TF*IDF
slide-34
SLIDE 34

CARME 2011 34

Results

34

Methods Acceleration gain min max mean Inverted files based on the contribution 2.7 17.3 6.9 Inverted files based on the quality of représentation 3.2 13.8 6.7

slide-35
SLIDE 35

CARME 2011 35

Parallelization of the filtering step

Filtering of non

relevant images

Use inverted files to

filter the images sharing few topics with the query

Refinement step

Sequential search

among the images candidates

Most of the time is for filtering

Computation time distribution for filtering step and refinement step for 1 million images

slide-36
SLIDE 36

CARME 2011 36

Results with Nister-Stevenius database +1 million images from FlickR

Method with a parallel filtering on GPU

  • 10 times faster than the method without parallelization of the

filtering step

  • 100 times faster than the exhaustive search

Methods P@3 Response time (ms) Acceleration gain Exhaustive 0.623 860.88

  • Filtering on

CPU 0.625 79.99 10.76 Filtering on GPU 0.625 7.53 114.33

slide-37
SLIDE 37

CARME 2011 37

Conclusion

Adaptation of FCA on images for indexing

and retrieval

Hierarchical CA to focus on some areas of a

factorial plane

slide-38
SLIDE 38

CARME 2011 38

Work of J.B Sivic and A Zisserman using

probabilistic latent semantic analysis and visual words in images