Image Understanding with Organic Computing Rolf P. W urtz - PowerPoint PPT Presentation

Neuroinformatik / Ruhr-Universit¨ at Bochum Rolf P . W¨ urtz Image Understanding with Organic Computing Rolf P. W¨ urtz Ruhr-Universit¨ at Bochum Institut f¨ ur Neuroinformatik http://www.neuroinformatik.rub.de rolf.wuertz@neuroinformatik.rub.de rolf.wuertz@organic-computing.org Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Overview Rolf P . W¨ urtz • Introduction • Problem of image understanding • Controlled generalization in face recognition • General object recognition • Learning of articulated models • Where to go from here Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Complexity problems Rolf P . W¨ urtz • Artificial systems are rapidly getting too complex to understand • Desirable are complex systems with trivial interfaces • This requires restrictions on possible behaviors • This asks for self-organization Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Image understanding . . . Rolf P . W¨ urtz . . . means establishing a symbolic description. Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Image understanding . . . Rolf P . W¨ urtz . . . in the brain is done under additional assumptions. Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Image understanding . . . Rolf P . W¨ urtz . . . requires extensive world knowledge. Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Image understanding . . . Rolf P . W¨ urtz . . . has a local-global problem. Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Face recognition Rolf P . W¨ urtz = � = Different situations yield very different images. Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Invariance problem Rolf P . W¨ urtz All the same? Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Invariance . . . Rolf P . W¨ urtz 6 6 � = . . . is task-dependent. Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum List of tasks Rolf P . W¨ urtz • We do need a formal model of images, but we don’t have any • We do not even have a formalization of the problem • Required is an imitation of human capability • Identify constraints of visual data autonomously • Learn computer vision routines from examples • Start a positive feedback loop of learning vision • Control generalization Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Controlled generalization Rolf P . W¨ urtz • Neural networks learn complicated functions from examples • They can generalize, but not always in the desired way • Visual invariances must be built in explicitly (Neocognitron, Convolutional NN, . . . ) • Exception: Slow feature analysis (Wiskott & Sejnowski, 2002) • Goal: Learn generalization dimensions from examples! Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Bunch Graphs Rolf P . W¨ urtz bunch of jets general face jet c: bunch graph a: Gabor wavelet b: Wiskott et al., 1997 Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Graph similarity Rolf P . W¨ urtz • Jet similarity function S J • Probe graph P with N nodes P n • Gallery graphs G g with N nodes G g,n each N 1 � g rec = arg max S J ( P n , G g,n ) . N g n =1 Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Face Graphs Rolf P . W¨ urtz Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Pose Variation Rolf P . W¨ urtz PM + 45 PM + 00 PM − 45 Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Pose Variation Rolf P . W¨ urtz Probe Gallery Similarity? G g P . . . . . . Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Rank Correlation Rolf P . W¨ urtz Probe Model Gallery S P S G π = [7 , 3 , 9 , . . . ] γ = [7 , 9 , 3 , . . . ] . . . . . . . . . . . . γ = [2 , 1 , 35 , . . . ] Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Neural rank list similarity Rolf P . W¨ urtz I K � − order ( a j ) � � E = exp w j λ j =1 w j = 1 � − order ( b j ) � K exp λ a j E w j Thorpe et al., 2001 Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Neural rank list similarity Rolf P . W¨ urtz A g w m,g 1 � − γ g ( m ) � w m,g = exp N M λ π ( m ) � − π ( m ) � � A g = exp w m,g λ m � � 1 − π ( m ) + γ g ( m ) � = exp N M λ m = S neural ( γ g , π ) 1 � g rec = arg max S neural ( γ g,n , π n ) N g n M¨ uller and W¨ urtz, ICANN 2009 Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Face recognition Mainz Hbf. Rolf P . W¨ urtz Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum CAS-PEAL Database Rolf P . W¨ urtz PM + 45 FM + 00 FM − 45 FM − 90 PM + 00 FD + 00 FD − 45 FD − 90 PM − 45 FU + 00 FU − 45 FU − 90 Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Rank Correlation: Results Rolf P . W¨ urtz Pose Illumination Recognition percentage with given situation 99.02 89.01 Percentage of correct situation estimation 99.89 ± 0.09 91.96 ± 0.89 Recognition percentage with automatically 97.75 ± 0.50 89.97 ± 1.36 determined situation Best recognition percentage reported in 71 51 database description Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Rank Correlation: Early stopping Rolf P . W¨ urtz 1 0.8 0.6 Recognition rate pose 0.4 illumination 0.2 0 0 20 40 60 80 100 Spike number Image Understanding with Organic Computing Hannover, 2010-02-23

Neuroinformatik / Ruhr-Universit¨ at Bochum Thanks to Rolf P . W¨ urtz Marco M¨ uller Rank correlation memory G¨ unter Westphal Object recognition Thomas Walther Body tracking Manuel G¨ unter Statistical face recognition Markus Lessmann Scene analysis Oliver Lomp Neuronal dynamics Mathis Richter Clustering of image patches Guillermo Donatti Object memory, Neural Map DFG, EU, NRW Funding All of you Attention Image Understanding with Organic Computing Hannover, 2010-02-23