Two Large Scale Structures Statistics of Natural Images and Models - PowerPoint PPT Presentation

Two Large Scale Structures

Statistics of Natural Images and Models David Mumford Jinggang Huang Division of Applied Math Division of Applied Math Box F, Brown University Box F, Brown University Space of Natural Images Providence, RI029 12 Providence, RI029 12 j huang@ David_Mumford@brown.edu cfm. brown .edu • Mumford Data Set (1999) Abstract ' natural im- Large calibrated datasets o 'random f ages have recently become available. These make possi- ble precise and intensive statistical studies of the local nature of images. We report results ranging from the simplest single pixel intensity to joint distribution of 3 Haar wavelet responses. Some of these statistics shed light on old issues such as the near scale-invariance of image statistics and some are entirely new. We fit mathematical models to some of the statistics and ex- plain others in terms of local image features. 1 Introduction There has been much attention recently to the statistics of natural images. For example, Ruder- man [7] discusses the approximate scale invariance property of natural images and Field [4] linked the Huang, Jinggang, and David Mumford. "Statistics of natural images and models." Proceedings. 1999 IEEE Computer Society design of the biological vision system to the statis- Conference on Computer Vision and Pattern Recognition (Cat. No PR00149). Vol. 1. IEEE, 1999. Figure 1: Four images from the data base tics of natural images. Zhu, Wu and Mumford [9] set up a general frame work for natural image mod- eling via exponential models. Simoncelli[l] uncovered intensity, and use statistics which do not contain the significant dependencies of wavelet coefficients in nat- constant (now an additive constant). We believe our ural image statistics. In most of these papers, sim- work here can serve as a solid starting point for fur- ple statistics are calculated from which some proper- ther image modeling and provide guidance in design ties are derived to prove some point. But little effort of image processing and image compression systems. has been made to systematically investigate the ex- We explain some symbols we will use in the paper: act statistics that underline natural images. Many of Assume X is a random variable on R, we use p and U ' these papers base their calculation on a small set of to represent the mean and variance of X . We define: images, casting doubt on how robust their results are. Also, because of the small sample sets, rare events E ( X - d 4 s = E ( X - PI3 (e.g. strong contrast edges) which are important vi- I C = U4 U3 sually may not show up frequently enough to stabilize where K is the kurtosis, S is the skewness. Assuming Y the corresponding statistics. We tried to overcome is another random variable on R, these problems by using a very large calibrated im- we denote the differ- ential entropy for X by X ( X ) , age data base (about 4000 1024 x 1536 images taken and denote the mutual information between X and Y by Z ( X , Y ) , by digital camera), provided by J.H. van Hateren (for both in details, see [5]). Figure 1 shows some sample images bits. We use differential entropy instead of discrete from this data base. These images measure light in entropy, because the variables are real valued. For de- tails, see [a]. All our pictures of probability distribu- the world up to an unknown multiplicative constant in each of the image. We will only work on the log tions (or of normalized histograms) will be shown with 1 54 IEEE 0-7695-0149-4/99 $10.00 0 1999

Space of Natural Images • 3x3 patches

Space of Natural Images • Most common patch — one colour H 1 � • Preprocessing 1. Remove constant patches 2. Normalize to norm 1 • Resulting structure: Klein Bottle Carlsson, Gunnar, et al. "On the local behavior of spaces of natural images." International journal of computer vision 76.1 (2008): 1-12. Adams, Henry, and Gunnar Carlsson. "On the nonlinear statistics of range image patches." SIAM Journal on Imaging Sciences 2.1 (2009): 110-117.

Space of Natural Images

Conformation Space of Cycleoctane Martin, Shawn, et al. "Topology of cyclo-octane energy landscape." The journal of chemical physics 132.23 (2010): 234115.

Conformation Space of Cycleoctane

Functionals on Persistence Diagrams

Rank Function 12 0 1 10 1 2 8 death 6 2 3 0 4 2 0 2 4 6 8 10 12 birth Bubenik, Peter. "Statistical topological data analysis using persistence landscapes." The Journal of Machine Learning Research 16.1 (2015): 77-102.

Persistence Landscapes 0 1 1 2 2 2 3 0 2 4 6 8 10 12 λ 1 2 λ 2 λ 3 0 2 4 6 8 10 12

Persistence Landscape 10 10 8 8 6 6 λ 1 4 4 λ 2 2 2 0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16 10 10 8 8 6 6 λ 1 λ 1 4 4 λ 2 λ 2 2 2 0 2 4 6 8 10 12 14 16 0 2 4 6 8 10 12 14 16

Persistence Images and Kernels Persistence diagrams Surface meshes D 1 Kernel SVM Task(s) : shape classification/retrieval k ( D 1 , D 1 ) k ( D 1 , D N )   · · · Kernel PCA (Surface meshes filtered by heat-kernel signature) . . ...   K = . .   . . D 2 Persistent   homology k ( D N , D 1 ) k ( D N , D N ) · · · Gaussian processes Images Kernel construction (our contribution) Task(s) : texture recognition D N (image data as weighted cubical cell complex) Reininghaus, Jan, et al. "A stable multi-scale kernel for topological machine learning." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015 surface diagram T ( B ) diagram B image persistence death birth birth Adams, Henry, et al. "Persistence images: A stable vector representation of persistent homology." The Journal of Machine Learning Research 18.1 (2017): 218-252.

Some Other Applications

Euler Calculus/Integration • Euler characteristic is a generalised measure • Developed by Schapira, Kashiwara, Viro • Counting trajectories t Baryshnikov, Yuliy, and Robert Ghrist. "Target enumeration via Euler characteristic integrals.” SIAM Journal on Applied Mathematics 70.3 (2009): 825-844.

Euler Calculus/Integration 1 0 0 1 0 1 2 2 1 2 1 1 2 3 0 4 2 1 4 0 3 1 2 1 3 3 2 1 2 2 2 3 0 1 2 2 1 1 2 0 1 2 1 0 0 χ { h > 3 } = 2 χ { h > 2 } = 3 χ { h > 1 } = 3 χ { h > 0 } = − 1

Mapper Singh, Gurjeet, Facundo Mémoli, and Gunnar Carlsson. "Mapper: a topological mapping tool for point cloud data." Eurographics symposium on point-based graphics. Vol. 102. 1991.

Mapper Wired: Analytics Reveal 13 New Basketball Positions (https://www.wired.com/2012/04/analytics-basketball/)

Mapper ER-- sequence 9.9E+5 3.2E+6 4.7E+6 7.0E+6 9.4E+6 1.2E+7 1.5E+7 FILTER COLOR SCALE Basal-like sparse data sparse data sparse data sparse data c-MYB+ tumors detached tumor bins very sparse data ER+ sequence Normal-Like & Normal Nicolau, Monica, Arnold J. Levine, and Gunnar Carlsson. "Topology based data analysis identifies a subgroup of breast cancers with a unique mutational profile and excellent survival." Proceedings of the National Academy of Sciences 108.17 (2011): 7265-7270.

Cohomological Coordinates Circle-valued functions: X Fundamental equation X X Z This gives an intrinsic parameterization of a circle De Silva, Vin, Dmitriy Morozov, and Mikael Vejdemo-Johansson. "Persistent cohomology and circular coordinates." Discrete & Computational Geometry 45.4 (2011): 737-759.

Cohomological Coordinates Recurrent Behaviour

Cohomological Coordinates see https://www.youtube.com/watch?v=NGQ-M2gdibQ Vejdemo-Johansson, Mikael, et al. "Cohomological learning of periodic motion." Applicable Algebra in Engineering, Communication and Computing 26.1-2 (2015): 5-26.

Cohomological Coordinates

Cohomological Coordinates

Cohomological Coordinates

Cohomological Coordinates

Cohomological Coordinates

Cohomological Coordinates