A Mann-Whitney spatial scan statistic for continuous data Lionel - PowerPoint PPT Presentation

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Data-based potential clusters ➨ Connected component of the edge i in G ( δ ) : N i ( δ ). ➨ A i ( δ ) = { x ∈ A : ∃ j ∈ N i ( δ ) , d ( x , x j ) ≤ δ } .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Data-based potential clusters ➨ Connected component of the edge i in G ( δ ) : N i ( δ ). ➨ A i ( δ ) = { x ∈ A : ∃ j ∈ N i ( δ ) , d ( x , x j ) ≤ δ } . ➨ Potential clusters : C= { A i ( δ ) : 1 ≤ i ≤ n , δ ∈ R + } .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Data-based potential clusters ➨ Connected component of the edge i in G ( δ ) : N i ( δ ). ➨ A i ( δ ) = { x ∈ A : ∃ j ∈ N i ( δ ) , d ( x , x j ) ≤ δ } . ➨ Potential clusters : C= { A i ( δ ) : 1 ≤ i ≤ n , δ ∈ R + } . ➨ Only n − 1 areas, arbitrarily shaped.

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Outline Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Outline Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Concentration indices

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Concentration indices We evaluate the concentration in Z ⊂ A .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Concentration indices We evaluate the concentration in Z ⊂ A . ➨ n ( Z ) = ♯ { i : X i ∈ Z } .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Concentration indices We evaluate the concentration in Z ⊂ A . ➨ n ( Z ) = ♯ { i : X i ∈ Z } . 1 � ➨ µ ( Z ) = i : X i ∈ Z C i . n ( Z )

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Concentration indices We evaluate the concentration in Z ⊂ A . ➨ n ( Z ) = ♯ { i : X i ∈ Z } . 1 � ➨ µ ( Z ) = i : X i ∈ Z C i . n ( Z ) ➨ σ ( Z ) 2 = � 2 . 1 � � C i − µ ( Z ) i : X i ∈ Z n ( Z )

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices We test the presence of a cluster in Z ⊂ A .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices We test the presence of a cluster in Z ⊂ A . ➨ H 0 : C i ∼ f 0 .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices We test the presence of a cluster in Z ⊂ A . ➨ H 0 : C i ∼ f 0 . 1( X i ∈ ¯ ➨ H 1 , Z : C i | X i ∼ f Z 1 1( X i ∈ Z ) + f ¯ 1 Z ). Z

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices We test the presence of a cluster in Z ⊂ A . ➨ H 0 : C i ∼ f 0 . 1( X i ∈ ¯ ➨ H 1 , Z : C i | X i ∼ f Z 1 1( X i ∈ Z ) + f ¯ 1 Z ). Z Likelihood ratio : I ( Z ) = L 1 , Z ( X 1 , · · · , X n , C 1 , · · · , C n ) µ ( Z ) > µ (¯ � � 1 1 Z ) . L 0 ( X 1 , · · · , X n , C 1 , · · · , C n )

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The Exponential model

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The Exponential model � � ➨ H 0 : C i ∼ E 1 /µ ( A ) .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The Exponential model � � ➨ H 0 : C i ∼ E 1 /µ ( A ) . 1 /µ (¯ 1( X i ∈ ¯ � � � ➨ H 1 , Z : C i | X i ∼ E (1 /µ ( Z ) 1 1( X i ∈ Z ) + E Z ) 1 Z ).

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The Exponential model � � ➨ H 0 : C i ∼ E 1 /µ ( A ) . 1 /µ (¯ 1( X i ∈ ¯ � � � ➨ H 1 , Z : C i | X i ∼ E (1 /µ ( Z ) 1 1( X i ∈ Z ) + E Z ) 1 Z ). Likelihood ratio : − n (¯ µ (¯ � � � � I exp ( Z ) = − n ( Z ) log µ ( Z ) Z ) log Z ) .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The homoscedastic Gaussian model

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The homoscedastic Gaussian model � µ ( A ) , σ ( A ) 2 � ➨ H 0 : C i ∼ N .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The homoscedastic Gaussian model � µ ( A ) , σ ( A ) 2 � ➨ H 0 : C i ∼ N . µ ( Z ) , σ 2 � � ➨ H 1 , Z : C i | X i ∼ N 1 1( X i ∈ Z ) 1 , Z µ (¯ 1( X i ∈ ¯ � Z ) , σ 2 � + N 1 Z ) 1 , Z

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The homoscedastic Gaussian model � µ ( A ) , σ ( A ) 2 � ➨ H 0 : C i ∼ N . µ ( Z ) , σ 2 � � ➨ H 1 , Z : C i | X i ∼ N 1 1( X i ∈ Z ) 1 , Z µ (¯ 1( X i ∈ ¯ � Z ) , σ 2 � + N 1 Z ) 1 , Z 1 , Z = n ( Z ) σ ( Z ) 2 + n (¯ Z ) σ (¯ Z ) 2 where σ 2 . n

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The homoscedastic Gaussian model � µ ( A ) , σ ( A ) 2 � ➨ H 0 : C i ∼ N . µ ( Z ) , σ 2 � � ➨ H 1 , Z : C i | X i ∼ N 1 1( X i ∈ Z ) 1 , Z µ (¯ 1( X i ∈ ¯ � Z ) , σ 2 � + N 1 Z ) 1 , Z 1 , Z = n ( Z ) σ ( Z ) 2 + n (¯ Z ) σ (¯ Z ) 2 where σ 2 . n Likelihood ratio : 1 I homgau ( Z ) = . σ 2 1 , Z

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The heteroscedastic Gaussian model

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The heteroscedastic Gaussian model � µ ( A ) , σ ( A ) 2 � ➨ H 0 : C i ∼ N .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The heteroscedastic Gaussian model � µ ( A ) , σ ( A ) 2 � ➨ H 0 : C i ∼ N . µ ( Z ) , σ ( Z ) 2 � � ➨ H 1 , Z : C i | X i ∼ N 1 1( X i ∈ Z ) µ (¯ Z ) , σ (¯ 1( X i ∈ ¯ Z ) 2 � � + N 1 Z ) .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Likelihood-based indices The heteroscedastic Gaussian model � µ ( A ) , σ ( A ) 2 � ➨ H 0 : C i ∼ N . µ ( Z ) , σ ( Z ) 2 � � ➨ H 1 , Z : C i | X i ∼ N 1 1( X i ∈ Z ) µ (¯ Z ) , σ (¯ 1( X i ∈ ¯ Z ) 2 � � + N 1 Z ) . Likelihood ratio : σ ( Z ) 2 � − n (¯ σ (¯ Z ) 2 � � � I hetgau ( Z ) = − n ( Z ) log Z ) log .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results No distribution assumption : Mann-Whitney test

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results No distribution assumption : Mann-Whitney test ➨ R j is the rank of C j among the C i , 1 ≤ i ≤ n .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results No distribution assumption : Mann-Whitney test ➨ R j is the rank of C j among the C i , 1 ≤ i ≤ n . ➨ RS ( Z ) = � i : X i ∈ Z R i .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results No distribution assumption : Mann-Whitney test ➨ R j is the rank of C j among the C i , 1 ≤ i ≤ n . ➨ RS ( Z ) = � i : X i ∈ Z R i . = M ( Z ) = n ( Z )( n +1) � � ➨ Under H 0 , E RS ( Z ) . 2

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results No distribution assumption : Mann-Whitney test ➨ R j is the rank of C j among the C i , 1 ≤ i ≤ n . ➨ RS ( Z ) = � i : X i ∈ Z R i . = M ( Z ) = n ( Z )( n +1) � � ➨ Under H 0 , E RS ( Z ) . 2 = V ( Z ) = n ( Z ) n (¯ Z )( n +1) � � ➨ Under H 0 , Var RS ( Z ) 12

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results No distribution assumption : Mann-Whitney test ➨ R j is the rank of C j among the C i , 1 ≤ i ≤ n . ➨ RS ( Z ) = � i : X i ∈ Z R i . = M ( Z ) = n ( Z )( n +1) � � ➨ Under H 0 , E RS ( Z ) . 2 = V ( Z ) = n ( Z ) n (¯ Z )( n +1) � � ➨ Under H 0 , Var RS ( Z ) 12 ➨ Under H 0 , RS ( Z ) − M ( Z ) √ − → N (0 , 1). V ( Z ) d

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results No distribution assumption : Mann-Whitney test ➨ R j is the rank of C j among the C i , 1 ≤ i ≤ n . ➨ RS ( Z ) = � i : X i ∈ Z R i . = M ( Z ) = n ( Z )( n +1) � � ➨ Under H 0 , E RS ( Z ) . 2 = V ( Z ) = n ( Z ) n (¯ Z )( n +1) � � ➨ Under H 0 , Var RS ( Z ) 12 ➨ Under H 0 , RS ( Z ) − M ( Z ) √ − → N (0 , 1). V ( Z ) d I rank ( Z ) = RS ( Z ) − M ( Z ) . � V ( Z )

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Outline Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Outline Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results The farms data set

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results The farms data set 26000 24000 lat[1:339] 22000 20000 18000 2000 4000 6000 8000 10000 long[1:339]

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results The farms data set

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results The farms data set 26000 24000 lat[1:339] 22000 20000 18000 2000 4000 6000 8000 10000 long[1:339]

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Astronomical data

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Astronomical data Observation of a cubic part of the Universe : (20 Mpc) 3 .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Astronomical data Observation of a cubic part of the Universe : (20 Mpc) 3 . 1 Mpc = 3 × 10 22 metres .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Astronomical data Observation of a cubic part of the Universe : (20 Mpc) 3 . 1 Mpc = 3 × 10 22 metres . ➨ Locations of the galaxies : X 1 , · · · , X n .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Astronomical data Observation of a cubic part of the Universe : (20 Mpc) 3 . 1 Mpc = 3 × 10 22 metres . ➨ Locations of the galaxies : X 1 , · · · , X n . ➨ Light intensities : C 1 , · · · , C n .

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Astronomical data Observation of a cubic part of the Universe : (20 Mpc) 3 . 1 Mpc = 3 × 10 22 metres . ➨ Locations of the galaxies : X 1 , · · · , X n . ➨ Light intensities : C 1 , · · · , C n . Goal : detecting areas where galaxies are ”redder”.

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Light intensities

Introduction 1-Potential clusters 2-A Mann-Whitney concentration index 3-Results Light intensities 150 100 Frequency 50 0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Color