Duality in a maximum generalized entropy model Shinto Eguchi Osamu - - PowerPoint PPT Presentation

Duality in a maximum

Shinto Eguchi

MaxEnt, 2014 21-26 September 2014 Chateau Clos Luce, Amboise

generalized entropy model

Osamu Komori Atsumi Ohara

MaxEnt in ecology

Presence data obtained by an ecological study Log-entropy (Boltzmann-Gibbs-Shannon) Equal mean space

2

Maximum entropy distribution

Problem : find a distribution f to maximize H( f under the constraints: Solution is Gibbs distribution

3

MAP estimator

Loss function

• Cf. adaptive Lasso, Zou (2006, JASA)

4

Sequential L1-regularization (Phillip, Anderson, Schapire 2006)

GIS (Geographic Information System) GBIF (Global Biodiversity Inventory Facility) in R

5

http://www.seascapemodeling.org/

Habitat map of right whales

Area Under ROC Curve

6

Problems in MaxEnt

7

The duality between the model and estimation is not robust (model misspecification, over-fitting to data, cf. Bayes robust)

MaxEnt jointly suggests the statistical model and estimation, (exponential model and maximum likelihood) MaxEnt is useful in presence-only-data in ecological studies

U-entropy leads to extension from MaxEnt to MaxU-Ent (The duality is extended in functional degree of freedom)

Model selection and model validation have simple forms

Hill’s diversity numbers

(effective number of species) (1973)

8

richness Rare species are highly weighted exp (log-entropy) All species are fairly weighted Simpson index Dominant species are highly weighted

Several measures of entropy

Tsallis entropy (1988) Hill’s diversity (1973) Simpson (1949)

9

U-entropy (Eguchi, 2006)

Generalized entropy

10

Generator U-entropy Example Convex conjugate

log entropy Renyi-Tsallis entropy

Generalized divergence

11

U-cross entropy U-divergence

MaxU-Ent

Problem : find a distribution f to maximize HU ( f under the constraints:

12

13

Minimum divergence geometry

Riemannian metric Linear connections

14

Dually flat

Riemann metric Linear connections Conjugate convexity: U-model

15

Pythagorean theorem for U-divergence

Theorem m-geodesic U-geodesic

Max-Entropy model

MaxEnt and MinDiv

Duality of totally geodesic

17

MaxU-Ent

= MaxU-model + MinU-divergence

18

Project

19

Which -MaxEnt is the best ? Can we select the best  based on presence data?

Test AUC comparison may suggest better Max -Ent.

Hill’s diversity numbers associate with

• ne parameter family of MaxEnt methods, { Max -Ent :  in R }

Can the family improve the classical Maxent?

Folivora ( Three-toed sloth )

20

Max-Ent

21

MaxEnt)  

22

 MaxEnt)

Max-Ent

U-MaxEnt U-entropy U-Gibbs distribution U-estimator U-divergence U-MaxEnt U-entropy U-Gibbs distribution U-estimator U-divergence

A class of MaxU-Ent

23

MaxEnt Log-entropy Gibbs distribution KL-divergence MLE U-MaxEnt U-entropy U-Gibbs distribution U-estimator U-divergence MaxU-Ent U-entropy U-Gibbs distribution U-divergence MinU-estimator

Thank you!

24

Model validation

25

A brief tutorial on Maxent (Phillips, 2006)

model loss function U-loss exp-model U-model

( U-estimator , U-model)

• log-likelihood

MLE Robust estimator Robust model

26

U-estimate

MaxU-Ent

Problem : find a distribution f to maximize HU ( f under the constraints:

27

Related problems

28

Maxent is equivalent to Poisson point process model ( Renner-Warton, 2013, Loyle et all. 2012, Aarts et al. 2012, Fithian-Hastie, 2012) -Maxent is equivalent to -Poisson point process model?

-Poisson distribution

5 10 15 20 0.05 0.10 0.15 0.20 0.25 0.30 0.35

5 10 15 20 0.05 0.10 0.15 0.20 0.25 0.30 0.35 5 10 15 20 0.05 0.10 0.15 0.20 0.25 0.30 0.35

5 10 15 20 0.05 0.10 0.15 0.20 0.25 0.30 0.35

29

Habitat map

30

 = 0  = 0.5