MinNorm approximation of MaxEnt/MinDiv problems for probability - - PowerPoint PPT Presentation

▶
minnorm approximation of maxent mindiv problems for
SMART_READER_LITE
LIVE PREVIEW

MinNorm approximation of MaxEnt/MinDiv problems for probability - - PowerPoint PPT Presentation

Feb 19, 2023 188 likes •792 views

MinNorm approximation of MaxEnt/MinDiv problems for probability tables Patrick Bogaert and Sarah Gengler UCL Rebuilding probability tables UCL Rebuilding probability tables Limited number of samples Poor estimates UCL Rebuilding

slide-1
SLIDE 1

UCL

MinNorm approximation of MaxEnt/MinDiv problems for probability tables

Patrick Bogaert and Sarah Gengler

slide-2
SLIDE 2

UCL

Rebuilding probability tables

slide-3
SLIDE 3

UCL

Rebuilding probability tables

  • Limited number of samples

Poor estimates 

slide-4
SLIDE 4

UCL

Rebuilding probability tables

  • Limited number of samples

Poor estimates 

  • How to integrate experts opinion ?
slide-5
SLIDE 5

UCL

Rebuilding probability tables

  • Limited number of samples

Poor estimates  Rewriting information as equality / inequality constraints 

  • How to integrate experts opinion ?
slide-6
SLIDE 6

UCL

Rebuilding probability tables

  • Limited number of samples

Poor estimates  Rewriting information as equality / inequality constraints 

  • How to integrate experts opinion ?
slide-7
SLIDE 7

UCL

Rebuilding probability tables

  • Limited number of samples

Poor estimates  Rewriting information as equality / inequality constraints 

  • Equality constraints MaxEnt

  • Inequality constraints Minimum divergence (MinDiv)

  • How to integrate experts opinion ?
slide-8
SLIDE 8

UCL

Rebuilding probability tables

  • Limited number of samples

Poor estimates  Rewriting information as equality / inequality constraints 

  • Equality constraints MaxEnt

  • Inequality constraints Minimum divergence (MinDiv)

 Need for an efficient methodology to rebuild probability tables from both equality and inequality constraints 

  • How to integrate experts opinion ?
slide-9
SLIDE 9

UCL

The MaxEnt problem

slide-10
SLIDE 10

UCL

The MaxEnt problem

  • Equality constraints
slide-11
SLIDE 11

UCL

The MaxEnt problem

  • Equality constraints
  • Entropy maximized subject to the equality constraints
slide-12
SLIDE 12

UCL

The MaxEnt problem

  • Equality constraints
  • Entropy maximized subject to the equality constraints
slide-13
SLIDE 13

UCL

The MaxEnt problem

  • Equality constraints
  • Entropy maximized subject to the equality constraints

Sequence of MinNorm problems for solving the MaxEnt problem 

slide-14
SLIDE 14

UCL

MinNorm as an approximation of MaxEnt

slide-15
SLIDE 15

UCL

MinNorm as an approximation of MaxEnt

  • Taylor series of ln pi around pi = ki
slide-16
SLIDE 16

UCL

MinNorm as an approximation of MaxEnt

  • Taylor series of ln pi around pi = ki
  • Truncating at degree one and summing over i
slide-17
SLIDE 17

UCL

MinNorm as an approximation of MaxEnt

  • Taylor series of ln pi around pi = ki
  • Truncating at degree one and summing over i
  • In particular, if ki = 1/n
slide-18
SLIDE 18

UCL

MinNorm as an approximation of MaxEnt

  • For any other choice of the ki ‘s, by completing the square
slide-19
SLIDE 19

UCL

MinNorm as an approximation of MaxEnt

  • For any other choice of the ki ‘s, by completing the square
  • Summing over i

Where

slide-20
SLIDE 20

UCL

The MinDiv problem

slide-21
SLIDE 21

UCL

The MinDiv problem

  • Divergence or Kullback-Leibler distance
slide-22
SLIDE 22

UCL

The MinDiv problem

  • Equality constraints
  • Divergence or Kullback-Leibler distance

= 0 Maximizing

slide-23
SLIDE 23

UCL

The MinDiv problem

  • Equality constraints

  • Divergence or Kullback-Leibler distance

= 0 Maximizing

slide-24
SLIDE 24

UCL

The MinDiv problem

  • Equality constraints

 

  • Divergence or Kullback-Leibler distance

= 0 Maximizing

slide-25
SLIDE 25

UCL

The MinDiv problem

  • Equality constraints

 Sequence of MinNorm problems for solving the MinDiv problem  

  • Divergence or Kullback-Leibler distance

= 0 Maximizing

slide-26
SLIDE 26

UCL

The MinDiv problem

  • Equality constraints

 Sequence of MinNorm problems for solving the MinDiv problem  

  • Divergence or Kullback-Leibler distance

Both Equality and Inequality constraints can be processed together by MinNorm approximations = 0 Maximizing

slide-27
SLIDE 27

UCL

MinNorm as an approximation of MinDiv

slide-28
SLIDE 28

UCL

MinNorm as an approximation of MinDiv

  • Taylor series around pi = ki and completing the square
slide-29
SLIDE 29

UCL

MinNorm as an approximation of MinDiv

  • Taylor series around pi = ki and completing the square
  • Summing over i

Where

slide-30
SLIDE 30

UCL

Application in drainage classes mapping

slide-31
SLIDE 31

UCL

Application in drainage classes mapping

  • Categorical data are found in a wide variety of applications
slide-32
SLIDE 32

UCL

Application in drainage classes mapping

slide-33
SLIDE 33

UCL

Application in drainage classes mapping

  • Categorical data are found in a wide variety of applications
  • 90 % of variables collected in soil surveys are categorical
  • Soil drainage, an important criterion in rating soils for various uses
slide-34
SLIDE 34

UCL

Application in drainage classes mapping

slide-35
SLIDE 35

UCL

Application in drainage classes mapping

slide-36
SLIDE 36

UCL

Application in drainage classes mapping

slide-37
SLIDE 37

UCL

Application in drainage classes mapping

slide-38
SLIDE 38

UCL

Application in drainage classes mapping

slide-39
SLIDE 39

UCL

Application in drainage classes mapping

slide-40
SLIDE 40

UCL

Application in drainage classes mapping

slide-41
SLIDE 41

UCL

Application in drainage classes mapping

slide-42
SLIDE 42

UCL

Integrating the lithological map : 4 cases

slide-43
SLIDE 43

UCL

Integrating the lithological map : 4 cases

slide-44
SLIDE 44

UCL

Integrating the lithological map : 4 cases

slide-45
SLIDE 45

UCL

Spatial prediction

slide-46
SLIDE 46

UCL

Integrating the lithological map : 4 cases

slide-47
SLIDE 47

UCL

Spatial prediction

slide-48
SLIDE 48

UCL

Integrating the lithological map : 4 cases

slide-49
SLIDE 49

UCL

Conclusions

slide-50
SLIDE 50

UCL

Conclusions

slide-51
SLIDE 51

UCL

Conclusions

 MinNorm Approximations

slide-52
SLIDE 52

UCL

Conclusions

 MinNorm Approximations

slide-53
SLIDE 53

UCL

Conclusions

 MinNorm Approximations

slide-54
SLIDE 54

UCL

Conclusions

 MinNorm Approximations

slide-55
SLIDE 55

UCL

Conclusions

 MinNorm Approximations

slide-56
SLIDE 56

UCL

Conclusions

 MinNorm Approximations

slide-57
SLIDE 57

UCL

Conclusions

 MinNorm Approximations

slide-58
SLIDE 58

UCL

Thank you for your attention

slide-59
SLIDE 59

UCL

References