The Software Package A NTS InFields A NTS InFields is a Software - - PowerPoint PPT Presentation

▶
the software package a nts infields a nts infields is a
SMART_READER_LITE
LIVE PREVIEW

The Software Package A NTS InFields A NTS InFields is a Software - - PowerPoint PPT Presentation

Jul 01, 2023 304 likes •588 views

National Research Center Institute of Biomathematics for Environment & Health & Biometry Mathematical Modelling in Ecology and the Biosciences Stochastic Simulation and Bayesian Inference for Gibbs Fields: The Software-Package A NTS

slide-1
SLIDE 1

◭

◮

National Research Center for Environment & Health Institute of Biomathematics & Biometry

Mathematical Modelling in Ecology and the Biosciences

Stochastic Simulation and Bayesian Inference for Gibbs Fields: The Software-Package ANTSInFields

Felix Friedrich Leipzig, 6.12.2002

slide-2
SLIDE 2

◭

◮

The Software Package ANTSInFields ANTSInFields is a Software Package for Simulation and Statistical Inference

  • n Gibbs Fields. It is intended for mainly two purposes: To support tea-

ching by demonstrating well known sampling and estimation techniques and for assistance in research.

ANTSInFieldsis available for download from http://www.antsinfields.de contact: friedrich@antsinfields.de

slide-3
SLIDE 3

◭

◮

History 1995–1998: Development of Voyager Portable and extensible System for simulation and data analysis 1998: Diploma thesis Parameter estimation on Gibbs fields in the context of statistical Image Analysis Need for implementation of

  • samplers for various Gibbs Fields (Ising Model and extensions)
  • parameter estimators on simulated or external data.

today: ANTSInFields Software for Simulation of and Statistical Inference on Gibbs Fields

slide-4
SLIDE 4

◭

◮

Aims / Requirements Aims

  • support teaching
  • assistance for research

Requirements

  • (really) easy to handle
  • interactive visualization turned out to be efficient tool for teaching
  • flexibility for research, testing new techniques etc.
  • extensibility for implementing new samplers etc.
slide-5
SLIDE 5

◭

◮

Realization

  • Strongly object oriented concept

allows implementation close to mathematical structure intuitive and self-explaining easy implementation of interaction and consistent visualization

  • modular design for extensibility, reusability
  • command language for flexibility on intermediate level

ANTSInFields is written in Oberon System 3 (ETH Z¨ urich, N.Wirth, Gutknecht, H.Marais, E.Zeller et al.) Oberon is also an Operating System. It runs on bare PC Hardware or as Emulation on Windows, MacOS, Linux,. . . (Portability) ANTSInFields uses the Voyager extension (University of Heidelberg, G.Sawitzki, M.Diller, F.Friedrich et al.)

slide-6
SLIDE 6

◭

◮

Scope ANTSInFields contains

  • Handling and visualization of 1D, 2D and 3D data
  • Gibbs and Metropolis Hastings Algorithms, Simulated Annealing, Exact

Sampling (CFTP)

  • Bayesian image reconstruction methods
  • parameter estimators on Gibbs fields
  • . . .

ANTSInFields is attached to 2nd Edition of G. Winklers Book ‘Image Analyis, Random Fields and Dynamic Monte Carlo Methods’, Sprin- ger Verlag In progress: Meta compiler (alpha) for easy implementing of new models Planned: Interface to R ( both command language and procedure calls).

Demonstration: Look and feel, Commands, Panels, Random Numbers

slide-7
SLIDE 7

◭

◮

Bayesian Image Restoration

slide-8
SLIDE 8

◭

◮

Random Fields Notation E finite space of states {•, •} S ⊂ Zd finite index set x = (xs)s∈S ∈ ES configuration X := ES space of configurations { , . . . , , . . . , , . . . , }

slide-9
SLIDE 9

◭

◮

Gibbs Fields We consider Neighbour-Gibbs fields Π(x) = exp(−H(x))

  • y∈X exp(−H(y)),

where H is of the form H(x) =

  • s∈S

f(xs, x∂(s)) with ∂(t) some neighbourhood of t, t ∈ S. ∂ =

slide-10
SLIDE 10

◭

◮

Ising Model Easiest nontrivial case: E = {−1, 1}, nearest neighbours and isotropy. An Ising Model with parameters β, h ∈ R is a Gibbs-Field with energy H(x) = −β

  • s∼t

xsxt + h

  • s

xs h: global tendency to take value 1 β: tendency of neighbours to be alike.

slide-11
SLIDE 11

◭

◮

Problems with sampling: P(X = x) = exp(−H(x))

  • y∈X exp(−H(y)) untractable,

but P(Xt = xt|Xs = xs, s = t) = exp(xt(h + β

s∂t xs))

cosh(h + β

s∂t xs)

easy to calculate Solution → MCMC techniques like

  • Gibbs Sampler
  • Metropolis Hastings Algorithm
  • Exact Sampling
slide-12
SLIDE 12

◭

◮

Markov Chains An irreducible Markov Chain with a stationary distribution µ is ergodic and fulfills (a) P t(i, j)

t→∞

− → µ(j) (b) ¯ fn

n→∞

− → Eµ(f(X))) in L2

if

Eµ(f(X)) < ∞,

where

¯ fn = 1 n

  • i=1...n

f(xi)

Demonstration: Reflected Markov Chain

slide-13
SLIDE 13

◭

◮

Algorithm for the Gibbs Sampler:

  • 1. 0 → n
  • 2. Sample x(0) from initial distribution, say uniform distribution on X
  • 3. Apply Kt on x(n) for all t ∈ S

i.e. sample from local characteristics Π1 in each point

  • 4. copy x(n) to x(n+1)
  • 5. n + 1 → n
  • 6. Return to step 3 until close enough to Π.

Algorithm is a realization of a Markov chain with stationary distribution Π

slide-14
SLIDE 14

◭

◮

A sweep

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥

slide-15
SLIDE 15

◭

◮

Visiting schedule

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ⑦

slide-16
SLIDE 16

◭

◮

Visiting schedule

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ⑦ ⑦

slide-17
SLIDE 17

◭

◮

Visiting schedule

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ⑦ ⑦ ⑦

slide-18
SLIDE 18

◭

◮

Visiting schedule

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ⑦ ⑦ ⑦ ⑦ ⑦

slide-19
SLIDE 19

◭

◮

Visiting schedule

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦

slide-20
SLIDE 20

◭

◮

Visiting schedule

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦

slide-21
SLIDE 21

◭

◮

Visiting schedule

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦

slide-22
SLIDE 22

◭

◮

Visiting schedule: Whole sweep finished

❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ❥ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦ ⑦

slide-23
SLIDE 23

◭

◮

Demonstrations

  • Ising Model, Gibbs Sampler
  • Ising Model + Channel noise, MMSE

H(x, y) = −β

  • s∼t

xsxt + h

  • s

xs + 1 2 ln( p 1 − p)

  • s

xsys

  • Cooling Schemes, Simulated Annealing, ICM
  • Grey-valued “Ising Model” (Potts and others)

H(x) = β

  • s∼t

ϕ(xs, xt) + h

  • s

xs

slide-24
SLIDE 24

◭

◮

  • Sampling from arbitrary Posterior Distributions

H(y, x) = β

  • x∼t

ϕ(xs, xt) + h

  • s

xs +

  • s

ϑ(xs, ys),

  • Φ-Model, Texture Synthesis

H(x) =

  • i

βi

  • s∼

i t

ϕ(xs, xt) + h

  • s

xs

slide-25
SLIDE 25

◭

◮

.

slide-26
SLIDE 26

◭

◮

Estimating (Hyper-)Parameters Assume observations on Λ = Λ + ∂Λ (Conditional) Maximum-Likelihood estimator (MLE)

  • θn := arg min

θ

− log(Pθ(Xt = xt, t ∈ Λ|Xs = xs, s ∈ ∂Λ)) = arg min

θ (log ZΛ(x∂Λ) − HΛ(xΛ | x∂Λ))

Problem: ZΛ not computable . Solution: Subsampling method (Younes(88), Winkler(01)) Alternative approach: Estimators regarding only the conditional distri- butions Pθ(Xt = xt|Xs, s ∈ ∂(t)) like: Coding, Maximum-Pseudolikelihood, Minimal least squares, Minimum Chi Square estimator etc.

slide-27
SLIDE 27

◭

◮

Coding Estimator(CE) With Λ+ = {t ∈ Λ | t1 + . . . + td even} the MLE on Λ+ becomes:

  • θn = arg min

θ

− log(P(Xt = xt, t ∈ Λ+|Xs = xs, s ∈ ∂Λ+)) = arg min

θ

− log

  • t∈Λ+

Pθ(Xt = xt|Xs, s ∈ ∂(t)). Maximum-Pseudolikelihood Estimator (MPLE) Coding Estimator with replacement: Λ+ − → Λ.

  • θn = arg min

θ

− log

  • t∈Λ

Pθ(Xt = xt|Xs, s ∈ ∂(t)) .

Demonstration: Estimating Parameters