Stochastic Exploration of Real Varieties David J. Kahle Associate - - PowerPoint PPT Presentation

▶
stochastic exploration of real varieties
SMART_READER_LITE
LIVE PREVIEW

Stochastic Exploration of Real Varieties David J. Kahle Associate - - PowerPoint PPT Presentation

Sep 01, 2023 455 likes •946 views

Stochastic Exploration of Real Varieties David J. Kahle Associate Professor Joint with Jon Hauenstein Department of Statistical Science David J. Kahle Stochastic Exploration of Real Varieties Overview 1. Motivation 2. Variety distributions

slide-1
SLIDE 1

Stochastic Exploration of Real Varieties

David J. Kahle

Associate Professor

David J. Kahle Stochastic Exploration of Real Varieties

Department of Statistical Science

Joint with Jon Hauenstein

slide-2
SLIDE 2

Overview

  • 1. Motivation

David J. Kahle Stochastic Exploration of Real Varieties

  • 2. Variety distributions
  • 3. Sampling and implementation
  • 4. Examples
  • 5. Concluding thoughts
slide-3
SLIDE 3

Motivation

slide-4
SLIDE 4

Motivation

David J. Kahle Stochastic Exploration of Real Varieties

slide-5
SLIDE 5

Motivation

David J. Kahle Stochastic Exploration of Real Varieties

? ?

slide-6
SLIDE 6

Motivation

David J. Kahle Stochastic Exploration of Real Varieties

Add bivariate normal noise Careful: not uniform on circle!

slide-7
SLIDE 7

Motivation

Problems for pattern recognition:

David J. Kahle Stochastic Exploration of Real Varieties

Applications: algebraic pattern recognition (datasets/stochastic

framework), TDA, solving nonlinear systems, optimization

Strategy for stochastically exploring real varieties

Create a distribution with mass near the variety of interest Sample from the distribution Magnetize the sampled points onto the variety with endgames Very limiting – only can generate points from parametric varieties No stochastic structure – distribution of estimators? etc. General problem – how to sample near varieties?

slide-8
SLIDE 8

Variety distributions

slide-9
SLIDE 9

Recasting the normal distribution

David J. Kahle Stochastic Exploration of Real Varieties

μ is the mean; the center of the bell curve The normal density is σ is the standard deviation; governs dispersion about μ

Partition function, normalizing constant dependent on parameters

Empirical rule –

68% of distribution within ±σ of μ 95% of distribution within ±2σ of μ 99.7% of distribution within ±3σ of μ

slide-10
SLIDE 10

Recasting the normal distribution

The normal density is

David J. Kahle Stochastic Exploration of Real Varieties

Probability mass concentrates near root of polynomial Same is true for arbitrary polynomials

exp{ –g2 } is largest on the variety, where it has value 1 Decays exponentially as you move away from variety

slide-11
SLIDE 11

Variety normal distribution – provisional

David J. Kahle Stochastic Exploration of Real Varieties

A random vector X has the variety normal distribution if with Example.

g is “given” in the sense that the vector β is known and the polynomial form is specified

slide-12
SLIDE 12

Variety normal distribution – provisional

David J. Kahle Stochastic Exploration of Real Varieties

σ = .10 σ = .20 σ = .30 σ = .40

slide-13
SLIDE 13

Variety normal* distribution – problems

David J. Kahle Stochastic Exploration of Real Varieties

  • 1. Non-compact varieties

If the variety is unbounded, then it obviously can’t be normalized Example: g(x, y) = y – x V(y–x)

slide-14
SLIDE 14

Variety normal* distribution – problems

David J. Kahle Stochastic Exploration of Real Varieties

  • 1. Non-compact varieties

Solution: Truncate or taper

  • 2. σ does not gauge variability globally
slide-15
SLIDE 15
  • 2. σ does not gauge variability globally

Variety normal* distribution – problems

David J. Kahle Stochastic Exploration of Real Varieties

Probability mass does not decay evenly across variety Example: Alpha curve, V(y2 – (x3 + x2))

slide-16
SLIDE 16

Variety normal* distribution – problems

David J. Kahle Stochastic Exploration of Real Varieties

Evenly spaced contours

slide-17
SLIDE 17
  • 2. σ does not gauge variability globally

Variety normal* distribution – problems

David J. Kahle Stochastic Exploration of Real Varieties

Probability mass does not decay evenly across variety Example: Alpha curve, V(y2 – (x3 + x2)) Cause: differing gradient sizes ⇒ differing change in variety Same shift upward Differing changes in root position Solution: normalize g by the size of its gradient (That doesn’t change the zero locus.)

slide-18
SLIDE 18

Variety normal* distribution – problems

David J. Kahle Stochastic Exploration of Real Varieties

Evenly spaced contours

slide-19
SLIDE 19

Variety normal* distribution – problems

David J. Kahle Stochastic Exploration of Real Varieties

slide-20
SLIDE 20

Variety normal* distribution – problems

David J. Kahle Stochastic Exploration of Real Varieties

  • 1. Non-compact varieties

Solution: Truncate or taper

  • 2. σ does not gauge variability globally

Solution: Normalize by gradient

  • 3. Awkward parameter space B

Non-trivial choices of β’s can make the variety empty or full B is not explicit: parameters don’t range over a convenient

  • pen subset of Rb
slide-21
SLIDE 21

Variety normal (VN) distribution

David J. Kahle Stochastic Exploration of Real Varieties

A random vector X has the variety normal distribution if with

x2 + (4y)2 – 1 (y – x)(y + x) (x2+y2)3 – 4x2y2 (x2+y2–1)3 – x2y3

slide-22
SLIDE 22

Variety normal (VN) distribution

David J. Kahle Stochastic Exploration of Real Varieties

A random vector X has the variety normal distribution if with

Whitney umbrella V(x2 – y2 z) for differing σ

slide-23
SLIDE 23

Multivariety normal (MVN) distribution

David J. Kahle Stochastic Exploration of Real Varieties

Systems of polynomials g1, …, gm are supported by the multivariety normal distribution The multivariate normal distribution has density The multivariety normal distribution has density

slide-24
SLIDE 24

Multivariety normal (MVN) distribution

David J. Kahle Stochastic Exploration of Real Varieties

The multivariety normal distribution has density

  • Example. V(x2 + y2 – 1, z)

corr = 0 corr = .9 corr = –.9 + correlation: mass aligns with same signed cells – correlation: mass aligns with opposite signed cells

slide-25
SLIDE 25

Variety induced distributions

David J. Kahle Stochastic Exploration of Real Varieties

The kernel of any PDF can be used to induce variety distributions via location-scale transformations

  • Example. Beta distributions scaled and shifted by 1/2
slide-26
SLIDE 26

Sampling and implementation

slide-27
SLIDE 27

Markov chain Monte Carlo

David J. Kahle Stochastic Exploration of Real Varieties

Markov chain Monte Carlo (MCMC) is a class of algorithms for sampling probability distributions

Stationary distribution is the target distribution Target distribution does not need to be normalized Foundational in Bayesian statistics ⇒ good software (BUGS, Stan)

Iterate two basic steps (MCMC used here)

  • 1. Generate an observation that might come from target (proposal)
  • 2. Accept/reject probabilistically according to Metropolis-Hastings

Best case: Starting anywhere, chain converges to draws from target distribution

slide-28
SLIDE 28

Random walk Metropolis

David J. Kahle Stochastic Exploration of Real Varieties

From current location, propose multivariate normal step Both problems get worse in high dimensions

If variability is too large, unacceptably low acceptance rate If variability is too small, unacceptably slow exploration

slide-29
SLIDE 29

Hamiltonian Monte Carlo (HMC)

David J. Kahle Stochastic Exploration of Real Varieties

From current, propose step from physics simulation

Impart random momentum, track position numerically, stop Marble rolling on (g2/σ2)’s surface, frictionless, given initial flick Introduce auxiliary momenta variables, track level curve of Hamiltonian numerically, project back down

slide-30
SLIDE 30

HMC is implemented in Stan, a probabilistic programming language and Bayesian engine

Stan

David J. Kahle Stochastic Exploration of Real Varieties

Stan specification Interfaces : R, Julia, Python, CLI, … Many chains can be run in parallel C++ Sample

slide-31
SLIDE 31

MVN is the posterior of the model with an improper flat prior on x and y = 0 is observed

MVN distribution as a posterior distribution

David J. Kahle Stochastic Exploration of Real Varieties

The MVN distribution can be represented as the posterior distribution of a non-identifiable model Bayes’ theorem is

Likelihood Prior Data Parameter Roles of data and parameter swap Bayes – Greek varies, Latin fixed/known Posterior Here – Greek fixed/known, Latin varies

slide-32
SLIDE 32

Bayes’ theorem is

Likelihood Prior Parameter

MVN distribution as a posterior distribution

David J. Kahle Stochastic Exploration of Real Varieties

Data Data Likelihood Parameter Given

×1

Prior Posterior

slide-33
SLIDE 33

HMC is implemented in Stan, a probabilistic programming language and Bayesian engine

Stan

Stan specification Interfaces : R, Julia, Python, CLI, … Many chains can be run in parallel C++ Sample

David J. Kahle Stochastic Exploration of Real Varieties

slide-34
SLIDE 34

VN(alpha curve, σ = .10); 100 x eight chains = 800 abs

Examples

David J. Kahle Stochastic Exploration of Real Varieties

slide-35
SLIDE 35

Examples

slide-36
SLIDE 36 −1 1 −1 1 x y −1 1 −1 1 x y −1 1 −1 1 x y −1 1 −1 1 x y

n = 25 n = 50 n = 100 n = 250 σ = .005

−1 1 −1 1 x y −1 1 −1 1 x y −1 1 −1 1 x y −1 1 −1 1 x y

σ = .025

−1 1 −1 1 x y −1 1 −1 1 x y −1 1 −1 1 x y −1 1 −1 1 x y

σ = .100

slide-37
SLIDE 37 −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y

n = 25 n = 50 n = 100 n = 250 σ = .005 σ = .025 σ = .100

slide-38
SLIDE 38 −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y

n = 25 n = 50 n = 100 n = 250 σ = .005 σ = .025 σ = .100

slide-39
SLIDE 39 −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y −2 −1 1 2 −2 −1 1 2 x y

n = 25 n = 50 n = 100 n = 250 σ = .005 σ = .025 σ = .100

slide-40
SLIDE 40

VN(torus, σ = .005/.100); 2000 points

Examples

David J. Kahle Stochastic Exploration of Real Varieties

slide-41
SLIDE 41

VN(whitney, σ = .010/.100); 2000 points

Examples

David J. Kahle Stochastic Exploration of Real Varieties

slide-42
SLIDE 42

VN(3d heart, σ = .005/.025); 2000 points

Examples

David J. Kahle Stochastic Exploration of Real Varieties

slide-43
SLIDE 43

VN(2-torus, σ = .005/.100); 2000 points

Examples

David J. Kahle Stochastic Exploration of Real Varieties

slide-44
SLIDE 44

The algorithm works remarkably well even for small σ

Moving to variety

David J. Kahle Stochastic Exploration of Real Varieties

For points on the variety, endgames can be used

Basic : Newton, gradient descent, etc. Harder : Projection with Bertini

slide-45
SLIDE 45

Concluding thoughts

slide-46
SLIDE 46

Experimentally the strategy seems to work well

David J. Kahle Stochastic Exploration of Real Varieties

Disconnected components are best found by initializing multiple chains with dispersed initial values Singularities manifest as over-dispersed regions Great references:

Betancourt, M. "A Conceptual Introduction to Hamiltonian Monte Carlo." arXiv. (2018) Neal, R. "MCMC Using Hamiltonian Dynamics" in Handbook of Markov Chain Monte Carlo. Eds. S. Brooks, A. Gelman, G. Jones, X. Meng. (2011)

σ cannot be set too large

slide-47
SLIDE 47

Thank you!!

www.kahle.io

This material is based upon work supported by the National Science Foundation under Grant Nos. 1622449 and 1622369.

David J. Kahle Stochastic Exploration of Real Varieties