What do noisy datapoints tell us about the true signal? C. R. Hogg - - PowerPoint PPT Presentation

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

What do noisy datapoints tell us about the true signal?

• C. R. Hogg

November 16, 2011

1/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Outline

Overview Bayesian Analysis Gaussian Processes Scattering Curves Conclusions

2/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Overview

Q (1/A) Scattered Intensity

500 1000 1500 2000 2500 2 3 4 5 6 7 8

3/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Overview

Q (1/A) Scattered Intensity

500 1000 1500 2000 2500 2 3 4 5 6 7 8

3/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Overview

Q (1/A) Scattered Intensity

500 1000 1500 2000 2500 2 3 4 5 6 7 8

3/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Overview

Q (1/A) Scattered Intensity

500 1000 1500 2000 2500 2 3 4 5 6 7 8

3/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Overview

Q (1/A) Scattered Intensity

500 1000 1500 2000 2500 2 3 4 5 6 7 8

3/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Overview

Q (1/A) Scattered Intensity

500 1000 1500 2000 2500 2 3 4 5 6 7 8

Ernest Rutherford (1871-1937)

“If your experiment needs statistics, you ought to have done a better experiment”

3/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Overview

Q (1/A) Scattered Intensity

500 1000 1500 2000 2500 2 3 4 5 6 7 8

Ernest Rutherford (1871-1937)

“If your experiment needs statistics, you ought to have done a better experiment”

(Or: use better statistics!)

3/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Goals for the talk

• 1. Explain Bayesian analysis at

conceptual level

4/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Goals for the talk

Uncertainty in single quantity:

Quantity of Interest

7 8 9 10 11 12 13

• 1. Explain Bayesian analysis at

conceptual level

• 2. Discuss quantifying uncertainty

in continuous functions

4/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Goals for the talk

Uncertainty in single quantity:

Quantity of Interest

7 8 9 10 11 12 13

• 1. Explain Bayesian analysis at

conceptual level

• 2. Discuss quantifying uncertainty

in continuous functions

4/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Goals for the talk

Uncertainty in single quantity:

Quantity of Interest

7 8 9 10 11 12 13

. . . in continuous functions:

• 1. Explain Bayesian analysis at

conceptual level

• 2. Discuss quantifying uncertainty

in continuous functions

4/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Goals for the talk

Uncertainty in single quantity:

Quantity of Interest

7 8 9 10 11 12 13

. . . in continuous functions:

• 1. Explain Bayesian analysis at

conceptual level

• 2. Discuss quantifying uncertainty

in continuous functions

4/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Goals for the talk

Uncertainty in single quantity:

Quantity of Interest

7 8 9 10 11 12 13

. . . in continuous functions:

same shape stays inside

• 1. Explain Bayesian analysis at

conceptual level

• 2. Discuss quantifying uncertainty

in continuous functions

4/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Why Bayes at NIST?

NIST’s mission: To promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve

• ur quality of life.

NIST’s vision: NIST will be the world’s leader in creating critical measurement solutions and promoting equitable standards. Our efforts stimulate innovation, foster industrial competitiveness, and improve the quality of life. NIST’s core competencies:

• Measurement science
• Rigorous traceability
• Development and use of standards
• Measurement is

extremely important at NIST

5/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Why Bayes at NIST?

NIST’s mission: To promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve

• ur quality of life.

NIST’s vision: NIST will be the world’s leader in creating critical measurement solutions and promoting equitable standards. Our efforts stimulate innovation, foster industrial competitiveness, and improve the quality of life. NIST’s core competencies:

• Measurement science
• Rigorous traceability
• Development and use of standards
• Measurement is

extremely important at NIST

• Must quantify

uncertainty:

“A measurement result is complete only when accompanied by a quantitative statement of its uncertainty.”a

5/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Why Bayes at NIST?

NIST’s mission: To promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve

• ur quality of life.

NIST’s vision: NIST will be the world’s leader in creating critical measurement solutions and promoting equitable standards. Our efforts stimulate innovation, foster industrial competitiveness, and improve the quality of life. NIST’s core competencies:

• Measurement science
• Rigorous traceability
• Development and use of standards
• Measurement is

extremely important at NIST

• Must quantify

uncertainty:

“A measurement result is complete only when accompanied by a quantitative statement of its uncertainty.”a

• Which language to

discuss uncertainty?

5/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Why Bayes at NIST?

NIST’s mission: To promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve

• ur quality of life.

NIST’s vision: NIST will be the world’s leader in creating critical measurement solutions and promoting equitable standards. Our efforts stimulate innovation, foster industrial competitiveness, and improve the quality of life. NIST’s core competencies:

• Measurement science
• Rigorous traceability
• Development and use of standards
• Measurement is

extremely important at NIST

• Must quantify

uncertainty:

“A measurement result is complete only when accompanied by a quantitative statement of its uncertainty.”a

• Which language to

discuss uncertainty?

• If “probabilities”:

Bayesian analysis

aNIST TN 1297 5/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

What is Bayesian Analysis?

6/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

What is Bayesian Analysis?

(Rev. Thomas Bayes, c. 1701 – 1761)

6/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

What is Bayesian Analysis?

(Rev. Thomas Bayes, c. 1701 – 1761)

Essence of Bayes:

• 2 questions for every

guess (i.e. every θ)

• 1. How likely does it make

the actual data?

• 2. How plausible is it?

6/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

What is Bayesian Analysis?

(Rev. Thomas Bayes, c. 1701 – 1761)

Essence of Bayes:

• 2 questions for every

guess (i.e. every θ)

• 1. How likely does it make

the actual data?

• 2. How plausible is it?
• Combine them to answer

the main question:

• 1. What is your new

probability, now that you’ve seen the data?

6/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

What is Bayesian Analysis?

(Rev. Thomas Bayes, c. 1701 – 1761)

Essence of Bayes:

• 2 questions for every

guess (i.e. every θ)

• 1. How likely does it make

the actual data? “LIKELIHOOD”

• 2. How plausible is it?

“PRIOR”

• Combine them to answer

the main question:

• 1. What is your new

probability, now that you’ve seen the data? “POSTERIOR”

6/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

What is Bayesian Analysis?

(Rev. Thomas Bayes, c. 1701 – 1761)

Essence of Bayes:

• 2 questions for every

guess (i.e. every θ)

• 1. How likely does it make

the actual data? “LIKELIHOOD”

• 2. How plausible is it?

“PRIOR”

• Combine them to answer

the main question:

• 1. What is your new

probability, now that you’ve seen the data? “POSTERIOR”

6/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Likelihood of function p(y|f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Example: artificial dataset
• Noise model: Poisson

p(y|f) = f ye−f y!

• Assume independent

pixels

7/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Likelihood of function p(y|f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Example: artificial dataset
• Noise model: Poisson

p(y|f) = f ye−f y!

• Assume independent

pixels

7/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Likelihood of function p(y|f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Example: artificial dataset
• Noise model: Poisson

p(y|f) = f ye−f y!

• Assume independent

pixels

7/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Likelihood of function p(y|f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Example: artificial dataset
• Noise model: Poisson

p(y|f) = f ye−f y!

• Assume independent

pixels

7/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Likelihood of function p(y|f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Example: artificial dataset
• Noise model: Poisson

p(y|f) = f ye−f y!

• Assume independent

pixels

• Problem: not plausible
• (What makes a function

“plausible”?)

7/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

“Plausibility” of function p(f)

• Assume smooth and

continuous

• No functional form

assumed

8/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

“Plausibility” of function p(f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Assume smooth and

continuous

• No functional form

assumed

8/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

“Plausibility” of function p(f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Assume smooth and

continuous

• No functional form

assumed

8/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

“Plausibility” of function p(f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Assume smooth and

continuous

• No functional form

assumed

8/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

“Plausibility” of function p(f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Assume smooth and

continuous

• No functional form

assumed

8/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

“Plausibility” of function p(f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Assume smooth and

continuous

• No functional form

assumed

8/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

“Plausibility” of function p(f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Assume smooth and

continuous

• No functional form

assumed

8/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

“Plausibility” of function p(f)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Assume smooth and

continuous

• No functional form

assumed

• Naturally: unrelated to data

8/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Posterior probability p(f|y)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• “Best of both worlds”:

a Plausible curves, which b fit the data

9/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Posterior probability p(f|y)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• “Best of both worlds”:

a Plausible curves, which b fit the data

9/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Posterior probability p(f|y)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• “Best of both worlds”:

a Plausible curves, which b fit the data

9/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Posterior probability p(f|y)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• “Best of both worlds”:

a Plausible curves, which b fit the data

9/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Posterior probability p(f|y)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• “Best of both worlds”:

a Plausible curves, which b fit the data

9/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Posterior probability p(f|y)

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• “Best of both worlds”:

a Plausible curves, which b fit the data

• To represent uncertainty:

show many guesses

• (Or, summarize them. . . )

9/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Posterior probability p(f|y)

Quantitative uncertainty visuals

5 10 15 20 25 30 35 40 0.2 0.4 0.6 0.8 1 Intensity (no. of counts) Q (1/A) Noisy Data

9/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Posterior probability p(f|y)

Quantitative uncertainty visuals

5 10 15 20 25 30 35 40 0.2 0.4 0.6 0.8 1 Intensity (no. of counts) Q (1/A) Noisy Data True Curve!

9/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Recap: Bayesian denoising

Plausible curves

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0 10/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Recap: Bayesian denoising

Curves which fit the data

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0 10/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Recap: Bayesian denoising

Plausible curves which fit the data

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0 10/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = ,

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , ,

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , ,

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , ,

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , ,

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , ,

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , ,

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , ,

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

x F(x)

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

• Random function F(x)?

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

x F(x)

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

• Random function F(x)?

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

x F(x)

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

• Random function F(x)?

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

x F(x)

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

• Random function F(x)?

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

x F(x)

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

• Random function F(x)?

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

x F(x)

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

• Random function F(x)?

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

x F(x)

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

• Random function F(x)?

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Random variables; Random functions

p = 1

6 1 6 1 6 1 6 1 6 1 6

F = , , , , , , , , , . . .

x F(x)

• Random variable F:

an uncertain quantity

• calculate probabilities

for its values

• take “random draws”

(roll the die, flip the coin. . . )

• Random function F(x)?

11/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation
• correlation × variance:

covariance

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to think about “random functions”?

F(x)

• Function: a collection of

individual values

• Every value is a random

variable, with. . .

• 1. variance
• 2. correlation
• correlation × variance:

covariance

• Gaussian Process:
• Every point is a

Random Variable

• Any (finite) subset has

Gaussian joint distribution

12/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to read a Covariance Matrix

Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

• How to read the matrix?

13/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to read a Covariance Matrix

Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

• How to read the matrix?
• 1. By individual entries

13/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to read a Covariance Matrix

Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

• How to read the matrix?
• 1. By individual entries

13/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to read a Covariance Matrix

Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

• How to read the matrix?
• 1. By individual entries

13/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to read a Covariance Matrix

Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

• How to read the matrix?
• 1. By individual entries
• 2. As a whole

(central stripe)

• Intensity:

height of features

• Width:

width of features

13/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to read a Covariance Matrix

Stripe Intensity = Feature Height X

• 100

100 1 2 3

X

1 2 3 1 2 3 Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

• How to read the matrix?
• 1. By individual entries
• 2. As a whole

(central stripe)

• Intensity:

height of features

• Width:

width of features

13/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

How to read a Covariance Matrix

Stripe Intensity = Feature Height X

• 100

100 1 2 3

X

1 2 3 1 2 3 Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

Stripe Width = Feature Width X

• 100

100 1 2 3

X

1 2 3 1 2 3

• How to read the matrix?
• 1. By individual entries
• 2. As a whole

(central stripe)

• Intensity:

height of features

• Width:

width of features

13/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Example 1: Hydrocarbon combustion

(Dave Sheen and Wing Tsang, NIST, Div. 632) Hydrocarbon burning simulations

• Need (many!) reaction rate

constants

• Measured individually
• Predictions are precise,

quantitative

14/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Example 1: Hydrocarbon combustion

(Dave Sheen and Wing Tsang, NIST, Div. 632) Hydrocarbon burning simulations

• Need (many!) reaction rate

constants

• Measured individually
• Predictions are precise,

quantitative, wrong

14/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Hydrocarbon combustion: flame speed experiments

Flame speed

Fuel-to-oxygen ratio Speed (cm/s)

10 20 30 40 0.6 0.8 1.0 1.2 1.4 1.6

• Datapoints (from

several experiments)

15/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Hydrocarbon combustion: flame speed experiments

Flame speed

Fuel-to-oxygen ratio Speed (cm/s)

10 20 30 40 0.6 0.8 1.0 1.2 1.4 1.6

• Datapoints (from

several experiments)

• Model: lengthscales

ℓ and σf

15/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Hydrocarbon combustion: flame speed experiments

Flame speed

Fuel-to-oxygen ratio Speed (cm/s)

10 20 30 40 0.6 0.8 1.0 1.2 1.4 1.6

• Datapoints (from

several experiments)

• Model: lengthscales

ℓ and σf

• ±1σ range

15/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Hydrocarbon combustion: flame speed experiments

Flame speed

Fuel-to-oxygen ratio Speed (cm/s)

10 20 30 40 0.6 0.8 1.0 1.2 1.4 1.6

• Datapoints (from

several experiments)

• Model: lengthscales

ℓ and σf

• ±1σ range
• See also: individual curves

15/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Hydrocarbon combustion: flame speed experiments

Flame speed

Fuel-to-oxygen ratio Speed (cm/s)

10 20 30 40 0.6 0.8 1.0 1.2 1.4 1.6

• Datapoints (from

several experiments)

• Model: lengthscales

ℓ and σf

• ±1σ range
• See also: individual curves
• But where did this model

come from. . . ?

15/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor, intro

• William of Occam
• c. 1288 - c. 1348
• Gave us Occam’s Razor

16/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor, intro

• William of Occam
• c. 1288 - c. 1348
• Gave us Occam’s Razor
• (slightly paraphrased in

the name of science)

16/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor, intro

• William of Occam
• c. 1288 - c. 1348
• Gave us Occam’s Razor
• (slightly paraphrased in

the name of science)

• Claim: use probability,

get this automatically

• (And, quantitative, too!)

16/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor, intro

MODEL 1: flat

• William of Occam
• c. 1288 - c. 1348
• Gave us Occam’s Razor
• (slightly paraphrased in

the name of science)

• Claim: use probability,

get this automatically

• (And, quantitative, too!)
• Example: 3 models. . .

16/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor, intro

MODEL 2: flat + line

• William of Occam
• c. 1288 - c. 1348
• Gave us Occam’s Razor
• (slightly paraphrased in

the name of science)

• Claim: use probability,

get this automatically

• (And, quantitative, too!)
• Example: 3 models. . .

16/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor, intro

MODEL 3: flat + line + wiggle

• William of Occam
• c. 1288 - c. 1348
• Gave us Occam’s Razor
• (slightly paraphrased in

the name of science)

• Claim: use probability,

get this automatically

• (And, quantitative, too!)
• Example: 3 models. . .

16/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor in action

• Some models can explain

more datasets

17/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor in action

• Some models can explain

more datasets

17/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor in action

• Some models can explain

more datasets

• Each model is probability

distribution:

• Same total probability

to distribute

17/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor in action

• Some models can explain

more datasets

• Each model is probability

distribution:

• Same total probability

to distribute

17/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor in action

• Some models can explain

more datasets

• Each model is probability

distribution:

• Same total probability

to distribute

• Which data actually
• bserved?

17/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

• Few datapoints (2):

simple models preferred

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

• Few datapoints (2):

simple models preferred

• New data, some models

can’t explain

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

• Few datapoints (2):

simple models preferred

• New data, some models

can’t explain

• Three tiers
• 1. Fit too poor
• 2. Fit too good
• 3. Just right

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

• Few datapoints (2):

simple models preferred

• New data, some models

can’t explain

• Three tiers
• 1. Fit too poor
• 2. Fit too good
• 3. Just right

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

• Few datapoints (2):

simple models preferred

• New data, some models

can’t explain

• Three tiers
• 1. Fit too poor
• 2. Fit too good
• 3. Just right

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

• Few datapoints (2):

simple models preferred

• New data, some models

can’t explain

• Three tiers
• 1. Fit too poor
• 2. Fit too good
• 3. Just right

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

• Few datapoints (2):

simple models preferred

• New data, some models

can’t explain

• Three tiers
• 1. Fit too poor
• 2. Fit too good
• 3. Just right
• Clear winner

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Occam’s razor and Gaussian Processes

ℓ ℓ ℓ σ σ σ

Probability

• f Model:

0 1/9 1

• 9 models, varying

complexity

• Few datapoints (2):

simple models preferred

• New data, some models

can’t explain

• Three tiers
• 1. Fit too poor
• 2. Fit too good
• 3. Just right
• Clear winner

18/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Example 2: Metal Strain

(Adam Creuziger and Mark Iadicola, NIST, Div. 655)

• Testing stress/strain of

steels (auto parts, etc.)

• Clamp flat plate; push

upwards on middle

• Measure:
• 1. Stress: X-ray diffraction
• 2. Strain: Digital imaging
• f spray-paint pattern

(Figures courtesy of Mark Iadicola)

19/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Example 2: Metal Strain

(Adam Creuziger and Mark Iadicola, NIST, Div. 655)

• Testing stress/strain of

steels (auto parts, etc.)

• Clamp flat plate; push

upwards on middle

• Measure:
• 1. Stress: X-ray diffraction
• 2. Strain: Digital imaging
• f spray-paint pattern
• Can’t paint

everywhere! (Figures courtesy of Mark Iadicola)

19/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Metal Strain: Preliminary Results

• Spheres represent

datapoints

20/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Metal Strain: Preliminary Results

• Spheres represent

datapoints

• Continuous surface

20/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Metal Strain: Preliminary Results

• Spheres represent

datapoints

• Continuous surface
• Uncertainty bounds ±1σ

20/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Metal Strain: Preliminary Results

• Spheres represent

datapoints

• Continuous surface
• Uncertainty bounds ±1σ

20/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Metal Strain: Preliminary Results

• Spheres represent

datapoints

• Continuous surface
• Uncertainty bounds ±1σ
• See also animations

20/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Metal Strain: Preliminary Results

• Spheres represent

datapoints

• Continuous surface
• Uncertainty bounds ±1σ
• See also animations
• Competing model:

anisotropic

20/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Metal Strain: Preliminary Results

• Spheres represent

datapoints

• Continuous surface
• Uncertainty bounds ±1σ
• See also animations
• Competing model:

anisotropic

• Occam’s razor lets us

choose!

• ∆ log(ML) = +183.4

20/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Metal Strain: Preliminary Results

• Spheres represent

datapoints

• Continuous surface
• Uncertainty bounds ±1σ
• See also animations
• Competing model:

anisotropic

• Occam’s razor lets us

choose!

• ∆ log(ML) = +183.4
• Suggestions for

experimental design

20/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Need to extend the model

Stripe Intensity = Feature Height X

• 100

100 1 2 3

X

1 2 3 1 2 3 Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

Stripe Width = Feature Width X

• 100

100 1 2 3

X

1 2 3 1 2 3

• Recall: how to read

covariance matrices “as a whole”

• Intensity:

height of features

• Width:

width of features

21/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Need to extend the model

Stripe Intensity = Feature Height X

• 100

100 1 2 3

X

1 2 3 1 2 3 Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

Stripe Width = Feature Width X

• 100

100 1 2 3

X

1 2 3 1 2 3

• Recall: how to read

covariance matrices “as a whole”

• Intensity:

height of features

• Width:

width of features

• Not flexible enough for

real data!

21/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Two extensions

Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

• Two main extensions. . .

22/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Two extensions

Varying Widths X

• 100

100 1 2 3

X

1 2 3 1 2 3 Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

• Two main extensions. . .
• 1. Varying Feature widths
• ℓ → ℓ(X)

22/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Two extensions

Varying Widths X

• 100

100 1 2 3

X

1 2 3 1 2 3 Cov 2500

X

• 100

100 1 2 3

X

1 2 3 1 2 3

Multiple Contributions X

• 100

100 1 2 3

X

1 2 3 1 2 3

• Two main extensions. . .
• 1. Varying Feature widths
• ℓ → ℓ(X)
• 2. Multiple contributions
• Background everywhere
• Localized “peak” regions

22/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Simulated XRD from 2 nm Au nanoparticles

500 1000 1500 2000 2500 5 10 15 20 25 30 35 I(Q) True function 110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) True function

• Core/shell structure
• Shell atoms vibrate more
• Correlated thermal

motion (Signature: hi-Q

• scillations)

23/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Simulated XRD from 2 nm Au nanoparticles

500 1000 1500 2000 2500 5 10 15 20 25 30 35 I(Q) True function 110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) True function

• Core/shell structure
• Shell atoms vibrate more
• Correlated thermal

motion (Signature: hi-Q

• scillations)

23/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Simulated XRD from 2 nm Au nanoparticles

500 1000 1500 2000 2500 5 10 15 20 25 30 35 I(Q) True function 110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) True function

• Core/shell structure
• Shell atoms vibrate more
• Correlated thermal

motion (Signature: hi-Q

• scillations)

23/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Simulated XRD from 2 nm Au nanoparticles

500 1000 1500 2000 2500 5 10 15 20 25 30 35 I(Q) Noisy data True function 110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) Noisy data True function

• Core/shell structure
• Shell atoms vibrate more
• Correlated thermal

motion (Signature: hi-Q

• scillations)
• Problem: Poisson noise

swamps these oscillations!

23/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Simulated XRD from 2 nm Au nanoparticles

500 1000 1500 2000 2500 5 10 15 20 25 30 35 I(Q) Noisy data True function 110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) Noisy data True function

• Core/shell structure
• Shell atoms vibrate more
• Correlated thermal

motion (Signature: hi-Q

• scillations)
• Problem: Poisson noise

swamps these oscillations!

• Changing feature widths:

use ℓ(Q)

X 1 2 3 1 2 3

23/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Denoising results

110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) Noisy data True function AWS (raw data)

• AWS: jagged; loses signal

at Q = 26A−1

24/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Denoising results

110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) Noisy data True function Wavelets

• AWS: jagged; loses signal

at Q = 26A−1

• Wavelets: smooth, but still

lose signal

24/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Denoising results

110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) Noisy data True function Bayesian

• AWS: jagged; loses signal

at Q = 26A−1

• Wavelets: smooth, but still

lose signal

• Bayes: also smooth, but

keeps signal

• Uncertainty bounds

capture true function

24/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Denoising results

110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) Noisy data Bayesian True function AWS (raw data) Wavelets

• AWS: jagged; loses signal

at Q = 26A−1

• Wavelets: smooth, but still

lose signal

• Bayes: also smooth, but

keeps signal

• Uncertainty bounds

capture true function

24/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Residuals

• Need: global fidelity

measure

25/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Residuals

Residuals vs. noisy data

MSR Probability Density

AWS AWS (raw data) Bayesian mean Wavelets Bayesian draws

0.240 0.245 0.250

• Need: global fidelity

measure

• Mean square residuals . . .
• 1. vs. noisy data
• AWS looks best

25/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Residuals

Residuals vs. noisy data

MSR Probability Density

AWS AWS (raw data) Bayesian mean Wavelets Bayesian draws

0.240 0.245 0.250

Residuals vs. true curve

MSR Probability Density

AWS AWS (raw data) Bayesian mean Wavelets Bayesian draws True curve

0.000 0.005 0.010

• Need: global fidelity

measure

• Mean square residuals . . .
• 1. vs. noisy data
• AWS looks best
• 2. vs. true curve
• Bayes is best
• AWS “good” score:

was overfitting noise!

25/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

TiO2 nanoparticles

NIST SRM 1898:

(Ohno et al., J. Catalysis, 2011)

• X-ray powder diffraction

from 20 nm TiO2 nanoparticles

• Motivations:
• 1. Real-world example

(Violates our assumptions)

• 2. More difficult data

(contains feature-free background regions)

26/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Comparing the fits

Q (1/A)

1000 2000 3000 1 2 3 4 5 6 7

Q (1/A)

200 400 600 800 1000 3.2 3.3 3.4

• All handle sharp peaks
• Every technique misses a

few features: AWS, wavelets, even Bayes

27/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Comparing the fits

Q (1/A)

1000 2000 3000 1 2 3 4 5 6 7

Q (1/A)

80 100 120 5.8 5.9

• All handle sharp peaks
• Every technique misses a

few features: AWS, wavelets, even Bayes

27/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Comparing the fits

Q (1/A)

1000 2000 3000 1 2 3 4 5 6 7

Q (1/A)

100 200 7.0 7.2

• All handle sharp peaks
• Every technique misses a

few features: AWS, wavelets, even Bayes

27/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Comparing the fits

Q (1/A)

1000 2000 3000 1 2 3 4 5 6 7

Q (1/A)

100 120 6.2 6.4

• All handle sharp peaks
• Every technique misses a

few features: AWS, wavelets, even Bayes

27/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Comparing the fits

Q (1/A)

1000 2000 3000 1 2 3 4 5 6 7

Q (1/A)

80 100 120 5.2 5.3

• All handle sharp peaks
• Every technique misses a

few features: AWS, wavelets, even Bayes

27/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Residuals

Residuals vs. quasi-true curve

MSR Probability Density

AWS Bayesian Mean Wavelets

0.025 0.030 0.035

• Bayes single-curve

comparable to benchmarks

28/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Residuals

Residuals vs. quasi-true curve

MSR Probability Density

AWS Bayesian Mean Wavelets

0.025 0.030 0.035

Training on Even Points

Mean MSR Overfitting

0.000 0.020

Bayes AWS Wavelets

0.170 0.180

Training on Odd Points

Mean MSR Overfitting

0.000 0.020

Bayes AWS Wavelets

0.170 0.180

• Bayes single-curve

comparable to benchmarks

• Cross-validation:

(Checking for overfitting) Bayes is best. . .

• 1. In both categories
• 2. For both training sets

28/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Recap: Bayesian Concepts

Scattering vector Q (1/A) Intensity (number of counts)

20 40 0.0 0.5 1.0

• Bayesian analysis:

using probabilities to describe uncertainty

• choose answers with

both plausibility and data fit

• a natural framework for

model selection concepts (Occam’s razor)

29/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Recap: Uncertainty in continuous functions

110 120 130 140 150 160 170 23 24 25 26 27 28 29 30 Q (1/A) Noisy data Bayesian True function AWS (raw data) Wavelets

• Gaussian Processes: can

stipulate smoothness, without worrying about functional form

• Open-source software

package

• Very flexible: can help a

variety of projects

30/31

. . Overview . . . . . . Bayesian Analysis . . . . . . . . . . Gaussian Processes . . . . . . . . Scattering Curves . . . Conclusions

Acknowledgements

• Team members: Igor Levin, Kate Mullen
• Collaborators:
• Flame speed: Dave Sheen, Wing Tsang
• Metal Strain: Adam Creuziger, Mark Iadicola
• WERB readers: Victor Krayzmann, Adam Pintar
• Statistical guidance: Antonio Possolo, Blaza Toman

31/31