Modernmachinelearningmethods fortrustworthyscience TomCharnock - - PowerPoint PPT Presentation

Modern
machine
learning
methods

 for
trustworthy
science

Tom
Charnock Institut
d'Astrophysique
de
Paris

 
 


Why
neural
networks
don't
work

 (and
how
to
use
them)

Tom
Charnock Institut
d'Astrophysique
de
Paris

 
 


Why
neural
networks
don't
work
 


Tom
Charnock Institut
d'Astrophysique
de
Paris

 
 


Apologies
about
the
term
bias

when
something
is
intrinsically
unknowable
it
is
biased if
there
is
some
offset,
which
could
in
principle
be corrected,
it
is
biased

Apologies
about
the
term
bias

when
something
is
intrinsically
unknowable
it
is
biased if
there
is
some
offset,
which
could
in
principle
be corrected,
it
is
biased

I
(almost
always)
mean
the
top
one


 An
approximation
to
a
model,


ℕℕ(, ) : →  : →

A
crazy
likelihood
surface
of
how
likely
we are
to
get
targets
from
data

What
are
we
actually
interested
in?

(|) = ∫ (|, , )(, )

(|) = ∫ (|, , )(, )


-
Posterior predictive
density
 How
likely
are
the
true targets
given
some
data?
 
 
 
-
Likelihood
 How
likely
are
the
targets
to be
generated
by
a
particular network? 
-
Probability
density
 What
is
the
probability
of
obtaining
a
particular
network
with particular
parameter
values?

(|) (|, , ) (, )

(|) = ∫ (|, , )(, )

Where
does
this
information
about
the weights
and
hyperparameters
come
from?

Training
and
validation
data

Training
and
validation
data

Training
data
and
targets:
 
 
 Validation
data
and
targets:
 Posterior
distribution
of
weights
and
hyperparameters

{, ≡ { , | ∈ [1, ]} }train train

• train
• train

{, ≡ { , | ∈ [1, ]} }val val

• val
• val

(, |{, , {, ) ∝ }train }val (, |{, , {, )(, ) }train }val

The
failing
of
traditional
training

The
failing
of
traditional
training


 
approximator
 
 Cost
function
and
likelihood 
 
 
 smooth
and
convex
 
 
 complex
and
non-convex in
 
and


 : → ℕℕ(, ) : → (, ) = − ln (|, , ) ∗ ∗ (, ) (|, , )

Optimising
(or
training)
a
network

Optimising
(or
training)
a
network

What
are
the
maximum
likelihood
estimates
of
the
weights? 


= [({ |{ , , )] MLE argmax

• }train

}train ∗

Local
maximum
likelihood
estimates

The
main
problem...

We
degenerate
the
posterior

(, |{, ) ∝ }train → (, |{, )(, ) }train ( − , − ) MLE ∗

We
degenerate
the
posterior

(, |{, ) ∝ }train → (, |{, )(, ) }train ( − , − ) MLE ∗

All
predictions
are
(probably
incorrect)
estimates
 


(|) = ()

There
is
no
way
to
interpret
how
close
 
is
to
 ...

There
is
no
way
to
interpret
how
close
 
is
to
 ...

• Because
the
likelihood
is
non-interpretably
complex

Are
there
better
methods?

Variational
inference

(|) = ∫ (|, , )(|, , {, )(, ) }train

Still
depends
on
xed
weights
in
the
complex
likelihood
surface
 and
choice
of
variational
distribution 


(|) = ∫ (|, , )(|, , {, ) }train × ( − , − ) MLE ∗ = ∫ (|, , )(| , , {, ). ∗ MLE ∗ }train

Bayesian
neural
networks

Bayesian
neural
networks

Sample
the
likelihood
of
the
training
data

(|) =

∝

∫ (|, , )(, |{, ) }train

∫ (|, , ) × ( | , , )(, ). ∏

• train

train

• train

Still
dependent
on
the
training
data!


 Classical
network
:
 Variational
inference
:
 Bayesian
networks
:


(, |{, ) → ( − , − ) }train MLE ∗ (, |{, ) = (| , , {, ) }train MLE ∗ }train (, |{, ) = ( | , , )(, ) }train ∏train

• train
• train

Problems
with
physical
models...

Problems
with
physical
models...

How
can
we
use
a
neural
network
then?

Build
it
into
the
physical
model

Method
1
:

 
 Infer
the
data,
physics
and
the
neural network

Method
2
:
 
 Understand
the
likelihood
(using
neural physical
engines)

Method
3
:

 
 Likelihood-free
inference

Compare
distance
between
observed
summaries
and simulation
summaries
and
select
results
within
 
 


Conclusions

Conclusions

Neural
networks
are
not
to
be
trusted They
can
make
trusty
companions
-
when
the
correct framework
is
introduced Using
statistics
we
can
build
neural
networks
into
the forward
model
to
get
unbiased
results

For
more
information
read
my
new
blog
 


bit.ly/ProbNN