Machine learning techniques to probe theoretical physics Intro In - - PowerPoint PPT Presentation

Machine learning techniques to probe theoretical physics

Akinori Tanaka (RIKEN AIP/iTHEMS)

find t machine learning OR deep learning and date 20xx->20xx+1

Intro

In inSPIRE, search

year number

• f

results

Intro

Mainly experimental, a few theoretical “Deep learning shock” in ILSVRC

Agenda

• II. Reviews on selected papers
• I. Reviews on Machine Learning
• III. Summary

• I. Reviews on Machine Learning

“supervised” “un-supervised” “reinforcement”

• Rough classification

• I. Reviews on Machine Learning

“supervised” “un-supervised” “reinforcement”

• Rough classification

• I. Reviews on Machine Learning

“supervised”

• Rough classification

“machine” 2 “machine” 5 “machine” ↑MNIST dataset

• I. Reviews on Machine Learning

“supervised”

• Rough classification

well trained

machine 6 bad machine 5   

I wrote it

f : X → Y

In general, trying to learn a “concept”

• I. Reviews on Machine Learning

“supervised” “un-supervised” “reinforcement”

• Rough classification

• I. Reviews on Machine Learning
• Rough classification

“un-supervised” “machine” “machine” “machine”

No answer given

• I. Reviews on Machine Learning
• Rough classification

“un-supervised”

well trained

machine

• 1. Feature extraction

“local coupling consts” (called features) “RBM”

• I. Reviews on Machine Learning
• Rough classification

“un-supervised”

well trained

machine

• 2. Generating data

• I. Reviews on Machine Learning
• Rough classification

“un-supervised”

well trained

machine

• 2. Generating data

towards a non-perturbative corrections to quantum gravity emergent entanglement entropy for 4d superconformal theory chiral transport and entanglement entropy in general relativity

All titles in hep-th (2016) Joking demo 😝

Bowman, et al. (2015)

Using

• I. Reviews on Machine Learning

“supervised” “un-supervised” “reinforcement”

• Rough classification

• I. Reviews on Machine Learning
• Rough classification

“reinforcement”

environment

• “machine”

• I. Reviews on Machine Learning
• Rough classification

“reinforcement”

environment

• “machine”

• I. Reviews on Machine Learning
• Shock of Deep Learning

“supervised” “un-supervised” “reinforcement”

Super fine generated images

ILSVRC top errors ％

↓DL

• T. Karras, et al. (2017)

AlphaGo zero

• D. Silver, et al. (2017)

• I. Reviews on Machine Learning
• Deep Learning

“machine” 2 ＝ Multi layered perceptron

• I. Reviews on Machine Learning
• Deep Learning

“machine” 2 ＝ Linear ∈ tunable params

• I. Reviews on Machine Learning
• Deep Learning

“machine” 2 ＝ Non Linear

• I. Reviews on Machine Learning
• Deep Learning

“machine” ＝

~ x

~ y

• I. Reviews on Machine Learning
• Deep Learning

~ x

~ y

input answer ~

d ~ d

“Error function”

E(~ y, ~ d) = |~

y − ~ d|2

• I. Reviews on Machine Learning
• Deep Learning

~ x

~ y

input answer ~

d ~ d

“Error function”

E(~ y, ~ d)

params E

• I. Reviews on Machine Learning
• Deep Learning

Data

~ x

~ d

E

W ← W − ✏@W E

• I. Reviews on Machine Learning
• Deep Learning

easy to start TensorFlow Keras Chainer … Many users … Easy to write … Pythonic and others…

https://developer.nvidia.com/deep-learning-frameworks

• I. Reviews on Machine Learning
• Deep Learning
• 1. DL works very well.
• 2. MLP ≠ DL.
• 3. MLP + tips = DL.

Comments: For more details: The most famous DL book DL book by a string theorist

• I. Reviews on Machine Learning
• hep-th & machine learning ?

stat.ML hep-th

• 1. applications
• 2. proposals

=

• 3. ??

Agenda

• 1. TH ← ML
• 2. TH → ML
• 3. TH = ML
• II. Reviews on
• I. Reviews on Machine Learning

• 1. TH ← ML
• Drawing phase diagrams
• ML Landscape
• Supporting MC simulations

• 1. TH ← ML
• Drawing phase diagrams

“machine”

~ x

~ y

Idea ↑ Configurations generated by MC simulations

• 1. TH ← ML
• Drawing phase diagrams

1.72

5.00

2.50

3.84

input

• utput ・Hot

・Cold

T

2.27

Cold Hot

Carrasquilla, Melko (2016)

• 1. TH ← ML
• Drawing phase diagrams

T

Cold Hot Test acc > 90% ←Given explicitly Tc = 2 log(1 + √ 2) = 2.27 . . .

Carrasquilla, Melko (2016)

• 1. TH ← ML
• Drawing phase diagrams

Carrasquilla, Melko (2016)

Training Known Model Application Other (similar) Models

T

Tc = 4 log(3) = 3.64 . . .

• 1. TH ← ML
• Drawing phase diagrams

1.72

5.00

2.50

3.84

Data

T

draw

F a

=

...

... ...

W

T

Update F, W

AT, Tomiya (2016)

• 1. TH ← ML
• Drawing phase diagrams

F a

=

...

... ...

W

T

...

...

T

Tc ~ 2.27

AT, Tomiya (2016)

• 1. TH ← ML
• Drawing phase diagrams

Ohtsuki, Ohtsuki (2016)

• f 3D TI → 2D image

4-layered MLP phases training consistent w/ result by transfer matrix |ψ(x, y, z)|2

• 1. TH ← ML
• Drawing phase diagrams
• ML Landscape
• Supporting MC simulations

• 1. TH ← ML
• Supporting MC simulations

Z

X

dx P(x)O(x)

Integrable

😅

Aut(X) ⊃ “good” symmetry

• therwise

(usually) Non-integrable

😃

→ MC 😄

• 1. TH ← ML
• Supporting MC simulations

Z

X

dx P(x)O(x)

Sampling x[i] ∼ P(x)(i.i.d.) ∼

N

X

i=1

1 N O(x[i])

How?

• 1. TH ← ML
• Supporting MC simulations

∼ …

x[0] x[1] x[N]

N

X

i=1

1 N O(x[i])

change x[i + 1] x[i] ˜ x[i + 1] Metropolis Test(x[i], ˜ x[i + 1]) = ～ P(x)

• 1. TH ← ML
• Supporting MC simulations

∼ …

x[0] x[1] x[N]

similar similar similar

N

X

i=1

1 N O(x[i])

• 1. TH ← ML
• Supporting MC simulations

Ising Model

τ

: one spin random flip ⇣ ⇣ Autocorrelation (similarity) : Γ(τ)

• 1. TH ← ML
• Supporting MC simulations

Ising Model : one spin random flip ⇣ ⇣ ↑∃ Faster update

Big picture: ML → Fast update ?

• 1. TH ← ML
• Supporting MC simulations

Self Learning Monte Carlo

Liu, Qi, Meng, Fu (2016)

H

MC w/ global update ML

Heff

• 1. TH ← ML
• Supporting MC simulations

Self Learning Monte Carlo

Liu, Qi, Meng, Fu (2016)

…

x[0] x[1] x[N]

˜ x[0]

˜ x[1]

…

˜ x[n]

↑update by Heff

˜ Metropolis Test(˜ x[0], ˜ x[n]) = x[i + 1]

• 1. TH ← ML
• Supporting MC simulations

Self Learning Monte Carlo

Liu, Qi, Meng, Fu (2016)

• 1. TH ← ML
• Supporting MC simulations

Self Learning Monte Carlo

Liu, Qi, Meng, Fu (2016)

Using MLP

Nagai, Okumura, AT (2018)

Heff(S) = E0 − j1 X

<ij>1

SiSj − j2 X

<ij>2

SiSj − . . . |Heff(Sdata) − H(Sdata)|2 choose j1 s.t. decreases. Heff(S) = MLP(S) (QMC, S = vertices on imaginary time circle)

• 1. TH ← ML
• Supporting MC simulations

Using Boltzmann Machines

AT, Tomiya (2017)

…

x[0] x[1] x[N]

usual update x[i + 1] ˜ x[i + 1] Metropolis Test(x[i], ˜ x[i + 1]) = x[i] x0[i]

Huang, Wang (2016)

• 1. TH ← ML
• Supporting MC simulations

Using Boltzmann Machines

AT, Tomiya (2017) Huang, Wang (2016)

Scalar lattice QFT

• 1. TH ← ML
• Drawing phase diagrams
• ML Landscape
• Supporting MC simulations

• 1. TH ← ML
• ML Landscape

“machine”

~ x

~ y

Idea ↑ geometric data ↑ Invariants h1,2 h2,1 χ … polytope, topic diagram,…

Landscape → SM-like theory

• 1. TH ← ML
• ML Landscape

He (2017) Carifio, Halverson, Krioukov, Nelson (2017)

・Usage of Mathematica package ・CY3s ∈ WP^4, CICY3s, CICY4s, Quivers MLP not MLP ・F-theory compactifications

Krefl, Seong (2017)

MLP ・Toric diagram → min(vol(SE))

Agenda

• 1. TH ← ML
• 2. TH → ML
• 3. TH = ML
• II. Reviews on
• I. Reviews on Machine Learning

• 2. TH → ML
• Boltzmann machines

n n

…

n n

= ⇢ +1 −1 Imitate Design H “Boltzmann Machine” Phand-written(x) Pising(x) = e−H(x) Z

Hinton, Sejnowski (1983)

• 2. TH → ML
• Boltzmann machines

Naive BM

H = X

i,j

xiWijxj

How to train W ? → Maximize relative entropy

hlog e−H Z iP

↑hard to compute (for non-local H)

Hinton, Sejnowski (1983)

• 2. TH → ML
• Boltzmann machines

Restricted BM

↑

h integrate out ～ ～ ～ P(x, h) = e−H(x,h) Z Ptrue(x) P(x) H(x, h) = xT Wh + xT Bx + BT

h h

Hinton, Sejnowski (1983)

h ∈ Z2

• 2. TH → ML
• Boltzmann machines

Riemann-Theta BM ～ ～

↑

h integrate out ～ P(x, h) = e−H(x,h) Z Ptrue(x) P(x)

Hinton, Sejnowski (1983)

H(x, h) = xT Wh + xT Σ−1x + hT σ−1h

h ∈ Z

∝ ˜ θ(xT W|σ−1)

Krefl, Carrazza, Haghighat, Kahlen (2016)

Agenda

• 1. TH ← ML
• 2. TH → ML
• 3. TH = ML
• II. Reviews on
• I. Reviews on Machine Learning

• 3. TH = ML
• MLP = AdS ?
• 1. “Top-down”

gravity gauge scalar D-brane AdS/CFT correspondence D3 → IIB/SYM

Maldancena (1997) Aharony, Bergman, Jafferis, Maldacena (2008)

D3→ D2 → M2 → M/SCS

• 3. TH = ML
• MLP = AdS ?

AdS/CFT correspondence

• 2. “Bottom-up”

No-stringy picture (?) Gravity A QFT B

Kitaev (2015)

AdS2 + dilaton/ random ψ4 “holographic principle”

• 3. TH = ML
• MLP = AdS ?

AdS/CFT correspondence

Gravity ← QFT data

(Assuming holographic principle)

ML

• How ?

QFT data → → consistency        bulk gravity

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

• 3. TH = ML
• MLP = AdS ?

AdS/CFT correspondence S[φ] = Z dx p −detg h − 1 2(∂µφ)2 − 1 2m2φ2 − 1 4!λφ4i “parameters” for the theory ・mass ・4-point coupling ・metric m2 λ g ・bulk action acquired after training

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

• 3. TH = ML
• MLP = AdS ?

AdS/CFT correspondence ・EOM ・Metric ansatz = ‘BH’-like metric ds2 = −f(⌘)dt2 + d⌘2 + g(⌘)d~ x2 η φin πin

1st ord EOM

πout φout

bdry horizon

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

• 3. TH = ML
• MLP = AdS ?

AdS/CFT correspondence η φin πin

πout φout

bdry horizon

= 0

b.c. ・EOM ・Metric ansatz = ‘BH’-like metric ds2 = −f(⌘)dt2 + d⌘2 + g(⌘)d~ x2

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

• 3. TH = ML
• MLP = AdS ?

φin

πin

πout φout

= 0 or not ”Answer”

φin πin

πout φout

”MLP” m2, λ, h(← gµν) ˜ m2, ˜ λ, ˜ h

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

• 3. TH = ML
• MLP = AdS ?

φin

πin

πout φout

= 0 or not ”Answer”

φin πin

πout φout

”MLP” ↓feed m2, λ, h(← gµν) ˜ m2, ˜ λ, ˜ h ↓train

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

• 3. TH = ML
• MLP = AdS ?

φin πin + smoothing regularization

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

• 3. TH = ML
• MLP = AdS ?

Good / Bad ”Answer”

φin πin

πout φout

”MLP” ↓train ˜ m2, ˜ λ, ˜ h

H M

Experiment

H M

Zero/non-zero

∆±, l, L

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

• 3. TH = ML
• MLP = AdS ?

M H smooth regularization + 1/ηregularization

experimental data →metric, mass, coupling

Hashimoto, Sugishita, Tanaka, Tomiya (2018)

Summary

• Many papers on
• Drawing phase diagrams
• ML Landscape
• Supporting MC simulations
• Gravity as a MLP → candidate of gravity dual
• Statistical model → learning machines