Learning Particle Physics by Example: Accelerating Science with - - PowerPoint PPT Presentation

Learning Particle Physics by Example:

Accelerating Science with Generative Adversarial Networks

@lukede0 @lukedeo lukedeo@manifold.ai https://ldo.io

Luke de Oliveira

Partner, Manifold Visiting Researcher, Lawrence Berkeley National Lab

arXiv:1701.05927, arXiv:1705.02355 S7666, May 11th, GPUTech 2017

Outline

• High Energy Particle Physics / LHC Physics
• How do we probe phenomena in the physical

sciences? What problems does this introduce?

• Location Aware GANs / CaloGANs
• Applications to other domains

Why this matters

• GANs provide a viable strategy for speeding up

numerically intensive simulation - extends to other disciplines (medicine, weather, nuclear)

• Basic science provides theory-driven success

metrics — feedback loop for pure ML research

High Energy Physics at the Large Hadron Collider

Physics @ LHC

• 27 km ring at the border of Switzerland and France
• Accelerate bunches of protons at 99.999999% the

speed of light - approx. 90μs per lap

• 14TeV collision energy let’s us probe constituents,

investigate rare decays, examine new theories (supersymmetry, extra dimensions, gravitons)

• Many experiments: ATLAS, CMS, LHCb, ALICE, …

What does a proton-proton collision look like? 💦

What is a Particle Detector?

ATLAS detector is a 3D camera with a temporal component that measures energy deposition in different materials and in different environments. What are we measuring? What does it look like?

Acquiring Labels

• Unlike many fields we’ve seen here at GPUTech, we

can’t label data in High Energy Physics

• The most brilliant physicist cannot look at a

detector read out and say what happened

• Use a constructivist approach, start with theory,

then simulate (we know labels from simulation)

Finding Physics using Simulation

• To perform searches for new physics or precision

measurements of known particles we simulate billions of collisions

• Allows us to calculate yield, discovery significance,

build classifiers

• This is very hard - we always have all problems

associated with transfer learning

High Energy Physics Simulation

• Encode infinite-dimensional integrals, full Quantum

Field Theory into Monte Carlo Simulation

• Using approximations, can bound the time required
• Can we do better?

Simulation Overview

Simulate proton-proton interactions Simulate showering QFT / Theory Interaction with detector materials / geometry …

Simulation Overview

Simulate proton-proton interactions Simulate showering QFT / Theory Interaction with detector materials / geometry …

CPU intensive (~60% of worldwide computing grid)

Why is this hard?

This is the first series of

• layers. We need to model

interactions on the 10-10 to resolve showering of particles

Trifecta of Issues

Time Disk Space Rare Events

Full Simulation is slow

Detector simulation can take minutes per event

Hard to model tail- statistics

To get a few rare events in kinematically unfavorable regions, we often have to simulate a large number of events

Petabytes of 
 Simulated Data

Large amounts of simulated data needs to be stored 
 and transferred

Our Panacea

Fast Portable Specialized

Our Panacea

Fast Portable Specialized

…GANs?

Location Aware Generative Adversarial Networks

arXiv:1701.05927

Generative Adversarial Networks (GAN)

tries to tell fake/real

Turn generative modeling into a two player, non-cooperative game.

tries to produce real looking samples

Defining a Step 0 - Jet Images

Avoid producing full event, focus on jets - cone shaped collections of hadrons (e.g., electrons) forming showers

η = 0 η = +∞ η = -∞

Defining a Step 0 - Jet Images

Key insight - unroll detector, well behaved manifold [arXiv/{1407.5675,1511.05190}]

From Particle Physics Computer Vision

Jet Images are 2D images where pixel intensity is transverse momentum (component perpendicular to proton path)

Single Jet Image Mean of 104 Jet Images

Step 0 Task

• Generate Jet Images using traditional software (PYTHIA)
• Learn a generative model to reproduce jets with high

fidelity

• Fidelity defined as ability to reproduce physics

quantities (manifolds) such as mass, other theoretically motivated quantities which we don’t have for most generative tasks

• Provide a speedup over traditional generation

Defining Characteristics

• Large (5 orders of magnitude) dynamic range
• Sparse (7%) occupancy of images
• Changing location of 1 pixel activation wildly

different characteristics in spacetime

• Important features have high Lipschitz constant

Location Aware GAN (LAGAN)

• Joint work with Michela Paganini (Yale) and Benjamin

Nachman (Lawrence Berkeley National Lab)

• arXiv:1701.05927
• Design domain-specific tweaks and modifications to GAN

architecture

• Use AC-GAN variant with 2 classes
• QCD (background) and W bosons (signal) — well

understood from Physics

LAGAN

Architectural Details

• Latent space:
• Perform Hadamard product between latent space

and lookup table for conditioning (AC-GAN)

• Rely on locally connected building blocks (i.e.,

convolutions without weight sharing) to leverage spacetime symmetries

LAGAN guidelines for sparse data

• Activation functions
• Batch Normalization
• Minibatch Discrimination

ReLU (chosen to induce sparsity) Leaky ReLU (rec. in GAN lit.) Stabilize gradients, help with HDR

Qualitative Assessment

5 random Jet Images nearest LAGAN-generated neighbor GAN-generated signal - background Real signal - background

Checking Against Theory

Can a GAN recover these distributions without being trained on them?

Checking Physical Properties

n-subjettiness jet mass

Our Solution

Fast Portable Specialized

Our Solution

Fast

Our solution Currently used benchmark

Our Solution

Specialized

We condition on particle class, but can easily condition on mass, momentum, …

Our Solution

Portable

Trained LAGAN weights: 20 MB Source code: 480 kB

Can we go one step further?

• We’ve shown that we can perform showering
• Can we simulate how positron / pion / photon

showers interact with materials? Most expensive part of simulation.

• Large simulation package, GEANT4, can take 5 mins

per electron to generate its interaction with a detector.

CaloGAN

arXiv:1705.02355

Solution: CaloGAN

• Simulate subatomic particles interacting with

calorimeter media

• Shoot charged pions, positrons, and photons at

cells of super-cooled liquid argon, use GAN to simulate what happens

How does an electron look in liquid argon?

Local Energy Deposit [MeV] 5 10 15 20 25 30 Depth from Calorimeter Center [mm] 200 − 150 − 100 − 50 − 50 100 150 200 direction [mm] η 200 − 150 − 100 − 50 − 50 100 150 200

• Geant4, Pb Absorber, lAr Gap, 10 GeV e

Depth from Calorimeter Center [mm] 200 − 150 − 100 − 50 − 50 100 150 200 direction [mm] η 200 − 150 − 100 − 50 − 50 100 150 200 Cell Energy [MeV] 100 200 300 400 500 600 700 800 900 1000 100 200 300 400 500 600 700 800 900 1000 100 200 300 400 500 600 700 800 900 1000 100 200 300 400 500 600 700 800 900 1000

• Geant4, Pb Absorber, lAr Gap, 10 GeV e

We simulate exact (x, y, z) What we can read out is this 2D slice

How does an electron look in liquid argon?

3 layers, unequal resolution

CaloGAN Architecture

• Condition on Energy - express conservation of

energy as a approx. sub-differentiable constraint

• Same principles as LAGAN
• Jointly train all three layers of a calorimeter

CaloGAN Generator

• Three independent streams, one per calorimeter

layer

• Custom attention mechanism decides how much

from one layer to carry to the next layer

CaloGAN Generator

Nearest Generated Samples (Positrons)

Real 1st layer deposition Real 2nd layer deposition Real 3rd layer deposition CaloGAN 1st layer deposition CaloGAN 2nd layer deposition CaloGAN 3rd layer deposition

Matching Physical Features

Faithfully reproduce energy-per-layer

Conditioning on Energy

GPU Based Speed-up

Up to a 105 speed-up!

Other Applications

• Recent successes with GANs in Cosmology
• Interest in fluids, weather, medicine (nuclear)
• Nuclear Physics (Pheno)
• Combustion

The Future

• Generative Models, GANs in particular, show

promise for modeling exotic scientific / engineering phenomena

• Let’s collaborate — at Berkeley Lab and Manifold,

we’re interested in practical uses of GANs and Machine Learning in general

Reproducible Research

• We have open-sourced our code, dataset, and analysis

procedure for both works.

• https://github.com/hep-lbdl/adversarial-jets
• https://github.com/hep-lbdl/CaloGAN

Thanks!