Search for Cosmic Ray Sources Using Deep Learning on Spherical Data - - PowerPoint PPT Presentation

Search for Cosmic Ray Sources Using Deep Learning

• n Spherical Data

Niklas Langner Martin Erdmann Marcus Wirtz

niklas.langner@rwth-aachen.de 1 27.09.2019

niklas.langner@rwth-aachen.de 2

✓ No signal probability of 1.0 in 10 million samples

Motivation

27.09.2019 2

Cosmic rays from one source

Passage through GMF

Rigidity-dependent deflection Multiplets

Identify multiplets to identify sources

Pattern recognition task

Use convolutional neural networks (CNNs) as those perform very well in pattern recognition tasks

Approach 1 Approach 2 Project spherical data into 2D-images Network needs to learn spatial relations Distortions, overlap → Data not optimally used Use a method of convolution on a sphere Use data as a whole in its spherical form

Spherical Convolutions

Classical CNN: analyze 2D-images → Challenge: skymap of CR-data is spherical

27.09.2019 niklas.langner@rwth-aachen.de 3

Let‘s try this!

Spherical Convolutions

• Use spherical data in the HEALPix format (pixelization into pixels of equal area)
• Module enabling convolutions on HEALPix-grids: DeepSphere

27.09.2019 niklas.langner@rwth-aachen.de 4

Convolution Operation

Filter a graph signal 𝑔 ∈ ℝ𝑂pix by a kernel ℎ: Computationally efficient:

Spherical Convolutions

27.09.2019 niklas.langner@rwth-aachen.de 5

Build Graph

from HEALPix map

Graph Laplacian 𝑴

matrix describing the connections of the graph vertices

Normalized graph Laplacian 𝑴sym

𝑴sym = 𝑽𝚳𝑽𝐔 𝑽 = [𝒗1, … , 𝒗𝑂pix]

The first 16 eigenvectors of the graph Laplacian, an equivalent of Fourier modes

ℎ 𝑴sym 𝒈 = 𝑽(ℎ 𝚳 𝑽T𝒈) ℎ 𝑴sym 𝒈 = ෍

𝑙=0 𝐿

𝜄𝑙𝑴sym,𝑙𝒈

Spherical convolutions

27.09.2019 niklas.langner@rwth-aachen.de 6

𝑴sym,𝑙

𝑗𝑘:

• sum of all weighted paths of length k

between 𝑤𝑗 and 𝑤𝑘

• weight is multiplication of all the edge

weights on the path

• non-zero if and only if 𝑤𝑗 and 𝑤𝑘 connected

by at least one path of length 𝑙 Filtering can thus be interpreted as weighted linear combination of neighboring pixel values. Analogous to the classical setting but with weights determined by 𝜄 and 𝑴, with one coefficient per neighborhood

ℎ 𝑴sym 𝒈 = ෍

𝑙=0 𝐿

𝜄𝑙𝑴sym,𝑙𝒈

Simulation: Minimalistic deflection model

Simulate 1000 cosmic rays (all He, 𝐹min = 40 EeV) with 5.5% originating from one randomly positioned source 1. Coherent deflection: Rotation with rotation angles motivated by typical Galactic magnetic field models (position independent) 2. Blurring: Use 50% of the maximal blurring of model (JF12) 𝜏 = 2.79 rad ⋅ EeV/𝑆

27.09.2019 niklas.langner@rwth-aachen.de 7

Isotropy Random source position

• 1. Rotation

∼ 1/𝑆

• 2. Blurring

∼ 1/𝑆 Random direction

Data preparation

27.09.2019 niklas.langner@rwth-aachen.de 8

Model input

Building a classifier

27.09.2019 niklas.langner@rwth-aachen.de 9

Multiple Spherical Convolutions Input 2 x 49152 pixels Feature Maps 64 x 48 pixels 𝜏2

2

𝜏1

2

𝜈1 𝜈2 𝜈3 … 𝜏3

2

Statistical layer (take 𝜈 and 𝜏2) provides invariance to rotation Multiple dense layers 𝑞signal 𝑞isotropy Vector 128 entries Results Probabilities (after softmax)

• f the two cases

N E

Classifying toy simulation

• Evaluate model on data set:

▪ JF12-based (position-dependent deflection and blurring) ▪ All He or mixed charges (15% H, 45% He, 40% CNO)

• Calculate p-values to judge model capability:

27.09.2019 niklas.langner@rwth-aachen.de 10

Isotropy

Network

Skymap with signal fraction sf

Network Median 𝑞signal

p-value: relative amount

• f isotropy 𝑞signal

larger or equal to signal median 𝑞signal

Building a classifier – random search

27.09.2019 niklas.langner@rwth-aachen.de 11

Lowest p- value # Conv. Layers

(4 / 3 / 6)

# Dense Layers

(1 / 2 / 4)

Conv-Ac.

(ReLU / sigmoid / tanh / leaky ReLU / softsign)

Dense-Ac

(ReLU / sigmoid / tanh / leaky ReLU / softsign)

Pooling

(average / max)

K-Order

(3 / 5 / 10)

Dropout

( 0.2 / 0.3 / 0.4)

L-rate

(1e-5 / 1e-4)

1st 6 1 ReLU Leaky- ReLU Average 5 0.2 0.0001 2nd 6 1 ReLU Leaky- ReLU Average 5 0.3 0.0001 3rd 6 1 ReLU ReLU Average 5 0.4 0.0001 4th 6 1 ReLU Leaky- ReLU Average 5 0.4 0.0001 5th 6 1 ReLU ReLU Average 5 0.3 0.0001

• Many common parameters between the best models
• Clear preference of K=5 and as much convolutions as possible

Optimizer Regularization Loss Polynomials Adam L2: 0.1 Cross Entropy Chebyshev

Performance on simulated data

27.09.2019 niklas.langner@rwth-aachen.de 12

Below 5𝜏 at ~30 signal cosmic rays (3% of 1000 cosmic rays) Number of signal cosmic rays

• Understand why the model makes its decision using layer-wise relevance propagation
• Each pixel is given a sensitivity that tells how much it contributed to the signal probability output
• f the network

Looking into the black box

27.09.2019 niklas.langner@rwth-aachen.de 13

𝑔

sig = 1%

Conclusion and Outlook

• DeepSphere is a useful tool in analyzing skymaps

▪ Can be used for classification or regression tasks

• On simulated data it is capable to identify multiplets of 1 source with a

sensitivity of 5𝜏 for skymaps of 1000 CRs with ~30 originating from the source

• Next: Simulate a universe with multiple sources and let networks distinguish the

simulated universe from isotropy

• No optimal usage of data with DeepSphere: Predefined grid with mostly zeros

→ Try dynamic graph network methods where the graph is build according to the CRs

27.09.2019 niklas.langner@rwth-aachen.de 14

1000 cosmic rays from 10 sources 1000 cosmic rays from 50 sources 1000 cosmic rays from 250 sources

Backup

27.09.2019 niklas.langner@rwth-aachen.de 15

Layer-Wise Relevance Propagation

• Deep neural network: feed-forward graph of

neurons: ▪ 𝑦𝑘

(𝑚+1) = 𝑕

൰ ൬0, σ𝑗 𝑦𝑗

𝑚 𝑥𝑗𝑘 𝑚,𝑚+1 + 𝑐 𝑘 𝑚+1

• Use the networks output 𝑔(𝒚) and a backward pass
• f same graph to calculate relevance scores

▪ 𝑆𝑗

𝑚 = σ𝑘 𝑨𝑗𝑘 σ𝑗′ 𝑨𝑗′𝑘 𝑆 𝑘 𝑚+1 with 𝑨𝑗𝑘 = 𝑦𝑗 𝑚 𝑥𝑗𝑘 𝑚,𝑚+1

▪ 𝑗: index of neuron at layer 𝑚; Σ𝑘 sums over al upper-layer neutrons to which neuron 𝑗 contributes ▪ Conservation property: ▪ σ𝑞 𝑆𝑞

1 = 𝑔(𝒚)

27.09.2019 niklas.langner@rwth-aachen.de 16