Deep Learning: a brief overview on the possibility for astrophysics - - PowerPoint PPT Presentation

▶
deep learning a brief overview on the possibility for
SMART_READER_LITE
LIVE PREVIEW

Deep Learning: a brief overview on the possibility for astrophysics - - PowerPoint PPT Presentation

Feb 25, 2023 120 likes •465 views

Deep Learning: a brief overview on the possibility for astrophysics Fran cois-Xavier Dup e (LIS/Aix-Marseille Universit e, France) Journ ee SKA @ LAM Introduction What is Machine Learning? The aim of Machine Learning is to build a

slide-1
SLIDE 1

Deep Learning: a brief overview on the possibility for astrophysics

Fran¸ cois-Xavier Dup´ e

(LIS/Aix-Marseille Universit´ e, France) Journ´ ee SKA @ LAM

slide-2
SLIDE 2

Introduction

What is Machine Learning?

The aim of Machine Learning is to build a mathematical function which solve a human task.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 2 / 25

slide-3
SLIDE 3

Introduction

What is Machine Learning?

The aim of Machine Learning is to build a mathematical function which solve a human task.

Today tasks include classification/regression; representation (or feature) learning; transfert learning; reinforcement learning; . . .

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 2 / 25

slide-4
SLIDE 4

Introduction

Some history about Machine Learning

1956: the Dartmouth workshop Proposal We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 3 / 25

slide-5
SLIDE 5

Introduction

Some history about Machine Learning (2)

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 4 / 25

slide-6
SLIDE 6

Introduction

A brief timeline

from Andrew L. Beam F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 5 / 25

slide-7
SLIDE 7

Introduction

Today’s talk

1 Deep learning 2 Interactions with astrophysics 3 What next?

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 6 / 25

slide-8
SLIDE 8

Deep learning

Today’s talk

1 Deep learning 2 Interactions with astrophysics 3 What next?

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 7 / 25

slide-9
SLIDE 9

Deep learning

What is it?

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 8 / 25

slide-10
SLIDE 10

Deep learning

What is it?

Deep learning ⇒ hierarchical learning with high order features.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 8 / 25

slide-11
SLIDE 11

Deep learning

The location of deep learning

From https://www.machinecurve.com/index.php/2017/09/30/ the-differences-between-artificial-intelligence-machine-learning-more/ F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 9 / 25

slide-12
SLIDE 12

Deep learning

The zoo

From http://www.asimovinstitute.org/neural-network-zoo/ F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 10 / 25

slide-13
SLIDE 13

Deep learning

AlexNet

ImageNet Classification with Deep Convolutional Neural Networks by

  • A. Krizhevsky et al (NIPS 2012)

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 11 / 25

slide-14
SLIDE 14

Deep learning

AlexNet (results)

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 12 / 25

slide-15
SLIDE 15

Deep learning

Long Short-Time Memory (LSTM)

Example of a recurrent neural network.

From http://colah.github.io/posts/2015-08-Understanding-LSTMs/ F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 13 / 25

slide-16
SLIDE 16

Deep learning

Long Short-Time Memory (LSTM): details

Input gate: encode the input data. Output gate: create the next output (from the input). Forget gate: remove information from input data. Block input: output data from another block.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 14 / 25

slide-17
SLIDE 17

Deep learning

Generative Adversarial Networks (GAN)

Idea: building a generator that can fool a discriminator Generator: a NN that produce new data from noise. Discriminator: a classifier which distinguish fake data from true. A set of real samples.

Generative adversarial nets by I. Goodfellow et al (NIPS 2014)

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 15 / 25

slide-18
SLIDE 18

Deep learning

Generative Adversarial Networks (GAN): some results

From https://adeshpande3.github.io/Deep-Learning-Research-Review-Week-1-Generative-Adversarial-Nets F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 16 / 25

slide-19
SLIDE 19

Interactions with astrophysics

Today’s talk

1 Deep learning 2 Interactions with astrophysics 3 What next?

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 17 / 25

slide-20
SLIDE 20

Interactions with astrophysics

Some examples

Star-galaxy Classification Using Deep Convolutional Neural Networks by Edward J. Kim Robert J. Brunner (MNRAS 2016) Idea: Use ConvNet on the reduced calibrated pixel values.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 18 / 25

slide-21
SLIDE 21

Interactions with astrophysics

Some examples

Star-galaxy Classification Using Deep Convolutional Neural Networks by Edward J. Kim Robert J. Brunner (MNRAS 2016) Idea: Use ConvNet on the reduced calibrated pixel values.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 18 / 25

slide-22
SLIDE 22

Interactions with astrophysics

Some examples (2)

Fast Cosmic Web Simulations with Generative Adversarial Networks by A.C. Rodr´ ıguez et al (arXiv:1801.09070) Idea: use deep learning to avoid the computational cost of N-body simulation.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 19 / 25

slide-23
SLIDE 23

Interactions with astrophysics

Some examples (2)

Fast Cosmic Web Simulations with Generative Adversarial Networks by A.C. Rodr´ ıguez et al (arXiv:1801.09070) Idea: use deep learning to avoid the computational cost of N-body simulation.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 19 / 25

slide-24
SLIDE 24

Interactions with astrophysics

Some other examples from arXiv

Deep learning for studies of galaxy morphology (arXiv:1701.05917) Detecting Solar-like Oscillations in Red Giants with Deep Learning (arXiv:1804.07495) Lunar Crater Identification via Deep Learning (arXiv:1803.02192) Deep learning from 21-cm images of the Cosmic Dawn (arXiv:1805.02699) Fast Point Spread Function Modeling with Deep Learning (arXiv:1801.07615)

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 20 / 25

slide-25
SLIDE 25

What next?

Today’s talk

1 Deep learning 2 Interactions with astrophysics 3 What next?

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 21 / 25

slide-26
SLIDE 26

What next?

Explainable Machine Learning

Deep learning a black-box?

Source: Department of Defense, Advanced Research Projects Agency. F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 22 / 25

slide-27
SLIDE 27

What next?

Evaluation

One of the open question in Machine Learning: how to fully evaluate a method?

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 23 / 25

slide-28
SLIDE 28

What next?

Evaluation

One of the open question in Machine Learning: how to fully evaluate a method? No free lunch theorem (Wolpert and Macready 1997) which ”states that any two optimization algorithms are equivalent when their performance is averaged across all possible problems”.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 23 / 25

slide-29
SLIDE 29

What next?

Evaluation

One of the open question in Machine Learning: how to fully evaluate a method? No free lunch theorem (Wolpert and Macready 1997) which ”states that any two optimization algorithms are equivalent when their performance is averaged across all possible problems”. Ugly Duckling theorem (Watanabe 1969) which states that perfect classification is impossible without some sort of bias.

By OswaldLR - From: A Corny Concerto (2).png, Public Domain F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 23 / 25

slide-30
SLIDE 30

Conclusion

Take away message

Evaluation is primordial! Must be coherent with the task.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 24 / 25

slide-31
SLIDE 31

Conclusion

Take away message

Evaluation is primordial! Must be coherent with the task. Learning representation is a way to avoid some bias.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 24 / 25

slide-32
SLIDE 32

Conclusion

Take away message

Evaluation is primordial! Must be coherent with the task. Learning representation is a way to avoid some bias. Deep learning asks for big datasets, but scale very well.

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 24 / 25

slide-33
SLIDE 33

Conclusion

Take away message

Evaluation is primordial! Must be coherent with the task. Learning representation is a way to avoid some bias. Deep learning asks for big datasets, but scale very well. There is no free lunch!

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 24 / 25

slide-34
SLIDE 34

Conclusion

Thank you for your attention. Any questions?

F.-X. Dup´ e (AMU) Deep learning: an overview 16 May 2018 25 / 25