Re Recent nt trends nds in n Aut Autom omated ed Machi chine - - PowerPoint PPT Presentation

▶
re recent nt trends nds in n aut autom omated ed machi
SMART_READER_LITE
LIVE PREVIEW

Re Recent nt trends nds in n Aut Autom omated ed Machi chine - - PowerPoint PPT Presentation

Apr 15, 2023 631 likes •832 views

Re Recent nt trends nds in n Aut Autom omated ed Machi chine ne Le Lear arni ning ng (AutoML) L) Su Summer r semester r 201 019 Ti Tim Meinhardt rdt and d Pro rof. Dr. r. Laura ra Leal-Ta Taix Ou Outline ne What

slide-1
SLIDE 1

Re Recent nt trends nds in n Aut Autom

  • mated

ed Machi chine ne Le Lear arni ning ng (AutoML) L)

Su Summer r semester r 201 019 Ti Tim Meinhardt rdt and d Pro

  • rof. Dr.
  • r. Laura

ra Leal-Ta Taixé

slide-2
SLIDE 2

Ou Outline ne

  • What is AutoML?
  • Organization

– General information – Course and paper matching – Presentations

  • Paper preview

AutoML seminar - Tim Meinhardt 2 24.01.19

slide-3
SLIDE 3

Wha What is AutoML? L?

Machine and Deep Learning Inputs

  • Tasks (Classification, Regression, etc.)
  • Datasets (research, real, non-vision)

AutoML seminar - Tim Meinhardt 3 24.01.19

slide-4
SLIDE 4

Wha What is AutoML? L?

Learn a task/dataset specific model:

  • Architecture design
  • Data processing
  • Optimization

Hy Hyperparame meter optimi mization!

AutoML seminar - Tim Meinhardt 4 24.01.19

slide-5
SLIDE 5

Wha What is AutoML? L?

  • Enhance progress on existing inputs
  • Produce state-of-the-art outputs for new inputs

– Research – Industry Machine learning experts (or graduate student descent)! Automated Mach chine Learning (AutoML)

AutoML seminar - Tim Meinhardt 5 24.01.19

slide-6
SLIDE 6

Ho How to Aut utoML ML?

Classic optimization:

  • Grid or random searches
  • Bayesian optimization (TPE, Spearmint, SMAC, etc.)

Learning to learn or Meta Learning

AutoML seminar - Tim Meinhardt 6 24.01.19

slide-7
SLIDE 7

Me Meta Learning

Leverage power of learning methods to improve learning:

  • Few shot learning
  • Pretraining on ImageNet
  • Multi-task initialization learning
  • Fast Reinforcement Learning
  • Learning architectures
  • Learning optimizers

AutoML seminar - Tim Meinhardt 7 24.01.19

AutoML

slide-8
SLIDE 8

Or Organi nization

General information

  • Website:

https://dvl.in.tum.de/teaching/automl_ss19/

  • Contact:

tim.meinhardt@tum.de

  • Room: MI 02.09.023
  • Time: 12 participants -> 6 sessions
  • Attendance is mandatory!

Schedule:

  • Pre-course meeting:

25th January 1 – 3 pm

  • Paper matching:

25th April 2 – 4 pm

  • Presentations:

Thursdays 2 - 4 pm, TBD

AutoML seminar - Tim Meinhardt 8 24.01.19

slide-9
SLIDE 9

Ma Match ching

  • Course matching (https://matching.in.tum.de/)

– See FAQ for details – Registration period: 8th - 13th February – Preference: I2DL or DL4CV grade (contact us if external student) – Announcement: 20th February

  • Paper matching

– Study our list of suggested papers (website 8th February) – Propose own paper until 20th April – On the 25th April

  • Match paper based on preferences (toss a coin if necessary)
  • Fix dates for the presentations

AutoML seminar - Tim Meinhardt 9 24.01.19

slide-10
SLIDE 10

Bef Befor

  • re

e th the e presen esenta tati tion

  • n
  • Read and work through the paper
  • Note questions and difficulties

Three weeks before: Arrange meeting to discuss and clarify paper One week before: Arrange meeting to discuss slides

AutoML seminar - Tim Meinhardt 10 24.01.19

slide-11
SLIDE 11

Pr Present ntation

  • Duration: 20 minutes talk + 10 minutes discussions
  • Finish talk on time!
  • Explain in own words
  • Complement paper content with additional material

and explanations (from an I2DL perspective)

  • Rule of thumb: 1-2 minutes per slide, i.e., 10-20 slides
  • Submit PDF until 1 week after presentation

AutoML seminar - Tim Meinhardt 11 24.01.19

slide-12
SLIDE 12

Pa Pape per pr previ view

As Asyn ynchron

  • nou
  • us Methods
  • ds for
  • r Deep Reinforcement

Le Learni ning

  • ng. Mnih et al.
  • Q-Learning
  • Advantage Actor-Critic

AutoML seminar - Tim Meinhardt 12 24.01.19

slide-13
SLIDE 13

Pa Pape per pr previ view

Pr Proxi ximal Po Policy Optimization Algorithms. Schulman et al.

  • (Proximal)Policy gradient methods
  • Trust region methods

AutoML seminar - Tim Meinhardt 13 24.01.19

slide-14
SLIDE 14

Pa Pape per pr previ view

Ne Neural Architecture Search wi with Reinforcement Le Learni ning

  • ng. Zoph et al.
  • Recurrent network to predict architectures (NAS)
  • Trained with RL

AutoML seminar - Tim Meinhardt 14 24.01.19

slide-15
SLIDE 15

Pa Pape per pr previ view

Le Learni ning ng Trans nsferable Archi hitectures for Scalable Image Rec Recog

  • gnition
  • n. Zoph et al.
  • Extension of NAS with new architecture search space
  • Applicable to large datasets

AutoML seminar - Tim Meinhardt 15 24.01.19

slide-16
SLIDE 16

Pa Pape per pr previ view

Le Learni ning ng to learn n by gradient nt descent nt by gradient nt de

  • descent. Andrychowicz et al.
  • Design of optimizer casted as a learning problem
  • Generalizes to unseen tasks

AutoML seminar - Tim Meinhardt 16 24.01.19

slide-17
SLIDE 17

Pa Pape per pr previ view

Se Searching for Activation Fu

  • Functions. Ramachandran et al.
  • Apply reinforcement learning to discover new

activation

  • New activation function Swish

AutoML seminar - Tim Meinhardt 17 24.01.19

slide-18
SLIDE 18

Pa Pape per pr previ view

Le Learni ning ng Step Size Cont ntrollers for Robust Neural Ne Netwo work Training. Daniel et al.

  • Learned learning rate scheduler
  • Reinforcement Learning

AutoML seminar - Tim Meinhardt 18 24.01.19