Chalmers Machine Learning Seminars Olof Mogren September ACL - - PowerPoint PPT Presentation

▶
chalmers machine learning seminars
SMART_READER_LITE
LIVE PREVIEW

Chalmers Machine Learning Seminars Olof Mogren September ACL - - PowerPoint PPT Presentation

Apr 23, 2023 476 likes •964 views

Chalmers Machine Learning Seminars Olof Mogren September ACL overview Title statistics: ACL overview Title statistics: LSTM+RNN+Neural Networks: + ACL overview

slide-1
SLIDE 1

Chalmers Machine Learning Seminars

Olof Mogren September 

slide-2
SLIDE 2

ACL  overview

  • Title statistics:
slide-3
SLIDE 3

ACL  overview

  • Title statistics:
  • LSTM+RNN+Neural Networks: +
slide-4
SLIDE 4

ACL  overview

  • Title statistics:
  • LSTM+RNN+Neural Networks: +
  • Embeddings: +
slide-5
SLIDE 5

ACL  overview

  • Title statistics:
  • LSTM+RNN+Neural Networks: +
  • Embeddings: +
  • Parsing: +
slide-6
SLIDE 6

ACL  overview

  • Title statistics:
  • LSTM+RNN+Neural Networks: +
  • Embeddings: +
  • Parsing: +
  • Translation: +
slide-7
SLIDE 7

ACL  overview

  • Title statistics:
  • LSTM+RNN+Neural Networks: +
  • Embeddings: +
  • Parsing: +
  • Translation: +
  • Dialogue/QA: +
slide-8
SLIDE 8

ACL  overview

  • Title statistics:
  • LSTM+RNN+Neural Networks: +
  • Embeddings: +
  • Parsing: +
  • Translation: +
  • Dialogue/QA: +
  • Summarization: +
slide-9
SLIDE 9

ACL  overview

  • Title statistics:
  • LSTM+RNN+Neural Networks: +
  • Embeddings: +
  • Parsing: +
  • Translation: +
  • Dialogue/QA: +
  • Summarization: +
  • Visitors: ~
slide-10
SLIDE 10

ACL  overview

  • Title statistics:
  • LSTM+RNN+Neural Networks: +
  • Embeddings: +
  • Parsing: +
  • Translation: +
  • Dialogue/QA: +
  • Summarization: +
  • Visitors: ~
  • Tracks: ~
slide-11
SLIDE 11

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Outstanding paper award
slide-12
SLIDE 12

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Outstanding paper award
  • User knows goal, but needs computer

to take actions

slide-13
SLIDE 13

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Outstanding paper award
  • User knows goal, but needs computer

to take actions

  • Computer does not know the goal,

and not the language

slide-14
SLIDE 14

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Outstanding paper award
  • User knows goal, but needs computer

to take actions

  • Computer does not know the goal,

and not the language

  • User enters commands in their

language of choice

slide-15
SLIDE 15

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Outstanding paper award
  • User knows goal, but needs computer

to take actions

  • Computer does not know the goal,

and not the language

  • User enters commands in their

language of choice

  • Pragmatics: Modelling mutual

exclusivity

slide-16
SLIDE 16

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Features: cross product of

Rule Semantics Set all() Color cyan|brown|red|orange Color → Set with(c) Set → Set not(s) Set → Set leftmost(s)|rightmost(s) Set Color → Act add(s,c) Set → Act remove(s)

slide-17
SLIDE 17

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Features: cross product of
  • N-gram features of input

Rule Semantics Set all() Color cyan|brown|red|orange Color → Set with(c) Set → Set not(s) Set → Set leftmost(s)|rightmost(s) Set Color → Act add(s,c) Set → Act remove(s)

slide-18
SLIDE 18

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Features: cross product of
  • N-gram features of input
  • Tree-gram features of parses

Rule Semantics Set all() Color cyan|brown|red|orange Color → Set with(c) Set → Set not(s) Set → Set leftmost(s)|rightmost(s) Set Color → Act add(s,c) Set → Act remove(s)

slide-19
SLIDE 19

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Features: cross product of
  • N-gram features of input
  • Tree-gram features of parses
  • Log-linear model:

pθ(z|x) ∝ exp(θTφ(x, z)) Rule Semantics Set all() Color cyan|brown|red|orange Color → Set with(c) Set → Set not(s) Set → Set leftmost(s)|rightmost(s) Set Color → Act add(s,c) Set → Act remove(s)

slide-20
SLIDE 20

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • Features: cross product of
  • N-gram features of input
  • Tree-gram features of parses
  • Log-linear model:

pθ(z|x) ∝ exp(θTφ(x, z))

  • Gradient updates

Rule Semantics Set all() Color cyan|brown|red|orange Color → Set with(c) Set → Set not(s) Set → Set leftmost(s)|rightmost(s) Set Color → Act add(s,c) Set → Act remove(s)

slide-21
SLIDE 21

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • That was probably the most fun thing I

have ever done on mTurk.

slide-22
SLIDE 22

Learning language games through interactions

Sida I. Wang, Percy Liang, Christopher D. Manning

  • That was probably the most fun thing I

have ever done on mTurk.

  • Wow this was one mind bending

games [sic].

slide-23
SLIDE 23

Lifetime achievement award

  • Joan Bresnan, Stanford
slide-24
SLIDE 24

Lifetime achievement award

  • Joan Bresnan, Stanford
  • Student of Noam Chomsky
slide-25
SLIDE 25

Lifetime achievement award

  • Joan Bresnan, Stanford
  • Student of Noam Chomsky
  • Founder of Lexical Functional Grammars, LFG
slide-26
SLIDE 26

Lifetime achievement award

  • Joan Bresnan, Stanford
  • Student of Noam Chomsky
  • Founder of Lexical Functional Grammars, LFG
  • Supervisor of Chris Manning
slide-27
SLIDE 27

Lifetime achievement award

  • Joan Bresnan, Stanford
  • Student of Noam Chomsky
  • Founder of Lexical Functional Grammars, LFG
  • Supervisor of Chris Manning
  • “Out of the garden, into the bush

”

slide-28
SLIDE 28

Joan Bresnan ’s talk

“Out of the garden, into the bush ”

  • Data shock
slide-29
SLIDE 29

Joan Bresnan ’s talk

“Out of the garden, into the bush ”

  • Data shock
  • Inpsiration from neural network

researchers

slide-30
SLIDE 30

Joan Bresnan ’s talk

“The first shock was my discovery that universal principles of grammar may be inconsistent and conflict with each other ”

slide-31
SLIDE 31

Joan Bresnan ’s talk

passive optional: He hits him He is hit by him passive obligatory: *He hit me I am hit by him active obligatory: I hit him *He is hit by me

slide-32
SLIDE 32

Joan Bresnan ’s talk

“You don ’t know how difficult it is to find something which will please everybody—especially the men. ” “Why not just give them cheques?’ I asked. ” “You can ’t give cheques to people. It would be insulting. ”

slide-33
SLIDE 33

Joan Bresnan ’s talk

“What I hope to see going forward are increasingly powerful applications of computational linguistic theory, techniques, and resources to deepen our understanding of human language and cognition. ”

slide-34
SLIDE 34

Other interesting papers

  • Together we stand: Siamese

Networks for Similar Question Retrieval, Arpita Das, Harish Yenala, Manoj Chinnakotla, and Manish Shrivastava

  • Assisting Discussion Forum Users

using Deep Recurrent Neural Networks, Jacob Hagstedt P Suorra, Olof Mogren

slide-35
SLIDE 35

REPLNLP , Invited Speakers

  • Katrin Erk, University of Texas
  • Animashree Anandkumar, University of California Irvine
  • Hal Daumé III, University of Maryland
  • Raia Hadsell, Google Deepmind
slide-36
SLIDE 36

Raia Hadsell, Google Deepmind

  • lines are blurring. Examples:
  • Pixel Recurrent Neural Networks, Van

den Oord et.al. (ICML )

  • Conditional Image Generation with

PixelCNN Decoders, van den Oord

  • Progresive nets: transferring learning

from one task to another

slide-37
SLIDE 37

7

slide-38
SLIDE 38

And some more

  • Thorough examination of CNN/Daily Mail reading comprehention

task (outstanding paper), Danqi Chen, Jason Bolton, Chris Manning

slide-39
SLIDE 39

And some more

  • Thorough examination of CNN/Daily Mail reading comprehention

task (outstanding paper), Danqi Chen, Jason Bolton, Chris Manning

  • Diachronic Word Embeddings Reveal Statistical Laws of Semantic

Change, William Hamilton, Jure Leskovec and Dan Jurfsky

slide-40
SLIDE 40

Coming Seminars

  • Time?
slide-41
SLIDE 41

Coming Seminars

  • Time?
  • Neural Machine Translation Olof Mogren
slide-42
SLIDE 42

Coming Seminars

  • Time?
  • Neural Machine Translation Olof Mogren
  • Causality, Fredrik Johansson?
slide-43
SLIDE 43

Coming Seminars

  • Time?
  • Neural Machine Translation Olof Mogren
  • Causality, Fredrik Johansson?
  • Joan Bresnan’s work, Prasanth Kolachina?
slide-44
SLIDE 44

Coming Seminars

  • Time?
  • Neural Machine Translation Olof Mogren
  • Causality, Fredrik Johansson?
  • Joan Bresnan’s work, Prasanth Kolachina?
  • Mikael Kågebäck?
slide-45
SLIDE 45

Coming Seminars

  • Time?
  • Neural Machine Translation Olof Mogren
  • Causality, Fredrik Johansson?
  • Joan Bresnan’s work, Prasanth Kolachina?
  • Mikael Kågebäck?
  • Non-linear PCA/SVD/CCA, “later this fall”, Jonatan Kallus
slide-46
SLIDE 46

Coming Seminars

  • Time?
  • Neural Machine Translation Olof Mogren
  • Causality, Fredrik Johansson?
  • Joan Bresnan’s work, Prasanth Kolachina?
  • Mikael Kågebäck?
  • Non-linear PCA/SVD/CCA, “later this fall”, Jonatan Kallus
  • Suggestions, The audience