LSTMs Exploit Linguistic Attributes of Data Nelson F . Liu, Omer - - PowerPoint PPT Presentation

▶
lstms exploit linguistic attributes of data
SMART_READER_LITE
LIVE PREVIEW

LSTMs Exploit Linguistic Attributes of Data Nelson F . Liu, Omer - - PowerPoint PPT Presentation

Jan 08, 2023 309 likes •419 views

LSTMs Exploit Linguistic Attributes of Data Nelson F . Liu, Omer Levy, Roy Schwartz, Chenhao Tan, Noah A. Smith UWNLP LSTMs work well for natural language data Are they particularly well-suited for language? Testbed Memorization Task

slide-1
SLIDE 1

LSTMs Exploit Linguistic Attributes of Data

Nelson F . Liu, Omer Levy, Roy Schwartz, Chenhao Tan, Noah A. Smith

UWNLP

slide-2
SLIDE 2

LSTMs work well for natural language data Are they particularly well-suited for language?

slide-3
SLIDE 3

Testbed Memorization Task

  • Given a constant-length sequence of k inputs, recall the

identity of the middle token.

  • Task is inherently non-linguistic, inputs can be arbitrary

sequences.

[ g m d r p j w f h c ] [ 3 5 6 8 4 0 2 7 9 1 ] [ ! " # ↩ % & ' ( ) * ]

Target

slide-4
SLIDE 4

Linguistic Data Improves Memorization Performance

slide-5
SLIDE 5

Linguistic Data Improves Memorization Performance

slide-6
SLIDE 6

Linguistic Data Improves Memorization Performance

slide-7
SLIDE 7

Linguistic Data Improves Memorization Performance

slide-8
SLIDE 8

So, are LSTMs particularly well-suited for language?

Yes, more than uniform data or data with selected linguistic attributes

slide-9
SLIDE 9

LSTMs solve the task by counting

slide-10
SLIDE 10

More Questions

  • How does the LSTM use linguistic patterns in training?
  • What happens when you add more hidden units?

if you want to know more...come to our poster!