SLIDE 1
Outline
2
Language grounding Visual dialogue Appendix: Current and future work
A
Q
A Q
Visually Grounded, Task-oriented Dialogue Elia Bruni Outline - - PowerPoint PPT Presentation
Visually Grounded, Task-oriented Dialogue Elia Bruni Outline - - PowerPoint PPT Presentation
Visually Grounded, Task-oriented Dialogue Elia Bruni Outline Language grounding Visual dialogue Q A Appendix: Current and future work Q A 2 Distributional semantics We found a cute, hairy wampimuk sleeping behind the tree 3
SLIDE 2
SLIDE 3
Distributional semantics
3
We found a cute, hairy wampimuk sleeping behind the tree
SLIDE 4
Distributional semantics
4
We found a cute, hairy wampimuk sleeping behind the tree
12 8 5 1 hairy cute tree money dangerous wampimuk
SLIDE 5
Distributional semantics
5
We found a cute, hairy wampimuk sleeping behind the tree
12 8 5 1 hairy cute tree money dangerous animal wampimuk dog money account wampimuk
SLIDE 6
6
Bruni et al., Distributional Semantics in Technicolor. 2012
The grounding problem
SLIDE 7
Language grounding
7
Bruni et al., Multimodal Distributional Semantics. 2014
SLIDE 8
Language grounding
8
Bruni et al., Multimodal Distributional Semantics. 2014
Enrichment of pure textual vectors with complementary information coming from perceptual visual features.
SLIDE 9
Language grounding
9
Bruni et al., Multimodal Distributional Semantics. 2014
Enrichment of pure textual vectors with complementary information coming from perceptual visual features.
SLIDE 10
Language grounding
10
Bruni et al., Multimodal Distributional Semantics. 2014
Enrichment of pure textual vectors with complementary information coming from perceptual visual features.
- Word meaning not learned in isolation
but from usage
SLIDE 11
Language grounding
11
Bruni et al., Multimodal Distributional Semantics. 2014
Enrichment of pure textual vectors with complementary information coming from perceptual visual features.
- Word meaning not learned in isolation
but from usage
- Missing the interactive and dialogical
aspects of language
SLIDE 12
12
Visual Dialogue
SLIDE 13
13
A man and a woman are holding umbrellas
A
Visual Dialogue
SLIDE 14
14
What color is his umbrella? A man and a woman are holding umbrellas
A
Visual Dialogue
SLIDE 15
15
What color is his umbrella? A man and a woman are holding umbrellas
A
Visual Dialogue
SLIDE 16
16
What color is his umbrella? A man and a woman are holding umbrellas
A
Visual Dialogue
SLIDE 17
17
His umbrella is black
A
What color is his umbrella? A man and a woman are holding umbrellas
A
Visual Dialogue
SLIDE 18
18
His umbrella is black
A
What color is his umbrella? What about hers? A man and a woman are holding umbrellas
A
Visual Dialogue
SLIDE 19
19
His umbrella is black
A
What color is his umbrella? What about hers? A man and a woman are holding umbrellas
A
Visual Dialogue
SLIDE 20
20
umbrella hers
A A
Visual Dialogue
What color is his umbrella?