Natural Language Interaction Gurpreet Singh Papers Learning to - - PowerPoint PPT Presentation

Natural Language Interaction

Gurpreet Singh

Papers

• Learning to Parse Natural Language Commands to a

Robot Control System (Cynthia Matuszek, Evan Herbst, Luke

Zettlemoyer, Dieter Fox)

• Interpreting Instruction Sequences in Spatial Language

Discourse with Pragmatics towards Natural Human- Robot Interaction(Juan Fasola and Maja J Matarić, Fellow)

Paper 1

• Learning to Parse Natural Language Commands to a

Robot Control System (Cynthia Matuszek, Evan Herbst, Luke

Zettlemoyer, Dieter Fox)

Overview

T erminology

• RCL(Robot Control Language)
• CCG(combinatory categorical grammars)

Robot Control Language

CCG(combinatory categorical grammars)

http://yoavartzi.com/tutorial/

Phases of System Design

Applying CCG For Extracting Meaning

Experiments

• Navigation of the robot through four maps in simulation

(two for training, two for testing)

• The base training set contained the 189 unique

sentences generated from non-experts.

• The enriched dataset S(enr) contained 418 NL routing

instructions along with corresponding RCL instructions.

Experiments Cont.

• Maps A and B were simple(CS building, Laboratory research

lab)

• Maps C and D were relatively complex (Apartment building,

academic support building)

• For each of 10 difgerent sets (every combination of five

difgerent participants), 1,200 paths through map D were randomly generated: 1,000 short paths described by single NL sentences and 200 paths that required multiple sentences to express(5,onaverage)

Results

Discussion

• Was the data collected suffjcient?
• Were the results satisfactory ?
• What if the experiments performed in real world?
• Possible applications of this paper.

Discussion Contd…

• What did we like about this paper?
• What did we not like?
• Suggestions?

Paper 2

• Interpreting Instruction Sequences in Spatial Language

Discourse with Pragmatics towards Natural Human- Robot Interaction(Juan Fasola and Maja J Matarić, Fellow)

Introduction

• Complex Spatial NL language instructions provided by

the non-expert users. (e.g., “Put the spices in the top cupboard on the left hand side of the kitchen”, “Stow away all of the trash bags under the sink”, “Place the vegetables in the bottom shelf of the refrigerator”, etc.).

Inroduction Contd…

• The robot keeps the history of the previous commands

to resolve the referential expressions.

• Use of Unconstrained grammar.

Modules of the System

• Syntactic Parser
• noun phrase (NP) grounding
• semantic interpretation
• planning
• action modules

Semantic Module

• T

akes the verb, prepositions, grounding figures and reference objects as input.

• Uses Naïve Bayes over labelled training datasets and

produce three outputs.(Command type, DSR type, static spatial relation)

• Eg: robot movement, object movement, learned tasks,

etc

Planning Module

• Plan and Search for appropriate robot task solutions.

Algorithm

• Step 1: Parts of speech tagging.
• Step 2: Parsing parts of speech array using grammar.
• Step 3: Find the maximum probable sentence
• Step 4: Find Instruction sequence candidate
• Step 5: Find maximum probable instruction sequence

Resolving References

• People often refer to entities which have been previously used

like (He, she, it, that)

• Real world entities are represented by unique numerical

numbers internally.

• As the entities are referred their names are added to global list.
• Upon encountering the anaphoric expressions the entities in

recent list are used as candidate references.

Experiments

• 14 spatial language commands to the robot, two for

each static spatial relation analyzed (near, away from, between, inside, outside, at) and two additional test runs for the spatial relation at.

• After each experiment run, the end location of the robot

was measured against the goal position generated by

• ur spatial language interpretation framework.

eg: Stand between the printer desk and the whiteboard.

Experiments Contd…

Results.

Experiments Contd..

Challenges

• Safety of the human beings.
• T

ransfer of the discretized map to robot motor motion

Discussion

• Were the results satisfactory ?
• Possible applications of this paper.
• What did we like about this paper?
• What we did not like about this paper?

Thanks!

Suggestions,Questions?