TOWARDS A BASIS FOR HUMAN-COMPUTER DIALOGUES Bengt Nordstrm, - - PowerPoint PPT Presentation

TOWARDS A BASIS FOR HUMAN-COMPUTER DIALOGUES

Bengt Nordström, Computer Science, Chalmers

Computational Logic Workshop 18 - 19 Nov 2011 St Andrews, Scotland

• A simple model of human-computer

dialogue

• Stenius garden game
• A simple model of human-human

dialogue

Overview

EXAMPLE OF A HUMAN-COMPUTER DIALOGUE

A travel agency: plane rental car hotel boat When you click on one of these you get some other alternatives. plane from: to: nr of persons: leaving date arrival date hotel where: nr of persons: leaving date arrival date

In both these cases the answer to one question decides the shape of the rest of the dialogue:

• If you know that you are talking about ordering a

hotel, then it doesn't make sense asking about the departure city.

• The number of days in a month depends on the

month and the year. So, the shape (type) of something depends on the value of something else.

COMPARE THIS WITH A HUMAN- HUMAN DIALOGUE:

A: What do you want to do? eat sleep work swim sail ... B: I want to eat A: Do you want to eat at home? restaurant home your place ... B: No, in a restaurant. A: Fine, I would like to eat in a scottish restaurant japanese swedish french scottish ... B: What about haggies? A: OK

The result of the dialogue is the construction of a mathematical object:

eat(restaurant, (scottish, haggies))

An essential part of the dialogue is the possibility of changing the object being constructed:

An essential part of the dialogue is the possibility of changing the object being constructed:

A: I want to eat Swedish food B: OK, let’s have surströmming at home

An essential part of the dialogue is the possibility of changing the object being constructed:

A: I want to eat Swedish food B: OK, let’s have surströmming at home

We have now changed the object:

eat(restaurant, (scottish, haggies) eat(home, surströmming)

Example of a datatype for a dialogue:

Place = data home restaurant forest : Place Food : Place -> Set Food home = SwedishFood Food restaurant = (x : Nationality, NationalFood x) NationalFood : Country -> Set NationalFood japan = JapaneseFood NationalFood sweden = SwedishFood NationalFood scotland = ScottishFood Food forest = Forestfood Action = data eat : (where : Place) (food: Food(where)) -> Action sleep: Place -> Action swim: From -> To -> Manner -> Action work: Place -> What -> Action sail : From -> To -> Passengers -> Action

SOME WORDS TO BE EXPLAINED:

• Syntactical correctness of a dialogue
• Top-down vs. bottom-up dialogue
• User driven vs. system-driven dialogue
• sequential vs. random access

GOAL

We want to model dialogue systems in which a human interacts with a computer to build an

• bject.

From Wikipedia: A dialogue system is a computer system intended to converse with a human, with a coherent

• structure. Dialogue systems have employed text,

speech, graphics, haptics, gestures and other modes for communication on both the input and output channel.

A DIALOGUE IS SEEN AS THE BUILDING OF AN OBJECT.

The human fills in information which the computer needs. Typical examples are:

• Reservation system for trains, concerts, calendars, rooms, medical

doctors, ...

• Navigation systems
• Control center for TV, radio, mp3-players, ...
• Editors for controlled languages like
• restaurant reviews
• recipies

TYPICAL STRUCTURE OF A DIALOGUE SYSTEM

speech input | textual input | syntax tree | semantic representation | system response content | system response utterance | system output speech recognition parsing semantic interpretation dialogue management generation speech synthesis

CHECKING CORRECTNESS

In all these systems there is a notion of syntactic correctness, which should be checked immediately after each input of the user. We will use a dependent type system to express the syntactical correctness.

SOME CHOICES:

• System driven dialogue: The system decides the
• rder to fill in the details.
• User driven dialogue: The user decides.
• sequential access: up, down, left, right...
• random access: any part of the object can be

changed.

Objects are treated as directed graphs

In principle, we have a graph like: where p, q and r are placeholders and c are functional constants.

q1 = c1 p1 . . . pn . . . qn = cn r1 . . . rm

SYSTEM DRIVEN DIALOGUE

The commands to the system is a list of (functional) constants: c1, …, cn each command replaces the leftmost placeholder. Here, each constant has a fixed arity

USER DRIVEN DIALOGUE

The commands to the system is a list of commands

• f the form:

q = c q1 … qn The command replaces the placeholder q.

The commands to the system is a list of commands of the form: q = c q1 … qn Bottom-up: The placeholder q is new. Top-down: The placeholder q is already in use. Sharing: Some of the placeholders qi are in use.

A SIMPLE EXAMPLE:

We want to define the constant 2 : Nat and we assume that we have the constant 0 : Nat and s : Nat -> Nat: two :: Nat -- declare the type of two two := s q1 -- the system can deduce that q1 : Nat q1 := s q2 -- the system can deduce that q2 : Nat q2 := 0

The set of expressions are:

Thick expressions e,t :: = t -> t' | (t, t’) | e, e’ | (c e) | q | c function types cartesian product pair application place holder constant

The set of expressions are:

Thick expressions e,t :: = t -> t' | (t, t’) | e, e’ | (c e) | q | c function types cartesian product pair application place holder constant Thin expressions e,t :: = q -> q' | (q, q’) | q, q’ | (c q) | q | c

The set of expressions are:

Thick expressions e,t :: = t -> t' | (t, t’) | e, e’ | (c e) | q | c function types cartesian product pair application place holder constant Thin expressions e,t :: = q -> q' | (q, q’) | q, q’ | (c q) | q | c Each thick expression can be represented as a list of thin expressions

OVERVIEW

Starting from a state with expressions with place holders, like: c1 : t1; c1 = q1; ...; cn : tn we want to use a series of commands to build up a new state c1 : t1; c1 = e1; ...; cn : tn; c1 = en It is not necessary that all constants have a definiens. But in the end all holes must be filled in (place holders must be defined).

APPLICATIONS

A way to structure a dialogue system: A generic editor (handling objects with dependent types) works as a framework for different dialogue systems. Y

• u can specify a dialogue system by
• a list of definitions of types and objects
• the concrete syntax of each constant
• the type of the object which is to be constructed

The dialogue is then controlled by typechecking

FURTHER WORK:

Extend the language of objects and types to:

• dependent types
• some notion of a decidable subset type
• strategies for under-specified information (the system cannot

deduce what placeholder to fill in) (Peter Ljunglöf)

• specification of concrete syntax

• A simple model of human-computer

dialogue

• Stenius’ garden game
• A simple model of human-human

dialogue Overview:

STENIUS’ GARDEN GAME

• There is a garden, a gardener and her assistant
• a, b, c, d are places for flowers
• Pa means that a is in flower
• Qa means that a is not in flower

1. You learn the game by writing a sequence Pa Qb … for each day.

• 2. Simple gestures are used to show that one understands the

game

• 3. When you understand this game, you can also play the

Report Game

THE REPORT GAME

• There is no need for the gardener to be in the garden
• The assistant reports the situation in the garden by a

sequence Pa Qb ...

• and the gardener understands the situation in the garden

THE COMMAND GAME

The gardener can now write instructions to the assistant: Pa, Qb, Pc, … and these are commands to the assistant to make sure that a shall be in flower etc.

THE COMBINED GAME

The two games can now be combined: ! (P a) = Make sure that a is in flower! T (P a) = It is true that a is in flower ? (P a) = Is a in flower?

Stenius analyzes a sentence according to M P where M is the mood (force) and P is a proposition (content) M can be !, ? or T

• A simple model of human-computer

dialogue

• Stenius’ garden game
• A simple model of human-human

dialogue Overview:

THE KITCHEN GAME

Imagine a number

• f persons in a

kitchen They read, eat, cook and talk.

The talk in the kitchen consists of a list of sentences of the shape: a1 -> b1 : M1, a2 -> b2 : M2, … a -> b : M stands for that the person a says M to b

THE KITCHEN GAME

THE KITCHEN GAME

Propositions are built up by applying a verb to its arguments: eat ( who= Martin,

what = spaghetti, where = Göteborg, when = 3 March 2011, how = sloppily, why = Martin is hungry)

Other verbs are: walk, read, cook, know

A sentence will be analyzed a la Stenius: M P , where M is the mood and P is a proposition M can be a question, command or assertion

THE KITCHEN GAME

Different kinds of questions:

w-questions (completion): Who eats spaghetti?, where is Martin?, who reads?. The types of these are: O -> Prop, e.g. [x]eat ( who = x, what = spaghetti), A reply to such a question Q can either be

• long: Q(Martin), i.e. Martin eats spaghetti), the function

applied to an arguments such that the resulting proposition is true

• or short: Martin an argument which makes the

application true.

THE KITCHEN GAME

Yes-no - questions (decision) Does Thea eat spaghetti? The type of these are Prop, e.g. eat ( who = Thea, what = spaghetti) A reply to such a question Q is either long: Yes Q or No Q (Thea does not eat spaghetti), i.e. the proposition decorated with yes or no

• r short:

Yes or No

Different kind of questions: Disjunction questions: Does Thea eat spaghetti or hot dogs? The type is Prop x Prop Enumeration questions: Who eats? Have the same type as completions, i.e. Obj -> Prop but a reply to such a question Q consists of an enumeration

• f all objects a1, …, an such that Q (ai) is true.

THE KITCHEN GAME

The general structure of an utterance is: a -> b : Y The person a says to b that Y This is particularly important for commands and assertions.

THE KITCHEN GAME

Command: a -> b : !(P , c) The person a says to b that c shall make P become true c = a : commission: The speaker commits to do something c = b : directive: A direct order to someone

THE KITCHEN GAME

Assertion: a -> b : T(P) The person a says to b that P is true The proposition P can say something about the world or about a or b. This is a correct statement if P is true

QUESTION:

Is it possible to get some inspiration for human-computer dialogues by analyzing human-human dialogues?

THANKS TO:

• Aarne Ranta, Computer Science, Chalmers
• Björn Bringert, Google, London
• Peter Ljunglöv, Computer Science, University of Gothenburg
• Robin Cooper, Linguistics, University of Gothenburg
• Erik Stenius: Mood and Language Game, Synthese 17:254-274

• Issa

Snail!

• Issa

Snail! Little by little climb up -

• Issa

Snail! Little by little climb up -

• Mt. Fuji
• Issa