Communication Knowledge Sasikumar M Overview Communication key to - - PowerPoint PPT Presentation

Communication Knowledge

Sasikumar M

Overview

• Communication key to tutoring
• Different modes impose different benefits and

challenges

• Scaffolded inputs and generated text (in audio

also) often used

• Brief look at VR, and language

Synthetic humans

• 3D human models
• Add speech input and output
• For body language/gesture
• Communication skills
• Marketing – real estate

VR

• NASA heavy investment
• Eg. training for living in outer space

– Simulate lack-of-gravity effect

• Students for atomic/molecular space, and high

speed phenomenon.

• Psychiatric treatment

Graphics techniques

• Facial animation for emotions
• Virtual characters (artificial life)

Social intelligence

• Human emotions integral to human existence.
• If a tutor is adaptive enough for a student to

believe that he is interacting with a human, then it is “socially intelligent”

• Affect and mood recognition important, and to

act on it.

• Detect boredom, doubt, motivation level, etc

SI: approach 1

• Behavioural analysis

– Problem solving time – Nature and number of mistakes – Help request

• Bayesian network – to convert the pieces into useful measure
• Can help infer, with machine learning

– To decide on current mood – Motivation – Engagement

• One target for educational data mining

SI: approach 2

• Visual recognition
• Camera based – stationary, and moving views

– Eye-brow position – Head movement – nod, shake

• eye-tracker
• Posture sensing devices
• Often head orientation itself provides a lot of

cues on “focus”

SI: approach 3

• Metabolic indicators
• Skin-conductance, heart rate
• Pressure sensors – on keyboard, mouse, etc
• Speech cue based recognition

Language

• Easiest to use; can be laborious if typing, etc
• Ideal – speech input and speech output

– With gestures and intonation

• Many challenges still on this.

Language

• Types

– Mixed initiative

• Both system and user can drive the interaction
• Can seek clarification to synchronise the mental models of both.
• Need to handle turn taking, grounding, and evaluation of input.

– Single initiative (system)

• User only answers questions to the point

– Single initiative (user)

• “I want to go from Boston to Miami this weekend”

– Directed

• Focussed, system driven, restricted dialogue

Issues

• Language generation

– What to say when and how.

• Language understanding

– What is said and meaning what

• Complexity of language

– Structure, ambiguity, etc

• Mixed initiative is hard – role management, turn taking,

information extraction

• Use with other technologies

– Facial features, audio, intonation, gestures, etc

Language generation

• Dear $name, you are now working on exercise

$ex of chapter $c

• As explained before, gravity is a force in the

downward direction.

• These are simple template filling. Can we do

concept to language?

• Explain a concept map or an equation?

Eliza

• Among the earliest natural language communication

system

– I am feeling very depressed? – Why are you feeling depressed? – My mother scolds me everday. – Who else scolds you? – My father does not like me. – Tell me more about your father. – He does not care. – Tell me more.

Eliza

• Simple pattern matches can go a long way

(apparently)

• No learner model, domain model, etc
• Multiple options for a pattern, to sound more

natural.

Eliza...

– I am feeling very depressed? – Why are you feeling depressed? – My mother scolds me everday. – Who else scolds you? – My father does not like me. – Tell me more about your father. – He does not care. – Tell me more.

Geometry tutor

• They are 180.
• Could you be more specific? What are they?
• The angles are 180.
• You are heading in the right direction. But

more precision is needed. Where does 180 come from? Are all angles 180?

...

• The angles in a triangle are 180.
• Is each angle measure equal to 180?

Precisely what is 180?

• The sum of angles in a triangle is 180.
• Correct.

Ms Lindquist

• Please write an expression for the total time

taken for the trip.

• HELP!
• Let me ask you an easier question. Please

compute the time actually on the bike, if the speed was 20 miles per hour, the break time was 2 hours, and the distance travelled was 100 miles.

• 7

...

• Correct. What math did you use to get it?
• 2+5
• Okay, how did you get 5?
• 100/20
• Correct. How did you get 7?
• 2+100/20
• Correct. Can you use the variables given in the problem, instead
• f the numbers?
• b+m/s.
• Correct.

Review

• What weakness and nice things do you see?
• What does it take to implement such

capabilities?

• What kind of knowledge is involved?

Auto-tutor

Language interaction

• Imagine the types of knowledge and

processing involved in realising such interaction

• Domain model – words, meanings, etc
• Student model – current context, previous

sentences

• Discourse knowledge
• Social psychology

Processing NL

• Syntax

– Nouns, verbs, inflections

• Semantics

– Meaning and content

• Pragmatics

– Contextualised meaning

• Discourse

– Multi-sentential text, turn taking, etc

Language interaction

• Highly interdisciplinary

– Computer science – Cognitive science – Social psychology – Discourse processes – Etc

• Advantage

– Restricted domain, ambiguity will be less.

• Negative

– Students may not follow grammar well!

Syntax Processing

• Tokens – words – constituents – parse tree
• I had dinner with him on Sunday; he left for London then.
• Connect R32 across V1 to V2, in series with R15.
• Word structure

– Inflections -> morphology -> analysis and synthesis – Enjoyable = enjoy + able – Misunderstanding = mis+understand+ing – mis+un+? mis+under+stand+ing

...

• Word category (part of speech – POS)

– Pos-tagger – I always bank on you. – Where is the axis bank? – The bank of Yamuna is so dirty.

• Sentence structure

– PP attachment – Phrase nesting – Complex sentences with/without connectives – I saw the boy on the hill with a telescope.

Parse Tree

S Subj VP NP Art Noun PP The bank

• f

Yamuna verb adv is dirty Tags and structure only indicative

Parsing

• Objective to get some handle on the structure
• f the sentence
• Deep parsing – full parse tree.

– Usually very hard, in general

• Shallow parsing – chunks and broad structure.

Semantic processing

• Identifying the entities referred to, and their

relationship with other entities.

• What is the information in the utterance?

Correctness?

• What does he want?
• Need domain model, world knowledge, etc

Resources

• Domain ontology – concepts, key terms,

relationships, etc

• Wordnet – general relations among words

– Synonym, generalisation/specialisation, etc

• POS taggers and chunkers for many

languages.

• Latent Semantic Analysis

ITS Student model Architecture Domain Model Tutor Model Model Based tutor Dialogue Based Tutor Constraint Based Tutor Buggy Model Overlay Model Perturbation approach has isa isa has

Meaning Representation

• We need meaning to map to the domain

model?

– Locate relevant rules, concepts, etc – And what processing is needed to answer/meet

the expectation.

Latent Semantic Analysis

• Words carry and represent the unit of meaning.
• The co-occurrence of words also significant.
• How to represent a document.
• Word-document mapping -> matrix
• Reduce dimension of this, with minimal loss of information.
• LSA – reformulated vectors derived from this.
• Used heavily in text matching (document retrieval etc)

Pragmatics processing

• Common conventions, and expectations.
• Is there a way?
• Can you tell me the time?
• How are you?
• In ITS, this can mean intervention models,

nature of hints, etc.

Discourse Processing

• Multi-sentence text with sentence to sentence

information transfer.

– Lal went to Dubai for shopping. He likes that

place.

• Dialogues

– Turns, different world models, etc – Additional language components in this case.

Discourse processing

• Insert/recognise discourse markers

– By the way, – Did you notice that... – Let us come back to ...

• Respond—Lead on –Invite

– “Correct. But will this work? What do you say?”

• Anaphora resolution

– Where do you put that?

• General protocols.

– Hmm, hi, yes, ok, etc.

Dialogue based tutoring

• Good for inquiry teaching domains.
• Usual classroom pattern:

– Initiation Response Evaluation

• Extend to: collaboratively improve the answer,

and Evaluation again.

• This can be teaching model.

Dialog moves

• Positive immediate feedback (yeah, right, etc)
• Neutral immediate feedback (hmm)
• Negative immediate feedback (no, not quite, etc)
• Pumping for more (tell me more.. what else)
• Prompting (this is K and ?)
• Hinting
• Elaborating
• Correcting
• Summarising
• Requestioning

Internal models

• Bayesian networks of concepts and

probabilities.

• Latent semantic analysis – gathering meaning

from the word fragments.

• Full generation of dialogue, not practical.

– A combination of intelligent pattern match, and a

number of tagged alternative responses (.. eliza)

Auto-tutor

• Attempt to imitate a non-expert tutor in style.
• Dialogue based tutoring.
• Domain: computer literacy course

...

• Modules:

– Curriculum script

• Representations for each lead question, possible answers, their variants, expected bad

answers, corrections, etc

– Language extraction – Speech act classification

• Classify input into assertion, WH question, yes/no question, directive, short response, etc
• NN used here.

– Latent semantic analysis

• Find the best match

– Topic selection – Dialog move generation – Talking head

Wrapping up

• User interface is critically important in ITS.
• Technologies such as virtual humans and VR

provides interesting avenues.

• Natural language + speech is ideal target

(zero learning curve), but hard to achieve.

• A lot more intensive knowledge capture and
• rganisation is needed.

– Dialog based tutoring

Ahead in the course

• Bayesian networks
• A bit of machine learning
• Personalised instruction and learning styles
• Learning theory?
• More case studies?
• More domain stories?
• ITS evaluation
• Others?

Thank you...