Constraint Based Models for ITS Sasikumar M Background Ohlssons - - PowerPoint PPT Presentation

Constraint Based Models for ITS

Sasikumar M

2

Background

• Ohlsson’s philosophy – refined over 1.5
• decades. University of Canterbury, New

Zealand is the primary home.

• Learning from performance errors.

– Declarative knowledge converted into procedural,

problem solving knowledge.

• A stage of error recognition, and then error

correction: Error recognition needs declarative knowledge

• Constraints capture the essence of this

knowledge to be learned.

3

Constraints

• Relevance condition – is the constraint

applicable here?

• Satisfaction condition – is it violated in this

case (assuming applicable)

– If violated, problem to be flagged. – Else, go ahead.

• Different constraints to check for different

aspects and types of knowledge.

• Specified in a simple pattern matching

language built around Lisp.

4

Constraints

• Broadly syntax constraints and semantic

constraints

– Syntax – properties of concepts – Semantic – relationships, domain knowledge, etc. – Syntax constraints easier to develop

• Granularity important – affects feedback.

– For pedagogic effectiveness, must focus on a

small aspect of the domain

5

Constraint example

6

Example constraints

7

….

• Constraints divide the set of solution states

into equivalence classes

– All states violating same set of constraints are in

• ne class
• Has same intervention options!

– All correct solutions in one class (violates zero

constraints)

8

Constraint based tutor

• No need for a problem solver, as in model

based tutoring.

• Instead of focusing on mistakes, focus on

constraints that all correct solutions must

• bey.
• Hence suited for ill-defined domains also.

– Planning, designing kind of problems – Programming, e.g.

9

Domain model

• Primarily constraints – derived and manually

created.

• Ontology defines the concepts.
• Problems and their ideal solutions – multiple

solutions ok.

• Solution decomposed into components which

are concept instances.

– Constraints can refer to these.

10

Ontology

• Common idea from AI literature, but different

definitions by different groups.

• CBM opts to use that to make the constraint

generation easier.

• Capture the concepts and their properties.
• Identify the key concepts, identify

important/relevant properties, and organise into a hierarchy/graph.

– Properties: min value, max value, type,

number of values, mandatory?, etc.

11

Another example

12

Matching

• Simple pattern matching with information on

ideal and student solution

• RETE algorithm for faster matching

– Repeating patterns matched only once

• A solution has from 70 to many hundred

relevance conditions satisfied.

13

Feedback Generation

• Feedback should tell:

– Where the error is – What constitutes the error – Re-iterate the domain principle violated by the

student

• Studies show such feedback is more effective.
• “Error is in the sum. Sum is given as 93,

should be 100, because the values are percentages.”

• Evaluation on the full solution, or on request.

– Make learner in control.

14

Feedback on solution

15

SQL-tutor

• A collection of databases, database of

problems, and a collection of constraints relevant to the domain.

– Some are problem specific constraints – Some are general domain constraints

• The first tutor based on the constraint based

modelling idea. Now many others.

• No problem solver here. Match the solutions
• r steps in a pre-planned way.

16

SQL-tutor

17

SQL-feedback

• Multiple levels:

– Correct/incorrect, error flag (where is error), hint – All errors, partial solution, complete solution

• Only first three as a routine – every submit

increments level.

• Last three only on explicit request.

18

Student model

• Every constraint has a percentage of correct

use – defines the student’s comfort.

• Problems requiring use of constraints in which

student is poor is selected –curriculum sequencing.

• Uses information only from last N problems –

to handle changing knowledge level, forgetting, etc

19

….

• Initial model was Overlay.

– Overlay in terms of constraints

• More complex models explored including

Bayesian networks

– Estimate probability of violating a constraint

based on student behaviour so far

• Used in problem selection: value of a problem

depends on predicted number of violated constraints.

20

Open student model

• Open student model

– Allow students to inspect the student model – Proper visualisation needed

• Bar-graph on knowledge-level of each constraint
• Ontology models?

– Support for OSM explored through SQL-tutor

21

Other examples

• EER-tutor: EER modelling
• CAPIT: English language

– For elementary school children – Uses probabilistic student model – Punctuation and capitalisation rules

• KERMIT: conceptual database design
• RDB normalisation – sequence of steps

22

…

• Collect UML – object oriented software design

– Design UML diagrams from textual description

• J-LATTE – for learning Java

– Concept mode – design the solution without code

• Pseudo code style, blocks like if-then, loop, etc.

– Coding mode – write the code

23

Scope

• Well defined problems with a clear correct

answer.

• Ill-defined problems with multiple correct

answers.

24

Authoring tool

• WETAS
• ASPIRE

25

ASPIRE

• Steps in developing a tutoring system

– Specify the domain characteristics – sub-

domains, problem solving steps, etc

– Compose the domain ontology – Model the problem and substructures – Design the student interface – Adding problems and solutions – Generate syntax and semantic constraints – Validate the constraints with expert – Deploy

26

Problem solving steps

• For addition of fractions
• Find the lowest common denominator (LCD)
• Convert fractions to LCD as denominator
• Add the resulting fractions
• Simplify the result

27

Ontology

Number whole-number Fractions LCD Improper Fraction Entities and their properties...

28

Model the problem

Step Soln component Concept Find LCD LCD LCD Convert fractions to LCD Fra1 numerator Improper fraction Fra2 numerator Fra1 denominator Fra2 denominator Sum the improper fractions Sum numerator Improper fraction Sum denominator Final reduced sum Final sum whole number Final sum numerator Final sum denominator Reduced fraction

29

Student interface

Lowest common denominator Fractions with LCD as denom + Sum of fractions Reduced sum

30

Use of ontology

• WETAS-ontology includes a tool to define the

domain ontology.

• From this many constraints can be auto-

generated to a large extent.

• For fraction, 11 syntactic constraints, and 39

semantic constraints generated, and used.

• 1:

– R: you are in step 1, computing LCD – S: LCD field must not be empty.

• 2:

– R: you are in step 1, computing LCD

31

Constraint generation

• General rules for concept:

– Concepts in student solution -> must also be in

ideal solution.

– Instances of each concept must match between

ideal and student soln

32

….

For features and values:

– For each selection concept, student has supplied

correct value.

– For each feature, if ideal solution has a value,

student must use that value.

– For each feature, if student has given value, ideal

solution must also have the same value (no extraneous things in solution)

– If a feature value is “required”, then it must be

same in student solution, and ideal solution.

33

Thank you…

Case Study: Design an ITS

Specification

• Teach conversion from active voice to passive

voice.

• He took a lecture -> a lecture was taken by him.
• I was rotis. -> Rotis were being made by me.
• Language: English
• Assume: vocabulary, basic language constructs

(verb, noun, etc), sentence structure, etc

Questions

• What is the teaching approach?
• Screen structure
• What kind of errors are possible?
• How to recognise the errors?
• What kind of feedback?
• Generation of exercises?
• What is the domain representation?
• Student model?

Transformation

He took a lecture A lecture him by was taken

Answers....