Student Model Sasikumar M IIT Bombay Overview What is SM and why - - PowerPoint PPT Presentation

IIT Bombay

Student Model

Sasikumar M

IIT Bombay

Overview

What is SM and why SM? Types of SM Relation to Domain models Ways to capture and update

– Examples

Open student model

IIT Bombay

Why Student Model?

Personalised Adaptive Tutoring Student Model

IIT Bombay

...

Student models needed for

– Determining help during problem solving – Diagnosis of errors – Choice of teaching strategy – To intervene or not – To give hint or not – Choose type of hint – Etc...

Note: SM depends on other modules...

IIT Bombay

Purpose

Short term immediate response Adapt media and resources Long term curriculum planning Enhance learner confidence Provide a sense of challenge Build curiosity

Affective Information

IIT Bombay

Contents of SM

A SM can contain a wide variety of information, in general:

– Related to knowledge of what is being taught – Other relevant background knowledge – Emotional profile (motivation, attention level, emotional state, etc) – Learner style – Cultural parameters (incl language)

IIT Bombay

Issues

Granularity Dependency on other modules

– No point in collecting information you cannot use! – What you collect must be usable by others.

Nature of representation Purpose and mode of use Change over time etc

IIT Bombay

Student Model

Short term vs long term model Single observation vs multiple observation Cost of building/updating vs utility

– Time and complexity

IIT Bombay

Classification

Many classifications based on various aspects Based on type of knowledge used by the system

– If-then rules – Semantic networks – Ontologies – Concept maps – And so on

And type of task performed

– Concept learning – Rules and constraints – Procedures

SM can usually be seen as an annotation on the domain model..

IIT Bombay

Domain Types

Problem solving domains – maths, physics, trouble shooting

– Relatively well understood to design SM

Analytic and unverifiable domains – law, ethics

– Empirical verification not usually possible

Design domains – architecture, music.

– Most complex and ill-structured – Hardest to define SM

IIT Bombay

Classification...

Based on knowledge in SM

– Models with course knowledge – Models with individual characteristic independent of course

• Learning style
• Personality traits
• mood

Most of the focus on “course knowledge”

– Dynamic update needed – Some interest in learning style for adaptive instruction.

IIT Bombay

Knowledge Models

Just a number 0-100? (eg. Exam score in the subject)

– A vector of numbers, for each major topic?

Overlay for further granularity Bug libraries Genetic models Technologies

– Bayesian networks – If-then rules – Semantic nets

IIT Bombay

Overlay model

Subject is a set of independent pieces. A value attached to each piece. Student knowledge subset_of expert knowledge

– Hence “overlay”, “cover”, etc.

IIT Bombay

Overlay model

Expert knowledge Student Knowledge

IIT Bombay

... problems

But, student knowledge not a subset of teacher knowledge

– Misconceptions – Wrong procedures – Dependencies

Bug libraries

– Perturbation models

IIT Bombay

Bug libraries

Concept of bug

– Enumerated? – Whatever is not right? – Concept, step, application of a concept/rule – Full enumeration difficult except in very small domains.

Perturbation approach

– Perturb the right way at each node/step – The modifications – bugs

IIT Bombay

Formulating bugs

Bugs in rules

– Wrong conditions – Wrong conclusions – Not used

Bugs in component

– Wrong component – Not connected properly – Not working correctly

Bugs in procedure

– Delete a part – Add incorrect part – Replace a correct part with an incorrect part.

IIT Bombay

...

Scoping required:

– Bugs can arise in everything, everywhere – Error in reading, faulty meter, ... ??

Handcompile from analysis of thousands of case logs. Collect from systematic analysis of domains. Theoretical perturbations may not be seen much in practice. Prioritisation of bugs.

IIT Bombay

... problems

Student knowledge evolves with time

– Levels of understanding – Refined with more precision and detail with time – Bug models change with these refinements

Layered student models

– Genetic graph models

IIT Bombay

Genetic Graph Models

Levels of knowledge, with own representation and models

– Node: rules, facts, etc – Links: evolutionary relationships

• Analogy
• Simplification
• Deviation
• ...

Levels interlinked to capture relationships and dependencies. Learner evolves through this! Curriculum Information Network (CIN) of DesignFirst.

IIT Bombay

More detailed SM ...

Build simulations to capture more behaviour Type of tutor

– Model tracing tutor – Constraint based model

AI techniques

– Formal logic – Bayesian networks – Expert system models

IIT Bombay

Model tracing tutor

Model tracing tutors traces through internal model to keep pace with the learner. Mostly for procedural problem solving tasks. Create a solution path, and augment with “wrong paths”. CTAT is a framework on this notion.

IIT Bombay

...

If-then rules or a graph as the knowledge representation. Probabilistic rule and bayesian network used for SM. Model's step need to be induced from the student's actions. Probabilities induced from student behaviour. If student is following steps correctly, he knows all the rules along that path.

IIT Bombay

...

Rules and misconceptions If the goal is to solve a(bx+c) = d, then a) rewrite as bx+c = d/a b) rewrite as abx + ac = d c) rewrite as abx + c = d

IIT Bombay

...

For each rule, count:

– Correct use – needed and used – Wrong use – not needed, used – Needed, and not used

Used to define “knowledge level” on the rule.

IIT Bombay

Constraint based Model

Set of constraints is the knowledge. Solution must not violate any; probabilities on the constraint. Ordered pair: relevance and satisfaction

• - relevance: is the constraint applicable
• - satisfaction: what is the constraint

When adding fractions (a/b, c/d), numerators can be added, when the denominators are same (b == d). Driven by Ohlsson – error based learning theory.

• - People learn when they make mistakes.

IIT Bombay

....

CBM evaluates knowledge of the student, and does not generate. Hence no need to induce strategy of student. Student can take any path, as long as constraints are maintained. It is pedagogy neutral – not dependent on any strategy. Easier to build SM; but feedback may not be as

• sharp. When multiple solution paths are possible,

partial solution correctness not easy to check.

IIT Bombay

CBM for knowledge state

Constraint help to detect nature of error and provide feedback. To estimate knowledge of topic, a constraint-concept tree is made. Constraints are leaf nodes; concepts, sub- topics, and topics as we go up. All children “known”, parent “known”.

IIT Bombay

Types of knowledge

Domain knowledge – propositions, concepts Reasoning knowledge – rules etc allowing to connect the pieces in DK. Monitoring knowledge – when to use the RK; strategy dependent. Reflective knowledge – problem solving process itself. Use of hints, etc

IIT Bombay

Technologies for SM

If then rules Bayesian networks Fuzzy rules Logic programming

IIT Bombay

... if then rules

Rules as pieces of knowledge Can also be “wrong” knowledge – misconceptions Tag rules as “known”, “unknown”, etc If rules are not well defined, explaining error may be hard [Guidon] Generating aggregate status?

IIT Bombay

... Bayesian networks

Model based on probability theory,

– Specifically Bayes' theorem

Nodes are knowledge units, links indicate causality. Joint probability distribution as product of conditional probability of all nodes

– Dependent on causal nodes

Formal model Impact of prior probability

– Some studies “not critical”, difference (change in prob) important.

[Detailed talk later...]

IIT Bombay

Fuzzy rules

Knowledge level as a fuzzy variable StatLady uses: remedial, intermediate, and mastered.

– Each in turn has low and high

mastered(x) – fuzzy variable Rules to increase and decrease membership value.

IIT Bombay

Logic programming

Domain knowledge + student model -> student behaviour +: as modified by Where student model has info, that over- rides what is in the DK.

IIT Bombay

Blame assignment

Finding responsible knowlege piece for a wrong answer is difficult.

– When answer is correct, all contributors get “+” support. – When answer is wrong, reduce all?

• Too much noise

– Use domain knowledge and relations among pieces to narrow down choices

• Worked well in BIP-II, WISOR, etc

IIT Bombay

Change over time

State of knowledge changes with time

– Forgetting, weakening – Due to time change, due to arrival of new knowledge (confusion)

Most systems do not address this aspect. In SQL-tutor, last four logs only checked to evaluate knowledge.

IIT Bombay

Wrapping up...

Student model key to effectiveness of ITS in most cases and scenarios Various approaches with associated + and -. Must match and go with Domain model. Little work on monitoring and reflective knowledge. ...[A few related topics to be covered later].

IIT Bombay

Thank you..

IIT Bombay

Updating SM

Based on analysis of student actions Based directly on student action Self updating models

IIT Bombay

...

Primary source is student His responses to questions, special assessment, etc Also additional rules

– A needs-knowledge-of B, and you know A – A more-complex-than B, and you know A

We can also ask the student to directly update his SM.

IIT Bombay

Open Student Model

SM to be visible for edits by the learner. He may be able to correct the model.