and Progression Analysis and Design Shuhan Wang Fang He - - PowerPoint PPT Presentation

β–Ά
and progression analysis and design
SMART_READER_LITE
LIVE PREVIEW

and Progression Analysis and Design Shuhan Wang Fang He - - PowerPoint PPT Presentation

Dec 07, 2022 187 likes β€’541 views

A Unified Framework for Knowledge Assessment and Progression Analysis and Design Shuhan Wang Fang He Erik Andersen Source: Center for Game Science Level 3 Level 2 Level 1 N5 Students Knowledge Current Education System

slide-1
SLIDE 1

A Unified Framework for Knowledge Assessment and Progression Analysis and Design Shuhan Wang Fang He Erik Andersen

slide-2
SLIDE 2

Source: Center for Game Science

slide-3
SLIDE 3

N5

Level 3 Level 2 Level 1

Student’s Knowledge

slide-4
SLIDE 4

Current Education System

educator Python Python! content classroom feedback

Separate

slide-5
SLIDE 5

Our Unified Framework

Knowledge Assessment Learning Progression Analysis Framework Performance Prediction

slide-6
SLIDE 6

Our Unified Framework

Knowledge Assessment Learning Progression Analysis Framework Performance Prediction

slide-7
SLIDE 7

Knowledge Organization

Study the relationship between practice problems & Build the hierarchical structure.

slide-8
SLIDE 8

Partial Ordering on Practice Problems

π‘ž1 is at least as hard as π‘ž2 if: π‘‘π‘™π‘—π‘šπ‘šπ‘‘ π‘ž1 βŠ’ π‘‘π‘™π‘—π‘šπ‘šπ‘‘ π‘ž2

slide-9
SLIDE 9

Practice Problems

A B AB BCC AABC ABCBD D ABCB

slide-10
SLIDE 10

Partial Ordering Graph

A B AB BCC AABC ABCBD D ABCB

slide-11
SLIDE 11

Our Unified Framework

Knowledge Assessment Learning Progression Analysis Framework Performance Prediction

slide-12
SLIDE 12

Property of Partial Ordering

If π‘ž2 is at least as hard as π‘ž1, then

  • Students who understand π‘ž2 will also understand π‘ž1
  • Students who don’t understand π‘ž1 will not understand π‘ž2
slide-13
SLIDE 13

Coloring Partial Ordering Graph

slide-14
SLIDE 14

Coloring Partial Ordering Graph

slide-15
SLIDE 15

Coloring Partial Ordering Graph

slide-16
SLIDE 16

Coloring Partial Ordering Graph

slide-17
SLIDE 17

Coloring Partial Ordering Graph

slide-18
SLIDE 18

Coloring Partial Ordering Graph

Knowledge Boundary

slide-19
SLIDE 19

Knowledge Boundary

Knowledge Boundary (K.B.) : the set of the hardest problems that a student can understand. We use Knowledge Boundary to model a student’s knowledge within the Partial Ordering Graph.

slide-20
SLIDE 20

Our Unified Framework

Knowledge Assessment Learning Progression Analysis Framework Performance Prediction

slide-21
SLIDE 21

Rasch Model

𝑄 πœ„, 𝑐 = π‘“πœ„βˆ’π‘ 1 + π‘“πœ„βˆ’π‘

Student Performance 𝑄 is a function of the difference between the student’s ability πœ„ and the problem’s difficulty 𝑐. Student Ability πœ„ Problem Difficulty 𝑐 Rasch Model Unidimensional Numeric Scores Our Model Knowledge Boundary Node in Partial Ordering Graph

slide-22
SLIDE 22

Distance to Knowledge Boundary

In order to measure the difference between student ability πœ„ and problem difficulty 𝑐, We calculate the distance between Knowledge Boundary and the problem(node) in Partial Ordering Graph.

slide-23
SLIDE 23

Experiments: A Japanese Assessment Tool

First 10 sentences:

  • Students answered whether they can

understand those sentences.

  • The responses were used for

assessing students’ knowledge. Next 5-8 sentences:

  • Students answered how well they

understand those sentences.

  • The responses were used as the test

set for performance prediction

slide-24
SLIDE 24

Results

slide-25
SLIDE 25

Our Unified Framework

Knowledge Assessment Learning Progression Analysis Framework Performance Prediction

slide-26
SLIDE 26

When I have a β€œLibrary” of practice problems

Which Problem should I learn first? This is too hard!! Am I learning too fast? When should I review what I have learned? Learning Progression

slide-27
SLIDE 27

In order to automatically design learning progressions, we need to study expert-designed learning progressions.

slide-28
SLIDE 28

Progression Analysis on Textbooks

We are Looking for General Principles of designing good learning progressions.

Genki Standard Japanese

slide-29
SLIDE 29

Progression Metric: Learning Pace

A student’s πΏπ‘œπ‘π‘₯π‘šπ‘“π‘’π‘•π‘“ 𝑇𝑗𝑨𝑓 is number of problems π‘ž s.t. the student has learned π‘ž or some other problem that is harder than π‘ž. 𝑄𝑏𝑑𝑓 = βˆ†πΏπ‘œπ‘π‘₯π‘šπ‘“π‘’π‘•π‘“ 𝑇𝑗𝑨𝑓 βˆ†π‘’π‘—π‘›π‘“ Both textbook progressions are following a similar, steady pace.

slide-30
SLIDE 30

Progression Metric: Balance of Learning and Review

Problem Knowledge Classification 1 A Introduction 2 B Introduction 3 BC Introduction 4 A Reinforcement 5 C Reinforcement 6 ABC Recombination We classify problems in a learning progression into Introduction, Reinforcement and Recombination.

slide-31
SLIDE 31

Progression Metric: Balance of Learning and Review

slide-32
SLIDE 32

Future work

  • Apply to Different Educational Domains
  • Especially Computer-Assisted Language Learning (CALL)
  • A Science of Progression Analysis
  • Pacing and Sequencing: Find the Best Principles.
  • Automatic and Adaptive Tutoring System
  • Rapid Initial Assessment
  • Progression Tailoring
slide-33
SLIDE 33

Summary

  • Organizing Practice Problems into Partial Ordering Graph
  • A hierarchical structure of knowledge
  • Knowledge Assessment within Partial Ordering Graph
  • Knowledge Boundary
  • - student modeling
  • Distance to K.B.
  • - performance prediction
  • Analyzing Learning Progressions from Textbooks
  • Learning pace
  • Balance of Learning and Review
slide-34
SLIDE 34