Outline Introduction Hypothesis testing (pre-study) Findings and - - PowerPoint PPT Presentation

▶
outline
SMART_READER_LITE
LIVE PREVIEW

Outline Introduction Hypothesis testing (pre-study) Findings and - - PowerPoint PPT Presentation

Sep 14, 2022 451 likes •837 views

University of Pittsburgh Intelligent Systems Program Outline Introduction Hypothesis testing (pre-study) Findings and challenges JavaParser Conclusion &Future works AIEDCS 2013 2 Introduction There are two possible ways

slide-1
SLIDE 1

University of Pittsburgh Intelligent Systems Program

slide-2
SLIDE 2

Outline

  • Introduction
  • Hypothesis testing (pre-study)
  • Findings and challenges
  • JavaParser
  • Conclusion &Future works

2 AIEDCS 2013

slide-3
SLIDE 3

Introduction

There are two possible ways for modeling student’s knowledge:

  • Coarse-grained knowledge modeling
  • Fine-grained knowledge modeling

3 AIEDCS 2013

slide-4
SLIDE 4

Motivation

Fine-grained indexing? Do we really need it ? ...

4 AIEDCS 2013

slide-5
SLIDE 5
  • An important aspect of task sequencing in Adaptive

Hypermedia is the granularity of the domain model and the task indexing.

  • In general, the sequencing algorithm can better

determine the appropriate task if the granularity of the domain model and the task indexing is finer.

  • However, fine-grained domain models that dissect a

domain into dozens or hundreds of knowledge units are much harder to develop and to use for indexing.

5 AIEDCS 2013

slide-6
SLIDE 6
  • Typical approach to present programming knowledge

which uses coarse-grained topics like “loops” and “increment” allows reasonable sequencing during

  • course. (Brusilovsky et al. 2009, Hsiao et al. 2010, Kavcic 2004, Vesin et
  • al. 2012)
  • However, this approach fails in providing advance

sequencing such as providing support for exam preparation or remediation.

6 AIEDCS 2013

slide-7
SLIDE 7

Outline

  • Introduction
  • Hypothesis testing (pre-study)
  • Findings and challenges
  • JavaParser
  • Conclusion &Future works

7 AIEDCS 2013

slide-8
SLIDE 8

Pre-study

Knowledge Maximizer:

  • An exam preparation tool for Java programming
  • based on a fine-grained concept model of Java

knowledge

  • assumes a student already completed a considerable

amount of work

  • goal is to help her define gaps in knowledge and try

to redress them as soon as possible.

8 AIEDCS 2013

slide-9
SLIDE 9

11 AIEDCS 2013

slide-10
SLIDE 10

KM Parameters:

  • How prepared is the student to do the

activity?

12

∑ ∑

′ ′ =

r r

M i i M i i i

w w k K

K: user knowledge level Mr :prerequisite concepts ki :knowledge in Ci wi : weight of Ci for activity

AIEDCS 2013

slide-11
SLIDE 11
  • How prepared is the student to do the

activity?

  • What is the impact of the activity?

13

∑ ∑

′ ′ − =

  • M

i i M i i i

w w k I ) 1 (

I : activity impact Mo : outcome concepts

AIEDCS 2013

slide-12
SLIDE 12
  • How prepared is the student to do the

activity?

  • What is the impact of the activity?
  • What is the value of repeating the

activity again?

14

1 1 + − = t s S

: inverse success rate for the activity

S

S : number of success in the activity t : number of times the activity is done

AIEDCS 2013

slide-13
SLIDE 13
  • Determining the sequence of top 10

activity with the highest rank using:

15

S I K R + + =

R : activity rank K : knowledge level in prerequisites of activity I : activity Impact : Inverse of success rate

S

AIEDCS 2013

slide-14
SLIDE 14

Evaluation

  • We conducted a classroom study for the Java Programming

undergraduate course.

  • Study started on Dec. 4th 2012 about a week before the

final exam.

  • The course also used QuizGuide (QG), and Progressor+ (P+)

systems to access Java questions (available from the beginning of the semester).

  • All these systems used same 103 parameterized Java

questions.

16 AIEDCS 2013

slide-15
SLIDE 15

Evaluation Measures

We grouped participants into two groups:

  • KM: those who made at least ten attempt using

KM (n = 9)

  • QG/P+: those who made no attempt using KM

and at least 10 attempt with QG/P+ (n = 16) For each group we measured:

  • Number of questions (attempts) done using each

system

  • Success Rate

17 AIEDCS 2013

slide-16
SLIDE 16

Outline

  • Introduction
  • Hypothesis testing (pre-study)
  • Findings and challenges
  • JavaParser
  • Conclusion &Future works

18 AIEDCS 2013

slide-17
SLIDE 17

System Usage Summary

19

6.20% 43.50% 50.20% 34.60% 45.30% 20.10% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00% Easy Moderate Complex

Attempts per Question Compelixty

KM QG,P+

AIEDCS 2013

slide-18
SLIDE 18

Results

20

58% 64% 0% 50% 100% KM QG,P+

Success rate

AIEDCS 2013

  • 12.80%
  • 13.70%

11.53%

  • 15.00%
  • 10.00%
  • 5.00%

0.00% 5.00% 10.00% 15.00% Class QG/P+ KM

Average Relative Progress Percentage

slide-19
SLIDE 19
  • Fine-grained knowledge modeling will push students

toward appropriate complex questions which does not lead to miserable fails in those questions .

  • During exam preparation, complex questions are more

useful, since they target more concepts at once.

  • This helps students fill the gaps in their knowledge in a

more efficient way. (Eg. : 6 easy question must be done to get the same outcome as only 1 complex question!)

22 AIEDCS 2013

slide-20
SLIDE 20

JAVA Ontology

http://www.sis.pitt.edu/~paws/ont/java.owl

23 AIEDCS 2013

~ 400 node in ontology ~ 160 concept (leaf nodes)

slide-21
SLIDE 21

24 AIEDCS 2013

  • 103 JAVA Parameterized

Question

  • 1-5 classes per question
  • 5-52 concepts per question

~ 40% of questions has more than 20 concepts

slide-22
SLIDE 22

25 AIEDCS 2013

52 concepts or more in a question ?

slide-23
SLIDE 23

26 AIEDCS 2013

JavaParser: A tool for automatic indexing of Java Problems

slide-24
SLIDE 24

Outline

  • Introduction
  • Hypothesis testing (pre-study)
  • Findings and challenges
  • JavaParser
  • Conclusion &Future works

27 AIEDCS 2013

slide-25
SLIDE 25

Java Parser

  • Developed using Eclipse AST Tree API
  • The AST tree is semantically analyzed using

the information in each of its nodes.

28

1 2 3

AIEDCS 2013

slide-26
SLIDE 26

Modifiers Name Parameters Body

Structural properties of a method declaration

29

Exceptions Return Type

slide-27
SLIDE 27

Example

30

Public Method Declaration

AIEDCS 2013

slide-28
SLIDE 28

Example

31

Return Type Void

AIEDCS 2013

slide-29
SLIDE 29

Example

32

FormalMethodParameter Single Variable Declaration

AIEDCS 2013

slide-30
SLIDE 30

Example

33

Exception

AIEDCS 2013

slide-31
SLIDE 31

Example

34

Super Method Invocation

AIEDCS 2013

slide-32
SLIDE 32
  • Current version of JavaParser is able to extract

98.77% of the concepts in 103 manually indexed questions.

  • Average number of 8 extra concepts for each

question indexed by automatic parsing.

35 AIEDCS 2013

slide-33
SLIDE 33

Missed concept

  • InheritanceBasedPolymorphism
  • SuperclassSubclassConversion
  • PolymorphicObjectCreationStatement
  • MethodInheritance
  • ….

AIEDCS 2013 36

slide-34
SLIDE 34

Demo

http://adapt2.sis.pitt.edu/javaparser/ParseQuestion.jsp

AIEDCS 2013 37

slide-35
SLIDE 35

Outline

  • Introduction
  • Hypothesis testing (pre-study)
  • Findings and challenges
  • JavaParser
  • Conclusion &Future works

43 AIEDCS 2013

slide-36
SLIDE 36

Future work

Use the results of fine-indexing for:

  • Improving the user modeling service
  • cross content sequencing and providing

remediation in case of failing a question.

  • Predicting parts of code that might lead to

student failure and provide hints accordingly.

  • Expand the parser to extract more elaborated

concepts and programming patterns .

44 AIEDCS 2013

slide-37
SLIDE 37

Thank you!

Contact: roh38@pitt.edu

45 AIEDCS 2013