Windows Into The Mind Richard J. Shavelson & Maria Araceli - - PowerPoint PPT Presentation
Windows Into The Mind Richard J. Shavelson & Maria Araceli Ruiz-Primo Stanford University Invited Talk University of Ancona Ancona, Italy July ????, 2000 Assessing Knowledge Structure DENNIS THE MENACE GEE! IM GLAD WE DID CONCEPT
Invited Talk University of Ancona Ancona, Italy July ????, 2000
July ????, 2000 Windows Into The Mind 2
DENNIS THE MENACE “GEE! I’M GLAD WE DID CONCEPT MAPS - NOW WE BOTH KNOW WHAT I LEARNED TODAY!!! ”
July ????, 2000 Windows Into The Mind 3
July ????, 2000 Windows Into The Mind 4
Declarative Procedural Strategic Knowledge Knowledge Knowledge
(Knowing the “that”) (Knowing the “how”) (Knowing the “which,” “when,” and “why”)
Proficiency
Low High
Extent
(How much?)
Structure
(How is it organized?)
Others
(Precision? Efficiency? Automaticity?)
Cognitive Cognitive Tools: Tools:
Planning Planning Monitoring Monitoring
Domain-specific content:
Domain-specific production systems Problem schemata/ strategies/
July ????, 2000 Windows Into The Mind 5
Is a
Is an example of Contains
Associative Semantic
July ????, 2000 Windows Into The Mind 6
consist of consist of is a
denote a represent linked by between a pair of
July ????, 2000 Windows Into The Mind 7
The following map shows the relationship among concepts related to genetic continuity. The concepts are arranged hierarchically and linked each other. Please examine the map and supply, in the space provided, a word or two for labeling each link such that association between concepts is made clear.
Ayita, I think the student had to simply write in a number (1-5) on the line from words given
Task Task Response Format Response Format Scoring System Scoring System
Genetic continuity Asexual reproduction Sexual reproduction Meiosis Male gamete Female gamete
levels beyond the last branching
two or more concepts are connected to the concept
branching
Source: Tamir, P. (1995). Science Assessment. In M. Birenbaum & F. J. R. C. Dochy (Ed.). Alternatives in assessment of achievements, learning processes, and prior knowledge. Boston: Kluwer Academic Publishers.
July ????, 2000 Windows Into The Mind 8
July ????, 2000 Windows Into The Mind 9
Construct a concept map that reflects what you know about what a concept map is. Organize the terms in relation to one another in any way you want.
concept map concepts relationships
composed of which elementary unit is composed of stand for with explanations show the essential relationship between composed of with explanations stand for
knowledge
can be represented to some extent by a determine the structure of a
linking lines nodes proposition
acquisition demands understanding the meaning of understanding demands clear explanation of
July ????, 2000 Windows Into The Mind 10
July ????, 2000 Windows Into The Mind 11
Quality of Proposition Definition
Outstanding proposition. Complete and correct. It shows a deep understanding of the relation between the two concepts. acids-compounds: < that gives off H+ when dissolved in water are Excellent Excellent 4 points Good Good Complete and correct proposition. It shows a good understanding
acids-compounds: > are examples of
3 points Poor Poor
Incomplete but correct proposition. It shows partial understanding
acids-compounds: < form 2 points Don’t Care Don’t Care Although valid, the proposition does not show understanding between the two concepts. acids-compounds: > is a different concept 1 points Invalid Invalid Incorrect proposition. acids-compounds: > made of 0 points
July ????, 2000 Windows Into The Mind 12
Examine the map, the blank nodes and the terms provided on the list below. Select the term that corresponds to each node and write it down inside the circle. Use each term only once. List of Concepts:
Score right or wrong
with
concept map
is a with
nodes
with represent linked by that describe the relationship between two nodes are called
July ????, 2000 Windows Into The Mind 13
Compounds Molecular Compounds Anions Ternary Ionic Compounds
ELEMENTS, ATOMS, AND COMPOUNDS
Atoms Elements are composed of the same type of Periodic Table are organized on a are negatively charged particles of have either a positive or a negative are formed when atoms lose or gain are combined chemically to form made up of
charged are composed
is a ternary ionic compound that contains sulfite which is a are located
side of the are elements located in the center of the contain one
with two elements are have to have hydrogen are composed of H+ and are composed of metals and tend to form are located on the upper right side
have a positive have a negative will form Metals containing molecules and sharing electrons are called
C
are composed of two Transition Metals Non-Metals Charge Ions Polyatomic Ions Cations Electrons Ionic Compounds N2O4 Sodium Sulfite Binary Ionic Compounds Acids is an example of a with equal number of electrons and protons have a neutral have a negative
July ????, 2000 Windows Into The Mind 14
different persons?
provide the same picture about students’ declarative knowledge structure?
cognitive demands to students?
Do concept maps measure something different than multiple-choice test?
July ????, 2000 Windows Into The Mind 15
Mapping Technique 2 Mapping Technique 1
No-Concepts Provided Concepts Provided
Molecules, and Compounds.
think are the most important in explaining the topic.
Molecules, and Compounds.
the map.
read it.
July ????, 2000 Windows Into The Mind 16
Percent of Total Variability and G Coefficients for Proposition Validity Total Scores No-Concepts, Sample A, and B Sample A & B Person (p) Rater (r) Sample (s) p x r p x s r x c prc,e ρ Φ ρ
2
Φ
2
71.64 78.67 .15 .79 22.81 17.64 .01 .18 5.37 2.69 .89 .88 .89 .88 .78 .78
(nr=1,ns=1) (nr=2,ns=2) (nr=2,ns=3)
July ????, 2000 Windows Into The Mind 17
Correlation Between Multiple-Choice Test Score and Concept Map Scores
Type of Score No-Concepts Sample A Sample B
Proposition Validity Congruence Salience .64 .66 .61 .63 .55 .50 .58 a .45
July ????, 2000 Windows Into The Mind 18
Construct-a-Map Fill-in-the-Linking Lines Fill-in-the-Nodes
provided to construct the map.
way you want.
someone else can read it.
map, the blank linking lines and the linking words provided.
that correspond to each blank linking line.
are filled-in.
map, the blank nodes and the terms provided.
corresponds to each blank node.
circle.
nodes are filled-in.
July ????, 2000 Windows Into The Mind 19
Construct-a-Map Fill-in-the-Linking Lines Fill-in-the-Nodes
Blank page to construct the map
Elements, Atoms, and Compounds
atoms compounds N2O4
made up
composed of with 3 or more elements are
ions ionic compounds molecular compounds ternary ionic
acids formed from molecules sharing
atoms compounds N2O4
Elements, Atoms, and Compounds
formed with cations and anions are formed from molecules sharing example of made up
composed of that begins with H are
lose or gain electrons to formwith 3 or more elements are
July ????, 2000 Windows Into The Mind 20
Means and Standard Deviations by Type of Map and Sample
Mean S.D.
Type of Skeleton Map Fill-in-the-nodes Sample 1 80 11.21 1.42 Sample 2 72 10.80 1.74 Fill-in-the-linking lines Sample 1 78 9.77 2.74 Sample 2 73 8.99 3.09
(Max = 12)
n
July ????, 2000 Windows Into The Mind 21
Means and Standard Deviations by Type of Map and Sequence
Fill-in-the-linking lines Fill-in-the-nodes Sequence
Mean S.D. Mean S.D. n
1 Nodes 1-Lines 1 43 11.09 1.52 9.72 2.84 2 Nodes 1-Lines 2 36 11.03 1.33 9.31 3.06 3 Nodes 2-Lines 1 35 10.63 1.81 9.83 2.65 4 Nodes 2-Lines 2 37 10.97 1.67 8.68 3.13 Total 152 11.02 1.59 9.36 2.93
July ????, 2000 Windows Into The Mind 22
Means and Standard Deviations across the Three Types of Assessments
Type of Assessment
Max n Mean S.D.
Construct-A-Map
Proposition Accuracy 152 135 53.91 22.17 Convergence 152 1 .50 .19 Salience 152 1 .73 .17
Fill-In
Fill-in-the-nodes 152 12 11.02 1.59 Fill-in-the-lines 151 12 9.39 2.93
Multiple-Choice Test
150 30 24.05 3.74
July ????, 2000 Windows Into The Mind 23
Type of Assessment
Construct-A-Map (C-M)
(.99)
Fill-in-the-nodes-NOD (FI-N)
.47 (.70)
Fill-in-the-lines-LIN (FI-L)
.44 .40 (??)
Multiple-Choice Test (MC)
.44 .37 .53 (.74) Structure Amount C-M FI-N FI-L MC
July ????, 2000 Windows Into The Mind 24
July ????, 2000 Windows Into The Mind 25
0.1 0.2 0.3 0.4 0.5 0.6
Explanation Monitoring Conceptual Errors No-Code
Contruct-A-Map Fill-In-Nodes Fill-In-Lines
Verbal Codes Proportion Scores
Low Directed High Directed
July ????, 2000 Windows Into The Mind 26
when complex judgments, such as proposition quality, are required
declarative knowledge structure. The construct-a-map method provides
respond by elimination or guessing, whereas constructed response do not.
positive and moderate suggesting that these two types of assessment measure overlapping but somewhat different aspects of declarative knowledge
July ????, 2000 Windows Into The Mind 27
July ????, 2000 Windows Into The Mind 28
Rate the similarity of each pair of terms by circling
(Closely Related)
1 2 3 4 5 6 7
(Unrelated)
DK
(Circle one number -or- "DK" (Don't Know))
(Closely Related)
1 2 3 4 5 6 7
(Unrelated)
DK
(Circle one number -or- "DK" (Don't Know))
July ????, 2000 Windows Into The Mind 29
Terms: Variability, Variance, Standard Deviation Variability Variance
measures measures is the square root of
to a procedure for making generalizations from a limited random sample of data?
July ????, 2000 Windows Into The Mind 30
uses sample to test ideas in the is used to test ideas about population in is used to test hypotheses in is used to test population-related addresses ideas about variables in population in the process of i s a p r
e d u r e i n s t a t i s t i c a l is one special case of the general linear model, along with i s a p r
e d u r e i n s t a t i s t i c a l refers to relationships within the
Hypothesis Sample Linear Regression Population Inference ANOVA
July ????, 2000 Windows Into The Mind 31
July ????, 2000 Windows Into The Mind 32
July ????, 2000 Windows Into The Mind 33
MC CM Pretest CM 0.155 SJ 0.554 ** 0.251 ** p<.01 MC CM Posttest CM 0.330 SJ 0.434 * 0.706 ** ** p<.01
July ????, 2000 Windows Into The Mind 34
July ????, 2000 Windows Into The Mind 35