3D-LOP: Three-Dimensional Learning Observation Protocol Becky - PowerPoint PPT Presentation

3D-LOP: 
 Three-Dimensional 
 Learning Observation Protocol � Becky Matz � CREATE for STEM Institute � WMU-MSU Institutional Transformation Symposium � Western Michigan University � May 22, 2014 �

The premise of the AAU project is three-fold � Engaging faculty to determine the scientific practices, crosscutting concepts, and core ideas for the gateway STEM courses � and changes in will lead to changes in assessment practices � classroom practice �

With RTOP, TDOP, OTOP, and COPUS already available, why bother developing another observation protocol? � • Existing assessment instruments focus on the “how” of teaching � • 3D-LOP also incorporates the “what” of teaching: � • Scientific practices � • Crosscutting concepts � • Disciplinary core ideas � • Phenomena �

The Classroom Observation Protocol for Undergraduate STEM (COPUS) � • Records how both instructors and students spend their time � • Each 2-minute interval is a unit of analysis � • Coders can be reliably trained in ~90 min � Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize University STEM Classroom Practices. CBE-Life Sciences Education , 12 (4), 618-627. �

The Classroom Observation Protocol for Undergraduate STEM (COPUS) � Instructor is doing: � Students are doing: � Adm � W � AnQ � SQ � Lec � D/V � 1o1 � CG � L � RtW � MG � AnQ � Fup � CQ � Ind � PQ � Coded by J.T. Laverty � Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize University STEM Classroom Practices. CBE-Life Sciences Education , 12 (4), 618-627. �

Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize University STEM Classroom Practices. CBE-Life Sciences Education , 12 (4), 618-627. �

Another way to view the COPUS data is to put it on a timeline � ! " # $ %& %! %" %# %$ !& !! !" !# !$ '& '! '" '# '$ "& "! "" "# "$ !"#$%&"&' ()*+,-). (&)"*")+,- .-"/0%12314+5# 6&#7%1"&'28+%#$"4& 9$+)%&$28+%#$"4& !%/$+1"&' :%,-;$"<%2=1"$"&' >4--472?@ @4#%28+%#$"4& /-.)0*1)20 .-"/0%128+%#$"4& 6&#7%1"&'28+%#$"4& A4*"&'B3+")"&' C24&2C D%<4BE"#+,-# 6)<"&"#$1,$"4& =,"$"&' Coded by J.T. Laverty � Smith, M. K., Jones, F. H., Gilbert, S. L., & Wieman, C. E. (2013). The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize University STEM Classroom Practices. CBE-Life Sciences Education , 12 (4), 618-627. �

The same classroom recording coded with 3D-LOP � First half of class � Coded by J.T. Laverty �

The same classroom recording coded with 3D-LOP � Whole class � Coded by J.T. Laverty �

A brief comparison of COPUS (top) and 3D-LOP (bottom) � ! " # $ %& %! %" %# %$ !& !! !" !# !$ '& '! '" '# '$ "& "! "" "# "$ !"#$%&"&' ()*+,-). (&)"*")+,- .-"/0%12314+5# 6&#7%1"&'28+%#$"4& 9$+)%&$28+%#$"4& !%/$+1"&' :%,-;$"<%2=1"$"&' >4--472?@ @4#%28+%#$"4& /-.)0*1)20 .-"/0%128+%#$"4& 6&#7%1"&'28+%#$"4& A4*"&'B3+")"&' C24&2C D%<4BE"#+,-# 6)<"&"#$1,$"4& =,"$"&'

7 teaching activities constitute the “how” of the observations � 1. Clicker questions � Students respond with personal response instruments � Mutually exclusive and complete � � � � �� 2. Tasks � Students work together or alone to solve a problem, construct a diagram, etc. � 3. Interactions � Substantive and possibly lengthy exchanges between the instructor and students � 4. Lecture � Instructor-directed presentation of content- related information � 5. Administration � “Housekeeping” items such as exam logistics, scheduling, and announcements � 6. Miscellaneous � Anything that does not fit above � 7. Questions � Content-related questions from the instructor �

Scientific practices, crosscutting concepts, core ideas and phenomena constitute the “what” of the observations � • Asking questions � 1. Scientific practices � • Developing and using models � • Constructing explanations � • Patterns � 2. Crosscutting concepts � • Scale, proportion, and quantity � • Stability and change � • The cell is the fundamental unit of life. � 3. Disciplinary core ideas � • DNA is the source of heritable information. � • Basic structural units define the function of all living things. � • When instructors contextualize learning for 4. Phenomena � students using real-world examples, videos, images, etc. �

Research questions � How do the following measures change between years 1 and 3 in the disciplines as the result of engaging faculty? � � How much class time involves: � scientific practices? � crosscutting concepts? � disciplinary core ideas? � student-centered activities (clicker questions, tasks, and interaction)? �� � How frequently is class time contextualized using phenomena? � � How frequently do instructors solicit input from students with � questions? � What does instruction look like that overlaps core ideas, practices, and crosscutting concepts? �

Teaching practice in these courses will be measured with � 3D-LOP in year 1 (2013-14) and year 3 (2015-16) of the project � • BS 161: Cell and Molecular Biology � Biology � • BS 162: Organismal and Population Biology � • CEM 141: General Chemistry � • CEM 142: General and Inorganic Chemistry � Chemistry � • CEM 151: General and Descriptive Chemistry � • CEM 152: Principles of Chemistry � • PHY 183: Physics for Scientists and Engineers I � • PHY 184: Physics for Scientists and Engineers II � Physics � • PHY 231: Introductory Physics I � • PHY 232: Introductory Physics II �

Each section of each course will be observed � three times for each year of data collection � For example, BS 161: Cell and Molecular Biology was recorded 27 times in 2013-14. � • Section 1 � • Section 2 � Fall 2013 � 12 recordings � • Section 3 � • Section 4 � • Section 1 � • Section 2 � Spring 2014 � • Section 3 � 15 recordings � • Section 4 � • Section 5 �

StudioCode is the software package that will be used to analyze the classroom recordings �

Measures for reliability, validity, and objectivity � • Reliability: Multiple iterations of multiple researchers coding the same recordings � • Validity: Vetting selections with disciplinary researchers � • Objectivity: One-third of year 1 videos will be blind coded with all year 3 recordings �

The software and our coding scheme will allow us to more deeply analyze individual teaching activities through labeling � Potential criteria (labels) for clicker questions: � � � Do students have time to think alone? � � � Do students have the time to discuss with � � � one another? � � �� � � Are the students re-polled? �

Acknowledgements � • Melanie Cooper � • Diane Ebert-May � • Joe Krajcik � • Sekhar Chivukula � • Rob LaDuca � • Danny Caballero � • Cori Fata-Hartley � • Bob Geier � • Sarah Jardeleza � • J.T. Laverty � • Tammy Long � • Lynmarie Posey � • Sonia Underwood � • Biological sciences faculty � • Chemistry faculty � • Physics faculty � • College of Natural Science Deans �

A possible question for discussion � • How to give feedback to faculty (if at all)? �