TRANSITIONING TO A CULTURE OF EVIDENCE: SUSTAINABLE ASSESSMENT - PowerPoint PPT Presentation

TRANSITIONING TO A CULTURE OF EVIDENCE: SUSTAINABLE ASSESSMENT PRACTICES Roundt dtable le D Discussion O Orga ganized b d by Asse ssess ssment S Subc bcommittee 20 2018 8 UW UWSP T Teaching Conference Janua nuary 1 y 18, 2 2018

Invited Speakers and Assessment Subcommittee Members ■ Mark k Tols olstedt, Professor of Media Studies, Division of Communication, College of Fine Arts and Communciation ■ Da David B Barr rry, Assistant Professor of Sociology, Department of Sociology and Social Work, Colleges of Letters and Science ■ Paula la DeHart, Professor of Education, School of Education, College of Professional Studies ■ AS Mem Member bers: Maggie Bohm-Jordan, Dave Dettman (Co-Chair), Paul Doruska, Becca Franzen, Dan Graf (Co-Chair), Matt Groshek, Jennifer Huffman, Vera Klekovkina, Cuiting Li, Tom Salek, Jasia Steinmetz ■ Mo Moder erator: Vera Klekovkina (Assessment Coordinator)

Our Goals ■ Describe some sustainable assessment practices developed on our campus ■ Give a quick preview of Campus Labs – an electronic assessment management system ■ Invite all participants to share or imagine other sustainable assessment practices that provide tangible and insightful assessment results while reducing the time and effort required

CULTURE OF ASSESSMENT VS. CULTURE OF EVIDENCE

Briefly, what is EVIDENCE? Assessment tools and results Grades versus Assessment Scoring rubrics & assessing rubrics Direct & indirect measure assessment Surveys & student responses Informal anecdotes & rough estimations

DIVISION OF COMMUNICATION SELECT ASSESSMENT PRACTICES: CLICKER QUESTIONS Mark rk T Tolstedt, Professor of Media Studies, Division of Communication, College of Fine Arts and Communciation

Assessment Methods and Measures: Direct Measures – COMM 201 research presentations – COMM 400 ePortfolios – COMM 100 v 400 clicker questions – ARTM 480 – COMM 400 InterviewStream

COMM 100 v. 400 ■ Background. Both COMM 100 Introduction to Communication and COMM 400 Communication Capstone are required of all majors. COMM 100 is designed for first year, transfer and newly declared majors. Students are supposed to take the class before they earn form the 7 credits in Communication. COMM 400 is taken during a student’s last semester. During spring semester 2016, one question for each PLO was asked of both 100 and 400. Clickers were used for questions and answers. There were 100 students in COMM 100 and 60 in COMM 400.

■ PLO#3: Question: Uncertainty reduction theory was the foundation for Results many contemporary theories of uncertainty (True or False) – i. COMM 100 74% correct ■ PLO#1: Question: Meeting face-to-face after – ii. COMM 400 85% correct interacting online for an extended ■ PLO#4: Question Studying period of time is likely to increase history allows us to assess evidence and uncertainty (True or False) interpretations (True or False) – i. COMM 100 31% correct – i. COMM 100 96% correct – ii. COMM 400 52% correct – ii. COMM 400 100% correct ■ PLO#2: Question: A ■ PLO#5: Question The absolutist theory is an idea that is unproven approach to decision making in any (True or False) profession means that all decision must be – i. COMM 100 84% correct based on the context of the individual situation (True or False) – ii. COMM 400 62% correct – i. COMM 100 48% correct – ii. COMM 400 56% correct

COMM 100 vs. 400 % giving correct answer 52 PLO1 31 62 PLO2 84 85 PLO3 74 100 PLO4 96 56 PLO5 48 0 20 40 60 80 100 120 COMM 400 COMM 100

InterviewStream ■ Background. In the COMM 400 class, students are asked to find a job they want to apply for upon graduation, create both a resume and a cover letter for the job and then, using the Interview Stream software, apply for that job. Questions based on the Division’s three emphasis areas are programmed into the software. For assessment purposes, the Communication Assessment Committee identified questions asked that were tied to the PLO’s. 18 student interviews were randomly selected: 7 from the IPO emphasis, 7 from the MS emphasis and 4 from the PR emphasis. The PLO rubric was applied to the identified questions for each student

Results (IPO) ■ PLO#2: PLO#3: – Question: Give me an – Question:Describe a project or example of any time when you situation where you demonstrated used tools such as survey your analytical abilities. data, library research or statistics as important ■ i. Developing: 1 contributors to definition of a ■ ii. Developed: 2 specific problem. ■ iii. Mastered: 4 ■ i. Developing: 2 ■ ii. Developed: 3 PLO#5: ■ iii. Mastered: 2 – Question:Describe a situation in – Question: Describe a which you had to draw a time when you were proud of conclusion quickly and take your ability to use your speedy action. mathematical knowledge or research techniques to solve a ■ i. Developing: 0 problem. ■ ii. Developed: 4 ■ i. Developing: 2 ■ iii. Mastered: 3 ■ ii. Developed: 4 ■ iii. Mastered: 1

srs.campuslabs.com Clicker questions 19651 51 ■ Measure students’ knowledge ■ Track students’ attendance

INDIRECT AND DIRECT MEASURES IN ONE QUESTIONNAIRE David Bar id Barry, Assistant Professor of Sociology, Department of Sociology and Social Work, Colleges of Letters and Science

Department of Sociology Student Questionnaire ■ Questionnaire featuring A) indirect measures that record students’ perception of their learning and their satisfaction with the instruction received and B) direct measure – an essay question – that records students’ understanding of the core concepts in sociology

Examples ■ Perception of proficiency (indirect) 7. Statistical skills to conduct research about society. a. Great extent b. Moderate extent c. Small extent d. Not at all ■ Actual proficiency (direct) – (Closed-ended) 26. Which of the following is NOT an example of qualitative data collection? a. observations b. in-depth interviews c. self-administered survey d. content analysis – (Open-ended) b. Based on the scenario you’ve chosen, identify the independent/dependent variables.

Reflection ■ Stre rengths – Uniform, quick (painless), and great for trend comparisons – Results valuable feedback ■ Difference between student perception and actual knowledge ■ Allows for constructive discussion on what content is core to our discipline ■ And how/where to make changes ■ Limit mitat atio ions – Not exhaustive due to device size and structure (e.g., methods measures) – Missing pre-test

SHARED SURVEYS : COMMUNITY

EXIT SURVEYS ON PROGRAM LEARNING OUTCOMES Pa Paula D a DeHar art, Professor of Education, School of Education, College of Professional Studies

Exit Surveys ■ School of Education – World Languages and Literatures

COMMON RUBRICS Ver era K Klek ekovk vkina, Assessment Coordinator and Associate Professor of French, Department of World Languages and Literatures, College of Letters and Science

PUTTING IT ALL TOGETHER OUTCOMES MODULE IN CAMPUS LABS

OUTCOMES MODULE

Check Outcome Language & Bloom’s Taxonomy Image source Course Learning Outcome 1: To augment your vocabulary and improve your oral skills by practicing paragraph-length narrations.

Check Outcome Language & Bloom’s Taxonomy Image source Course Learning Outcome 1 (NEW): Students will be able to construct a paragraph-length narration in order to practice their oral skills in the target language.

Reworking Course Learning Outcomes

Diagnostics, Reflections, Measurement of Learning Outcomes

QUESTIONS, SUGGESTIONS, IDEAS? ■ AS Co S Co-Ch Chair airs: s: Dav ave Dettman man & & Da Dan Gr n Graf ■ Vera Klekovkina (Assessment Coordinator) ■ AS Members: – COFAC: Tom Salek & Matt Groshek – COLS: Maggie Bohm-Jordan, Vera Klekovkina & Dan Graf – CNR: Paul Doruska & Becca Franzen – CPS: Cuiting Li & Jasia Steinmetz – University College: Jennifer Huffman & Dave Dettman

THANK YOU!