TRANSITIONING TO A CULTURE OF EVIDENCE: SUSTAINABLE ASSESSMENT - - PowerPoint PPT Presentation

TRANSITIONING TO A CULTURE OF EVIDENCE:

SUSTAINABLE ASSESSMENT PRACTICES

Roundt dtable le D Discussion O Orga ganized b d by Asse ssess ssment S Subc bcommittee

20 2018 8 UW UWSP T Teaching Conference Janua nuary 1 y 18, 2 2018

Invited Speakers and Assessment Subcommittee Members

■ Mark k Tols

• lstedt, Professor of Media Studies, Division of Communication,

College of Fine Arts and Communciation ■ Da David B Barr rry, Assistant Professor of Sociology, Department of Sociology and Social Work, Colleges of Letters and Science ■ Paula la DeHart, Professor of Education, School of Education, College of Professional Studies ■ AS Mem Member bers: Maggie Bohm-Jordan, Dave Dettman (Co-Chair), Paul Doruska, Becca Franzen, Dan Graf (Co-Chair), Matt Groshek, Jennifer Huffman, Vera Klekovkina, Cuiting Li, Tom Salek, Jasia Steinmetz ■ Mo Moder erator: Vera Klekovkina (Assessment Coordinator)

Our Goals

■ Describe some sustainable assessment practices developed on our campus ■ Give a quick preview of Campus Labs – an electronic assessment management system ■ Invite all participants to share or imagine other sustainable assessment practices that provide tangible and insightful assessment results while reducing the time and effort required

CULTURE OF ASSESSMENT

• VS. CULTURE OF EVIDENCE

Briefly, what is EVIDENCE? Assessment tools and results

Grades versus Assessment Scoring rubrics & assessing rubrics Direct & indirect measure assessment Surveys & student responses Informal anecdotes & rough estimations

DIVISION OF COMMUNICATION

SELECT ASSESSMENT PRACTICES: CLICKER QUESTIONS

Mark rk T Tolstedt, Professor of Media Studies, Division of Communication, College of Fine Arts and Communciation

Assessment Methods and Measures: Direct Measures

– COMM 201 research presentations – COMM 400 ePortfolios – COMM 100 v 400 clicker questions – ARTM 480 – COMM 400 InterviewStream

COMM 100 v. 400

■

• Background. Both COMM 100 Introduction to Communication and COMM 400

Communication Capstone are required of all majors. COMM 100 is designed for first year, transfer and newly declared majors. Students are supposed to take the class before they earn form the 7 credits in Communication. COMM 400 is taken during a student’s last semester. During spring semester 2016, one question for each PLO was asked of both 100 and 400. Clickers were used for questions and

• answers. There were 100 students in COMM 100 and 60 in COMM 400.

Results

■ PLO#1: Question: Meeting face-to-face after interacting online for an extended period of time is likely to increase uncertainty (True or False) –

• i. COMM 100

31% correct –

• ii. COMM 400

52% correct ■ PLO#2: Question: A theory is an idea that is unproven (True or False) –

• i. COMM 100

84% correct –

• ii. COMM 400

62% correct ■ PLO#3: Question: Uncertainty reduction theory was the foundation for many contemporary theories of uncertainty (True or False) –

• i. COMM 100

74% correct –

• ii. COMM 400

85% correct ■ PLO#4: Question Studying history allows us to assess evidence and interpretations (True or False) –

• i. COMM 100

96% correct –

• ii. COMM 400

100% correct ■ PLO#5: Question The absolutist approach to decision making in any profession means that all decision must be based on the context of the individual situation (True or False) –

• i. COMM 100

48% correct –

• ii. COMM 400

56% correct

COMM 100 vs. 400

48 96 74 84 31 56 100 85 62 52 20 40 60 80 100 120 PLO5 PLO4 PLO3 PLO2 PLO1

% giving correct answer

COMM 400 COMM 100

InterviewStream

■

• Background. In the COMM 400 class, students are asked to find a job they want to

apply for upon graduation, create both a resume and a cover letter for the job and then, using the Interview Stream software, apply for that job. Questions based on the Division’s three emphasis areas are programmed into the software. For assessment purposes, the Communication Assessment Committee identified questions asked that were tied to the PLO’s. 18 student interviews were randomly selected: 7 from the IPO emphasis, 7 from the MS emphasis and 4 from the PR

• emphasis. The PLO rubric was applied to the identified questions for each student

Results (IPO)

■ PLO#2: – Question: Give me an example of any time when you used tools such as survey data, library research or statistics as important contributors to definition of a specific problem.

■ i. Developing: 2 ■ ii. Developed: 3 ■

• iii. Mastered: 2

– Question: Describe a time when you were proud of your ability to use your mathematical knowledge or research techniques to solve a problem.

■ i. Developing: 2 ■ ii. Developed: 4 ■

• iii. Mastered: 1

PLO#3: – Question:Describe a project or situation where you demonstrated your analytical abilities. ■ i. Developing: 1 ■ ii. Developed: 2 ■ iii. Mastered: 4 PLO#5: – Question:Describe a situation in which you had to draw a conclusion quickly and take speedy action. ■ i. Developing: ■ ii. Developed: 4 ■ iii. Mastered: 3

Clicker questions

■ Measure students’ knowledge ■ Track students’ attendance

srs.campuslabs.com

19651 51

INDIRECT AND DIRECT MEASURES IN ONE QUESTIONNAIRE

David Bar id Barry, Assistant Professor of Sociology, Department of Sociology and Social Work, Colleges of Letters and Science

Department of Sociology Student Questionnaire

■ Questionnaire featuring A) indirect measures that record students’ perception of their learning and their satisfaction with the instruction received and B) direct measure – an essay question – that records students’ understanding of the core concepts in sociology

Examples

■ Perception of proficiency (indirect)

• 7. Statistical skills to conduct research about society.

a. Great extent b. Moderate extent c. Small extent d. Not at all

■ Actual proficiency (direct) – (Closed-ended)

• 26. Which of the following is NOT an example of qualitative data collection?

a.

• bservations

b. in-depth interviews c. self-administered survey d. content analysis

– (Open-ended)

• b. Based on the scenario you’ve chosen, identify the independent/dependent

variables.

Reflection

■ Stre rengths

– Uniform, quick (painless), and great for trend comparisons – Results valuable feedback

■ Difference between student perception and actual knowledge ■ Allows for constructive discussion on what content is core to our discipline ■ And how/where to make changes

■ Limit mitat atio ions

– Not exhaustive due to device size and structure (e.g., methods measures) – Missing pre-test

SHARED SURVEYS : COMMUNITY

EXIT SURVEYS ON PROGRAM LEARNING OUTCOMES

Pa Paula D a DeHar art, Professor of Education, School of Education, College of Professional Studies

Exit Surveys

■ School of Education – World Languages and Literatures

COMMON RUBRICS

Ver era K Klek ekovk vkina, Assessment Coordinator and Associate Professor of French, Department of World Languages and Literatures, College of Letters and Science

PUTTING IT ALL TOGETHER

OUTCOMES MODULE IN CAMPUS LABS

OUTCOMES MODULE

Check Outcome Language & Bloom’s Taxonomy

Course Learning Outcome 1: To augment your vocabulary and improve your oral skills by practicing paragraph-length narrations.

Image source

Check Outcome Language & Bloom’s Taxonomy

Course Learning Outcome 1 (NEW): Students will be able to construct a paragraph-length narration in order to practice their oral skills in the target language.

Image source

Reworking Course Learning Outcomes

Diagnostics, Reflections, Measurement of Learning Outcomes

QUESTIONS, SUGGESTIONS, IDEAS?

■ AS Co S Co-Ch Chair airs: s: Dav ave Dettman man & & Da Dan Gr n Graf ■ Vera Klekovkina (Assessment Coordinator) ■ AS Members: – COFAC: Tom Salek & Matt Groshek – COLS: Maggie Bohm-Jordan, Vera Klekovkina & Dan Graf – CNR: Paul Doruska & Becca Franzen – CPS: Cuiting Li & Jasia Steinmetz – University College: Jennifer Huffman & Dave Dettman

THANK YOU!