GENERAL EDUCATION ASSESSMENT RESULTS ALL FACULTY MEETING OCTOBER, 24, 2017 Presentation of General Education Assessment Goals, Procedures, Data, and Conclusions – Thad Meeks (Chair of General Education Assessment Committee)
Objectives for General Education and the Baccalaureate Degree • Analytic, Problem Solving, and Decision-Making Skills • Oral and Written Communication • Foundations in Liberal Arts • Value of Diversity • Scientific Literacy • Ethics
First Cycle of Gen Ed Assessment First Cycle Analytic Communication Scientific Foundations Communication Value of Ethics Problem (Written) Literacy in Liberal (Oral) Diversity Solving Arts Identify Measurable Outcomes '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 Develop rubrics '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '14-'15 '14-'15 Map outcomes to the gen ed '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 curriculum Collect evidence '13-'14 '13-'14 '14-'15 '15-'16 '15-'16 '15-'16 '15-'16 Score with rubrics (team of '14-'15 '14-'15 '14-'15 '15-'16 '15-'16 '15-'16 '15-'16 faculty) Analyze results '14-'15 '14-'15 '14-'15 '15-'16 '15-'16 '16-'17 '16-'17 Act on findings to improve '15-'16 '15-'16 '15-'16 '16-'17 '16-'17 '16-'17 '16-'17 curricular and cocurricular Revise assessments '15-'16 '15-'16 '15-'16 '16-'17 '16-'17 '16-'17 '16-'17
Second Cycle of Gen Ed Assessment Second Cycle Analytic Communication Scientific Foundations Communication Value of Ethics Problem (Written) Literacy in Liberal (Oral) Diversity Solving Arts '19- ‘20 '19- ‘20 Collect evidence '17-'18 '17-'18 '17-'18 '18-'19 '18-'19 '19- ‘20 '19- ’20 Score with rubrics (team of '17-'18 '17-'18 '17-'18 '18-'19 '18-'19 faculty) '19- ‘20 '19- ‘20 ‘20 - ’21 ‘20 - ’21 Analyze results '18-'19 '18-'19 '18-'19 '19- ‘20 '19- ‘20 ‘20 - ’21 ‘20 - ’21 Act on findings to improve '18-'19 '18-'19 '18-'19 curricular and cocurricular '19- ‘20 '19- ‘20 ‘20 - ’21 ‘20 - ’21 Revise assessments '18-'19 '18-'19 '18-'19
Written Communication • Early attempts were made to collect existing artifacts from a variety of classes and score them on different written communication and critical thinking criteria. • The mean scores for the upperclassmen were statistically higher than for the lowerclassmen. • The primary limitations were the inconsistencies in the types of and requirements for the written assignments. • Led the committee to seek a more standardized way to assess written communication (and perhaps other objectives)
Critical Thinking Assessment Test (CAT) • 15-item test designed to assess and promote the improvement of critical thinking and real-world problem solving skills • Developed by the Center for Assessment and Improvement of Learning at Tennessee Tech University with funding from NSF • The test is scored by the faculty at the participating institution. • Assesses four broad skills that align with our goals of Analytic Problem Solving, Written Communication, and Scientific Literacy • 1. Evaluation and Interpretation of Information (Scientific Literacy) • 2. Problem Solving (Analytic Problem Solving) • 3. Creative Thinking (Scientific Literacy and Analytic Problem Solving) • 4. Effective Communication (Written Communication)
CAT Procedure • Thad Meeks (Chair of General Education Assessment Committee) and Zenia Agustin (Director of General Education) trained on the CAT process • Collected data from freshmen in NFS course ( n = 141) and seniors in IS courses ( n = 105) in the spring of 2015 • Gen Ed Assessment Committee scored the freshman data over the course of the spring 2015 semester and the senior data at the beginning of summer 2015. • At least two raters per item (a third rater settled disagreements) • Ratings were sent to the Center for Assessment and Improvement of Learning for scoring.
CAT Results; * = Significant Group Difference * * * *
Conclusions from CAT • The positives were that overall (and with three subscales), there were statistical differences between freshmen and seniors. • Despite this, the effects were not necessarily large and no effect in Evaluating/Interpreting Information. • The limitations of the process included class selection, the timing of testing, the motivation of the students, and the scoring process. • We are currently collecting and will soon be assessing new CAT data. We are attempting to address some of these limitations.
Oral Communication Procedure • In the spring of 2016, we collected data from recorded student presentations in ACS 101 ( n = 109) and various other sources for upperclassmen (e.g., IS courses, senior assignment presentations), ( n = 69). • Students’ oral presentations were scored by the General Education Assessment Committee using a rubric on a 1-4 scale (higher numbers represented better performance). • Three subscales: • 1. Organization • 2. Language • 3. Delivery
Oral Communication Results; * = Significant Group Difference * * * * * * * *
Conclusions from Oral Communication Data • Overall, and on all three subscales, there were significant differences favoring the upperclassmen. • The primary limitation of this process was that the variety of presentations that were assessed made scoring consistency difficult.
Diversity and Ethics • These data were gathered as a part of NSSE Focus Groups in May of 2016 (60 total students). • One question related to students’ exposure to different types of perspectives (i.e., ethnic, political, economic, and social). • 43 responders claimed a combination of classes as their means of exposure. • One question related to students’ exposure that helped them develop their own code of values and ethics. • 32 responders claimed classroom experiences (e.g., discussion) as the source of this exposure. • What can we conclude from these data? How can we better assess this?
Overall Conclusions • In a positive sense, we showed statistically significant differences between our lower and upperclassmen, except in one subscale of the CAT. • Challenges • Better measures • More student motivation • More consistent sources of data • More consistent scoring of data • But all of this requires more buy-in for assessment • Need for faculty assistance • Need to “close the loop”
Recommend
More recommend
