Department of Psychology and Philosophy Assessment Report AY2017-18 - - PowerPoint PPT Presentation

Department of Psychology and Philosophy

Assessment Report AY2017-18

Assessment Procedures 2017-18

• According to the 5 year assessment plan filed in Fall 2017, Goal 2 –Interpret, design, and conduct basic psychological

research was scheduled to be assessed. Assessment of this goal required the development of rubrics for and assessment

• f artifacts from the capstone courses AY2017-18.
• Due to Work to Rule and failure to reach a contract agreement, the department decided to assess Goal 1-Describe key

concepts, principles, and overarching themes in psychology. This goal was originally scheduled for assessment in AY 2018- 19 but was moved up due to the ease of assessment with the ACAT Comprehensive Content measure commercially

• available. The department agreed that administering this test would not violate work to rule agreement and the review

could be completed during the summer months with additional compensation for summer work.

• The ACAT allows for selection of specific content domains to be assessed. The empirical research domain was selected as a

direct measure for Goal 1. The additional domains of Development, Social, and Learning and Cognition were selected to provide information on knowledge base in 3 of 4 Content Domain areas within the Psychology major at FSU.

• All capstone instructors agreed to administer the exam during the last week of April or the first week of May 2018.
• The department decided on providing an incentive for completing the task. The assessment committee obtained incentive

choices of : $20 University Bookstore Credit, Two Cinema Movie Passes, or $20 in vouchers for taxi services.

• The cost of the ACAT was $23.50 per student for a total cost, including shipping, of $2,171.00. A total of 85 exams were
• rdered for the 85 non-overlapping student enrollment in the capstone courses. A total of 71 students completed the

exam and the unused forms were returned.

ACAT Assessment Data

Administered to Capstone courses during the last week of April and first week of May 2018 total possible 105, Sample completing=71.

• History and Systems (n=20)
• Seminar: Humor (n=20)
• Seminar: Prejudice (n=20)
• Thesis (n=18, n=10)
• Internship (n=17)
• Students provided incentive in form of choice of $20.00 Bookstore Credit, two

cinema passes, or $20.00 in taxi fee. Evenly divided between bookstore and cinema passes, no taxi fee chosen.

• 71 students completed the exam and chose NOT to complete the alternative

assignment in order to receive the incentive.

ACAT 2018 FSU Psychology Capstone Students Gender (N=71)

67 4 10 20 30 40 50 60 70

Women Men

Number of Students

ACAT 2018 FSU Psychology Capstone Students Native versus Transfer (N=71)

51 20 10 20 30 40 50 60

Native Transfer

Number of Students

ACAT 2018 FSU Psychology Capstone Students Self Report GPA (N=71)

2 13 32 21 2 9 30 27 5 10 15 20 25 30 35

2.1-2.5 2.6-3.0 3.1-3.5 3.6-4.0

Number of Students

Self Report GPA Ranges

Overall Major

ACAT 2018 Overall Score (N=71) FSU Psychology Compared to National Sample

8.5 7 18.3 29.6 23.9 8.5 1.4 2.8 4.3 8.1 13.3 17.7 18.4 16.1 11.7 6.3 4.1 5 10 15 20 25 30 0--4 4--11 11--23 23--40 40--60 60--77 77--89 89--96 96--100

Percent

Scores by Percentile

FSU National

ACAT 2018 FSU Psychology Capstone Students Average Percentile by Content Areas (N=71)

31 39 27 31 32 10 20 30 40 50 60 70 80 90 100

Overall Experimental Developmental Learning and Cognition Social

Percentile

P&P Dept. Supplemental Survey Classification (N=80)*

1 21 56 10 20 30 40 50 60

Sophomore Junior Senior

Number of Students

*Numbers varied from the ACAT and P&P Dept. Survey

P&P Dept. Supplemental Survey Course in the Biological Domain (N=80)

11 37 39 6 10 20 30 40 50 60

Biopsych Health S&P NONE

Number of Students

P&P Dept. Supplemental Survey Course in the Developmental Domain (N=80)

55 23 24 10 20 30 40 50 60

Child Adolescent Adult

Number of Students

P&P Dept. Supplemental Survey Course in the Learning/Cog Domain (N=80)

34 37 22 1 10 20 30 40 50 60

Learning Cognitive BX Mod NONE

Number of Students

P&P Dept. Supplemental Survey Course in the Sociocultural Domain (N=80)

32 31 20 16 2 10 20 30 40 50 60

Personality Social Cultural Women NONE

Number of Students

P&P Dept. Supplemental Survey Number of Courses taken in the Biological Domain (N=80)

6 64 7 3 10 20 30 40 50 60 70

NONE One Course Two Courses Three Courses

Number of Students

P&P Dept. Supplemental Survey Number of Courses taken in the Developmental Domain (N=80)

60 16 4 10 20 30 40 50 60 70

One Course Two Courses Three Courses

Number of Students

P&P Dept. Supplemental Survey Number of Courses taken in the Learning/Cog Domain (N=80)

1 64 15 10 20 30 40 50 60 70

NONE One Course Two Courses

Number of Students

P&P Dept. Supplemental Survey Number of Courses taken in the Sociocultural Domain (N=80)

2 56 18 2 2 10 20 30 40 50 60 70

NONE One Course Two Courses Three Courses Four Courses

Number of Students

P&P Dept. Supplemental Survey Number of Students Taking Each Domain Course (N=80)

11 16 20 22 23 24 31 32 34 37 37 39 55 10 20 30 40 50 60 Bio Women Cultural BX Adol Adult Social Person Learn Health Cog S&P Child

Number of Students

Course Names

Issues and Concerns for Second Administration of ACAT and Student Survey:

• Inability to link FSU student survey information to ACAT data due to

inconsistency or missing ID numbers on FSU survey.

• Discrepancy between student reported course enrollment on ACAT and FSU
• survey. Course names not consistent and students may have not correctly

identified similar course listings.

• FSU survey formatting needs to be changed for ease of analysis and

comparison.

• Collect data on 300 level courses taken to date.
• Pre-assign student ID numbers in future.
• Scheduled testing times instead of class administered if possible.
• Limit to only juniors/seniors
• Administer in fall to obtain info on students graduating in Fall.

Key Points:

• Assessed Goal 1: Describe key concepts, principles, and overarching

themes in psychology. Assessment committee decided to assess in four content areas: Developmental, Experimental, Social, and Learning and Cognition.

• Scores on the ACAT revealed that compared to a national sample, our

students would fall in the range of a low in Developmental of 27% to a high of 39% in the Experimental domains.

• Examination of the number of courses students reported taking in the

different departmental domains offered by the department yields some insight into the percentages obtained.

Key Points: ACAT Experimental Design and Course Experience

• All 71 students completing the ACAT had successfully completed Research

Methods 1 and Research Methods 2 at FSU.

• The ACAT score students achieving at or below the 39% level for this domain may

reflect the performance of students completing the course requirements at FSU.

• A grade of C- or better is required to progress through the sequence and on to

the capstones. This grade reflects written performance and statistical performance as well as an understanding of experimental design. The sequence is heavily biased toward writing and understanding statistics while the ACAT references a good deal more advanced experimental design concepts.

• The ACAT score may indeed accurately reflect the “content” emphasized in these

courses and therefore may be more satisfactory than first glance would suggest.

Key Points: ACAT Developmental Domain and Course Experience

• While Developmental domain scores were only in the 27th percentile, students

reported taking the most courses (123) overall in that domain.

• However, the child development enrollments of 55 far exceeded the enrollments

in adolescent and adult development.

• A total of 60 students took only one course in this domain suggesting that the

bulk of exposure to developmental content would have come primarily from child development.

• The domain test of the ACAT is evenly divided over the three areas of

development and therefore may not accurately reflect the “developmental” perspectives that are being highlighted in our department.

Key Points-ACAT Scores on Human Learning and Cognition Related to Course Experience

• The Score on the Human Learning and Cognition ACAT results

indicates students scoring at or below the 31% level.

• When examining the courses taken in this domain Learning and

Cognitive enrollments were approximately equal with a little over

• ne-third of the students taking each course and 64 completing only
• ne course in the domain.
• Given this discrepancy in course content exposure the results on the

ACAT may reflect an absence of students completing the content in the curriculum.

Key Points: ACAT Scores on Social Domain related to Course Experience

• About 1/3 of the students reported taking Social and 1/3 reported

taking Personality. Slightly less than 1/3 reported taking either Cultural or Psychology of Women.

• However, a total of 56 students reported taking only one course in

this domain. The exposure to the concepts on the Social ACAT domain test content may be limited in those students not having taken Social.

Key Points: Biological Domain Course Enrollments

• Enrollment in the Biological Domain courses reflects a more even

distribution of course enrollments between Health and Sensation and Perception with lower enrollments in Biopsychology.

• These enrollments may reflect the enrollments of the neuroscience

minors in Biopsychology with students taking the Bio also taking the

• ther two courses.
• It is of note that 6 students had not taken any courses in this domain.

Assessment Recommendations AY2018-19:

• Proceed with goal assessment schedule as published 2017/18.
• Assess Goal 1: Develop assessment rubrics for capstone artifacts during Fall 2018.

Capstone course instructors upload artifacts to blackboard site for assessment in Summer 2019.

• Reassess Goal 2: Administer the ACAT again in Fall and/or Spring after addressing issues
• f ID number confusion and course identification confusion. Consider assessing Fall and

Spring capstone courses. Over lap students who may have taken capstone in Fall would not be tested. Compare results with 2017-18 findings.

• Use data from enrollments in 200 level Domain courses to review curriculum offerings

each term.

• Collect enrollment information on 300 level courses on juniors/seniors for review of

course completions as well as impact on scheduling course offerings.

• Develop rubrics for assessing 300 level course artifacts for Goal 3 and Goal 4.
• Begin revision of senior survey and alumni survey questions for assessment of Goal 5.