The Teacher Candidate Dispositions Index University of North Texas - - PowerPoint PPT Presentation

Irene Frank Assistant Director, Student Advising Office Brandon Bush Accreditation/Compliance Specialist Lisbeth Dixon-Krauss Associate Dean for Educator Programs Kelley King Associate Professor of Curriculum and Instruction

The Teacher Candidate Dispositions Index

University of North Texas 2016 AACTE Annual Meeting

• University of North Texas

– One of the five largest public universities in Texas. – Research I Status – Spring 2016 – Around 1,000 students certified per year – 24 undergraduate and graduate-level teacher education programs.

Background

• National – CAEP accreditation
• State – “Screening” teacher candidates
• Texas Educator Code of Ethics
• Teacher Disposition Index

– (Schulte, Edick, Edwards, & Mackiel, 2004)

Purpose

• Published teacher disposition measurement tools

reveal unique lists of desired dispositions for each EPP, as well as a wide range in the number of dispositions measured.

(Almerico, 2011)

• Significant overlap

(Alawiye & Williams, 2010; Johnston, Almerico, Henriott, & Shapiro, 2011). – open-mindedness – reflective practice – intellectual curiosity – impartiality

Literature Review

• Each EPP must operationalize desired

dispositions so that both teacher candidates and the individuals evaluating these candidates have a common understanding of indicative behaviors

(Alawiye & Williams, 2010; Almerico et al., 2011).

Literature Review

• In some EPPs, dispositions measurement is used for

gate-keeping.

(Almerico et al., 2011; Brewer, Lindquist, & Altemueller, 2011)

• In some, candidates who continuously have poor

disposition scores are referred to retention committees (Almerico, 2011).

– Remediation/growth plans (Brewer et al., 2011). – If there is a preponderance of evidence & remediation is fruitless, the candidate is dismissed (Almerico et al., 2011; Brewer et al., 2011).

Literature Review

• 1. Which teaching dispositions are important to

faculty at UNT?

• 2. How can a dispositions instrument be developed

and implemented across such a large EPP?

• 3. How can the data from a dispositions instrument

inform programs and administration at a large EPP?

Guiding Questions

• Create the Dispositions/Admissions Committee

– Faculty, Administrative Staff, Clinical Practice Staff, Academic Advisors.

• Reviewed other programs’/institutions' processes
• Decided to create our own, short, instrument combining available

measurement tools.

Timeline

Spring 2014 Summer 2014 Fall 2014 Spring 2015 Fall 2015

• Nine Disposition Instruments consolidated into 19 Items:

– UNT’s Teacher Disposition Index – Texas State’s Fitness to Teach Checklist – Texas A&M’s Disposition Checklist – Greenville College’s Teacher Disposition Behavior Checklist – Northwest Missouri State University’s Disposition Instrument – The Iowa Disposition Model – Niagara University’s Disposition Instrument – University of West Florida’s Disposition Instrument – Idaho State University Teacher Education Program Admission Standards and Indicators

Timeline

Spring 2014 Summer 2014 Fall 2014 Spring 2015 Fall 2015

• Surveyed Faculty

– rate importance of measuring each of the 19 items

• 77 Faculty responded

– Adjunct Faculty – Full-time Faculty – Graduate Assistants

• 7 items eliminated

– Redundant – Limited to the clinical practice setting – Not measurable

Timeline

Spring 2014 Summer 2014 Fall 2014 Spring 2015 Fall 2015

TCDI Alignment

Teacher Candidate Disposition Instrument

InTASC Standards Texas Teacher Standards UNT Conceptual Framework

Timeline

Spring 2014 Summer 2014 Fall 2014 Spring 2015 Fall 2015

• Piloted instrument

– Elementary and Middle School Foundation Course – Science/Math Foundation Course – Student Teacher Supervisors – 65 Teacher Candidates, 5 Faculty

• Faculty comments on pilot instrument:

– “This seems more concise and to the point.” – “Covers the essential elements for effective teaching.” – “Good range of criteria.”

Timeline

Spring 2014 Summer 2014 Fall 2014 Spring 2015 Fall 2015

Timeline

Spring 2014 Summer 2014 Fall 2014 Spring 2015 Fall 2015

• Implemented in student teaching semester with all programs; assessment

completed by both cooperating teacher and university supervisor.

• Implemented at two earlier transition points in undergraduate programs.

Elem./Middle Secondary STEM KINES Beginning EDEE 3320 EDSE 3800 EDSE 3500 KINE 3550 KINE 3560 Middle EDEE 4350 EDRE 4860 EDRE 4870 EDSE 4840 EDSE 4000 KINE 4100 End Clinical Teaching (Student Teaching)

Spring 2015 – Elem./Middle Program TCDI Item Means

Results

TCDI EDEE 3320 EDEE 4350 EDRE 4860 EDRE 4870 Clinical Teaching – Faculty Clinical Teaching – CT N 120 101 83 90 133 133 Q1 2.16 2.33 2.65 2.52 2.77 2.59 Q2 2.17 2.40 2.83 2.64 2.74 2.73 Q3 2.23 2.36 2.76 2.71 2.78 2.71 Q4 0.92 2.47 2.39 2.17 2.63 2.63 Q5 1.53 2.40 2.36 2.44 2.78 2.76 Q6 2.10 2.37 2.70 2.54 2.74 2.63 Q7 2.10 2.42 2.81 2.63 2.80 2.73 Q8 1.25 2.45 2.64 2.14 2.60 2.49 Q9 1.53 2.38 2.80 1.88 2.74 2.51 Q10 2.18 2.40 2.75 2.59 2.79 2.71 Q11 1.53 2.45 2.82 2.36 2.77 2.77 Q12 2.14 2.45 2.82 2.40 2.84 2.79

Results

TCDI EDSE 3800 EDEE 4840 Clinical Teaching – Faculty Clinical Teaching – CT N 136 47 49 49 Q1 2.29 2.72 2.69 2.57 Q2 2.42 2.74 2.73 2.82 Q3 2.46 2.74 2.59 2.80 Q4 1.40 2.74 2.57 2.67 Q5 2.30 2.79 2.63 2.82 Q6 2.27 2.70 2.59 2.61 Q7 2.34 2.74 2.73 2.82 Q8 1.94 2.70 2.51 2.57 Q9 2.21 2.70 2.53 2.59 Q10 2.30 2.74 2.61 2.82 Q11 2.20 2.70 2.45 2.86 Q12 2.31 2.72 2.45 2.84

Spring 2015 – Secondary Program TCDI Item Means

Spring 2015 – STEM Program TCDI Item Means

Results

TCDI EDSE 3500 EDSE 4000 Clinical Teaching – Faculty Clinical Teaching – CT N 34 28 26 26 Q1 1.82 1.82 2.46 2.62 Q2 2.09 2.00 2.42 2.73 Q3 2.06 1.82 2.46 2.62 Q4 0.00 1.04 2.46 2.54 Q5 2.09 1.89 2.50 2.73 Q6 1.76 1.75 2.42 2.54 Q7 2.09 1.93 2.42 2.73 Q8 0.00 0.00 2.23 2.38 Q9 1.94 0.75 2.23 2.42 Q10 1.85 1.86 2.50 2.65 Q11 0.00 2.00 2.15 2.69 Q12 2.09 2.00 2.54 2.69

Spring 2015 – Elementary Program TCDI Reliability

Results

Courses EDEE 4350.001 EDRE 4860.001 EDEE 4350.002 EDRE 4860.002 EDEE 4350.003 EDRE 4860.003 EDEE 4350.004 EDRE 4860.004 EDEE 4350.005 EDRE 4860.005

Role Adjunct Lecturer Adjunct GA GA GA GA Adjunct Adjunct Adjunct N 15 16 14 18 17 Faculty Variance (%) 8.5 26.8 98.3 89.8 2.8 Reliability 0.71 0.17 0.00 0.01 0.53 Absolute SE 0.07 0.15 0.47 0.48 0.11

Note: GA = Graduate Assistant

Results

TCDI across multiple semesters

Student Example EDSE 4840 (middle of the program) Graduate Assistant Comment: Student knows his content very well. He must improve organizational skills. 2 2 2 2 2 2 2 2 2 2 2 2 Total: 24

Results

Secondary Student Teaching (end of the program) Faculty Teacher has progressed as the semester developed. He listened to suggestions and criticisms to help him to become a stronger teacher. Teacher acknowledged areas that he knew he had to work on and did a good job to try to change and develop the suggestions. He has a good understanding of ELA and wants to share that knowledge with students. He will do a good job in the future because he wants to do a good job and wants to teach. 3 2 2 2 2 2 2 2 3 2 3 3 Total: 28 Cooperating Teacher The student continued to struggle with the same things throughout his student teaching. Each week as we sat down to do a reflection together, we had the same conversations again and again. I gave him every suggestion and piece of advice I could come up with from my own experiences, as well as advice I sought out for him through

• ther teachers, but continually saw little improvement in several areas. These areas include thinking on his feet

and adapting lessons in the moment and planning ahead to set up success in a lesson, creating student centered and driven lessons, driving the lesson and being a leader in the classroom, and gaining student respect and buy in. 2 1 2 1 1 1 1 2 2 1 2 2 Total: 18

Results

STEM Student Teaching (end of the program) Cooperating Teacher This student often procrastinated with lesson planning and pre-writing her questions for each lesson as shown when she came unprepared for lessons. This student also struggles to receive constructive criticism and does not seek out professional growth opportunities. This student did not like to be observed because she did not like the feedback portion of the observation. After discussing her mock interview, I suggested finding professional growth

• pportunities and researching best practices and she seemed more open to looking into professional growth in

her time off. Towards the end of student teaching, This student started self-reflecting on lessons, but lacks a growth mindset. This student is courteous in her interactions with peers, colleagues and students and remains rational and calm during conversations. This student was open to learning about educational laws and policies and strived to follow all mandates with integrity. Faculty This student is ready for her own students.

Spring 2016 User Survey

• Electronic questionnaire sent to all users:

Cooperating Teachers, Course Instructors, and Field Supervisors.

• 66 responses.
• Questions included:
• Were you able to fairly and accurately answer most of the questions on the

TCDI? Please explain.

• What would help you complete the TCDI more accurately?
• Did you encounter any problems with a student not addressed by the

TCDI? Please explain.

• Do you have any additional concerns or suggestions regarding this instrument?

User Survey Results

Q1: Were you able to fairly and accurately answer most of the questions

• n the TCDI? Please explain.

100% 73% 96% 100% 92% 0% 27% 4% 0% 8% 0% 20% 40% 60% 80% 100% 120% Cooperating Teacher (26) Course Instructor (12) Field Supervisor* (26) Other (2) TOTAL

Positive and Negative Answers By Role

User Survey Results

Q2: What would help you complete the TCDI more accurately?

28 2 13 10 1 1 1 5 10 15 20 25 30 No Change Write comments More student info Clearer criteria Subject specific C Confidentiality Video Review

Q2 Responses

User Survey Results

Q3: Did you encounter any problems with a student not addressed by the TCDI? Please explain.

• 45 responses indicated no additional problems:

“I think the items comprehensively cover the kinds of dispositional problems I

have encountered in my 30 years as a teacher educator.”

• 2 unrelated to dispositions
• 2 already included on TCDI
• 1 response indicated professional attire
• 1 response indicated personal advocacy

“I would like to see something about student advocacy versus personal

• advocacy. We create a student centered environment, yet the students will soon

be in an environment that expects them to deny their own needs...”

User Survey Results--Comments

Most-Noted Concerns

• Users did not have enough information about students:

“These are discussed with both the mentor and intern, but there is no verification process.”

• Users did not understand the criteria for scoring:

“I'm concerned about the reliability of the assessment scale. What does "Meets expectations" mean? For one instructor it could mean one thing, and differently with

• another. Not only that, how does this apply to students at different levels within their

preparation program?” “Is the ‘not applicable needed’? All of these items are applicable to good teaching.”

User Survey Results--Comments

Additional concerns

• Relationships/confidentiality
• Seeing the instrument in advance

“If I knew the questions in advance I could have a conversation when meeting to try and get more information on these areas."

• Specific wording and interpretation
• Request for a subject specific instrument

Areas of Strength

Key Findings:

Valid & Easy to Use

• Appropriate dispositions identified by instrument
• Progression of growth, ex. 0 2.5
• Users indicated that this was short, yet comprehensive

Implications:

Program can use to flag for intervention:

• Admission, Review and Retention Committee
• Growth Plans

Areas for Further Development

Key Findings:

Inter-Rater Reliability is low. Ex.: Supervisors vs. Mentor Teachers Users want clarification for scoring

Implications:

Develop rubric (Performance level descriptors) Train users (especially UNT)

• Alawiye, O., & Williams, H. (2010). Disposition profile inventory: An assessment tool for

measuring the professional attitudes and behaviours of teacher education candidates. National Social Science Journal, 34(2). Retrieved from http://www.nssa.us/journals/2010-34-2/2010-34-2-01.htm

• Almerico, G. M. (2011). Pre-service teacher dispositions at work. Research In Higher

Education Journal, 12, 1-22. http://www.aabri.com/manuscripts/11830.pdf

• Brewer, R. D., Lindquist, C. & Altemueller, L. (2011). The dispositions improvement
• process. International Journal of Instruction, 4(2), 51-68.
• Johnston, P., Almerico, G. M., Henriott, D., & Shapiro, M. (2011). Descriptions of

dispositions for assessment in pre-service teacher education field experiences. Education, 132(2), 391-401.

• Schulte, L.E., Edick, N., Edwards, S., and Mackiel, D. (2004). The development and

validation of the Teacher Dispositions Index," Essays In Education, 12. Retrieved from http://www.usca.edu/essays/vol122004/schulte.pdf

References