school-age pqa external methods? What might create an obstacle to - PDF document

Before we Assess Check your bias!  What triggers your judgment?  What styles of instruction do you tend to like or not like? What about school-age pqa external methods?  What might create an obstacle to assessor reliability your being able to be objective? training Remember, the goal is to collect Training Tips Slides objective data ! Trust the tool to do the assessment. David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training The Role of an External Assessor Overall Program Quality  You are here to be a reliable rater.  This means your scores provide a touchstone or point of reference for self-assessment scores.  Being reliable means you have a more complete understanding of the language and intent of each item in the tool.  Set aside your biases. Recognize what you are and are not scoring — see next slide.  Score the tool as defined and written. David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training Tips for Completing and Scoring the PQA Quality Continuum for an Individual Item  Use the Observation Guide when you observe; it lists every item and will help trigger your memory and remind you to ask follow up questions when necessary.  Always try to see multiple items in every interaction and cross-reference constantly.  Look to a preponderance of evidence but favor higher scores; give programs the benefit of the doubt . Remember: NO 2 ’ s or 4 ’ s.  Fill in the front pages! All the site data is necessary: YOU are part of our growing sample! David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training 1

More Tips on Scoring the PQA Tips for These Physical Safety Scales  Only ask follow-up questions for items with  Nourishment : Water fountain in hallway OK. question(s) in the evidence column. Ask the  Emergency Preparedness: Only score additional safety questions as they are printed. equipment if safety equipment is necessary for activity.  Anecdotal evidence needs to be entered for every  Emergency Preparedness: Only score outdoor program row in order to score. space if outdoor space is present and utilized.  Emergency Preparedness: If session is completely  Score an X if specified items or scales are not outdoors, mark X for supervised indoor entrances. to be scored . (See notes for which items/scales  Emergency Preparedness: Note, if the program is outside, this is allowed — i.e. managing feelings scale if no the emergency procedures and fire extinguisher may be disruptive feelings or conflict are present.) inside — score 3 if it is accessible.  Read each level of the item carefully. Use  Emergency Preparedness: Doors locked from outside do handbook and Frequently Confusing Items sheet! not need to be monitored in the same ways as a main door. David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training Tips for Emotional Safety Tips for Warm Welcome  “ Warm and Respectful ” and “ Positive Body  This scale is one of the most potentially subjective. Language ” are potentially subjective items.  Collect as much evidence as possible and assess  Look for evidence of disrespect, sarcasm, insults, the overall climate. dismissive hand gestures, etc.  Observe both children and staff in order to collect  A wide variety of styles can still be assessed as a “ 5. ” evidence:  Item 1: Opening “ hurrah ” does not count as a  Interactions between children and staff. greeting. With a greeting, youth presence is  Interactions among children. acknowledged, as individuals or a group.  Individual children behavior David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training Tips for Session Flow Tips for Active Engagement  Item 2: Materials simply need to be in the room and  “ Active Learning ” is not necessarily “ hands-on ” easily accessible for them to be “ ready. ” learning. Includes children actively engaging with  They are NOT ready if staff or youth have to leave the ideas. room to get them.  Item 2: “ Talking about what they are doing ” must  Item 4: To assess clear explanations and be a structured opportunity, not something that just appropriate time, observe how youth react to staff: happens.  What do they do after instructions are given?  What do they say or do to indicate whether they feel rushed, bored, or the pace is just right?  Item 5: Use evidence from activities observed rather than activities that are planned. David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training 2

Tips for Skill Building Tips for Encouragement  Item 1: Non-evaluative language:  Most of this scale relies on a specific skill or set of  Praise is not “ bad. ” It is usually present in a session that skills being addressed in the session. is assessed with a “ 5. ”  Listen for staff to follow-up an evaluative statement with  Item 1: What youth are going to “ do ” is not specific, non-evaluative language. necessarily the same as what they will “ learn. ”  Look for at least one instance: a specific praise, an  Note how staff frames learning objective. objective description of what the child did, staff asking  Items 2 & 5: Observing youth behavior will give child to describe his or her work you clues about whether or not they are being  Item 2: Take notes on all of the questions you hear. challenged to improve and/or struggling. Determine whether or not they were open-ended when you  Note staff responses to wrong answers and errors. go to score the tool.  Observe staff interactions with youth to assess the scale. David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training Tips for Child-Centered Space Tips for Responsibility  2 nd item should be scored based on evidence from When to use : 1 st item.  This scale envisions a space that is arranged for a  Focus here is on routine activities – distinct from particular type of program that uses interest centers where children select and direct their own work. group process skills or projects.  Mostly for younger children (K-2).  Tasks that support communal responsibility.  Interest centers : Supplies set in particular corner, space, set apart. Has variety of supplies around a particular theme, type of activity. David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training Tips for Managing Feelings Tips for Belonging  More focused on feelings than Youth PQA  Item 1: Look for structured opportunities to get to Reframing Conflict. (That ’ s why it ’ s in the know each other better. Interaction domain.)  Item 2: Carefully observe how children interact with  Determine whether or not to score the scale. If no each other. strong feelings or serious negative behavior that  Observe complete interactions whenever possible. disrupts learning, do not score. Put Xs.  Item 3: Can be a difficult item for programs to  If scoring, fit the real scenario to the scoring criteria achieve a high assessment score. the best you can.  “ Identify strongly ” = identify positively David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training David P. Weikart Center for children Program Quality : School-Age PQA External Assessor Reliability Training 3