Using the Wheel Without Reinventing It: How to examine Noyce - - PowerPoint PPT Presentation

1

Using the Wheel Without Reinventing It:

How to examine Noyce projects Part II

Presented by: John Sutton, Ph.D. Brandie Ward Emma Espel, Ph.D.

1

2

Goals

We hope you leave the webinar better prepared to answer two questions: What instruments could I use for my Noyce project evaluation?

3

Who are we?

External evaluators from RMC Research Corporation in Denver, CO 6 current Noyce projects evaluations across CO, NE, and LA

• 1 capacity building project
• 2 scholarship and stipend projects
• 2 TF/MTF projects
• 1 evaluation of the Midwest Regional Connections

Noyce project

4 4

5

Outline

5

Brief review of Noyce grant types Common guidelines for what makes a good instrument Rubric review Instruments & Dialogue Let’s share!

6

Capacity Building Scholarship & Stipend

(S&S)

Teaching Fellows (TFs) /Master Teaching Fellows (MTFs)

Conferences/ Events

Noyce Project Types

7 7

What is your primary role with a Noyce project?

• a. PI or Co-PI
• b. Evaluator
• c. Project Staff
• d. Other?

Webinar Poll #1

8

Imagine yourself…as the evaluator for a TF/MTF program funded for five years. Your evaluation plan includes annual online surveys of teachers about their content knowledge, instructional strategies used, and demographics.

After three years of program implementation and evaluation, the TF/MTF program leaders would like the survey to collect information about teachers’ agreement with productive or unproductive statements about the teaching and learning of mathematics. They are also interested in the extent to which their students are achieving the Standards for Mathematical Practices.

How might you customize the survey to include these new scales, three years into the project? What are some considerations you need to think about when adding the scales?

What would you do?

z

Chat

9 9

Perhaps…

Retrospective & Concurrent

• Identify participants’ agreement with the

statements prior to participation in the program

• Ask participants to reflect on whether or not their

agreement has changed over the past three years.

• Include items with concurrent wording in all

upcoming assessments in order to monitor moving forward.

• Do any previous measures tap similar constructs?

10

Common guidelines for what makes a good instrument

10

11 11

What qualities do you look for in a good instrument?

z

Chat

12 12

Perhaps…

Instrument content that helps answer evaluation questions Appropriate for the context

• type of respondents

Appropriate psychometric properties

• reliability, validity, wording

13

Review of our evaluation rubric

13

14 14

Provide an example of an evaluation question you may commonly use across any type of evaluation that addresses stakeholders’ attitudes & beliefs about the project.

z

Chat

15

Construct

Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences)

To what extent…

Implementation

are scholarships/ stipends awarded according to review criteria?

Attitudes & Beliefs

do stakeholders view the project as a viable

• ption for partner

institutions? do scholars perceive the project as implementing a high quality teacher preparation program focused on STEM? do TFs/MTFs apply new ideas, knowledge, and information in classroom practice? do participants in sponsored activities perceive new ideas, knowledge, or information as useful and applicable?

Content Knowledge

does the scholarship project have an impact

• n scholars’ Praxis

scores?

Student Outcomes

do TFs/MTFs impact student outcomes?

Satisfaction

are participants satisfied with event activities?

Sustainability has progress been

made towards sustaining or “scaling up” project activities?

15

16

Construct

Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences)

To what extent…

Implementation

are scholarships/ stipends awarded according to review criteria?

Attitudes & Beliefs

do stakeholders view the project as a viable

• ption for partner

institutions? do scholars perceive the project as implementing a high quality teacher preparation program focused on STEM? do TFs/MTFs apply new ideas, knowledge, and information in classroom practice? do participants in sponsored activities perceive new ideas, knowledge, or information as useful and applicable?

Content Knowledge

does the scholarship project have an impact

• n scholars’ Praxis

scores?

Student Outcomes

do TFs/MTFs impact student outcomes?

Satisfaction

are participants satisfied with event activities?

Sustainability has progress been

made towards sustaining or “scaling up” project activities?

16

17

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

17

18

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

18

19

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

19

20

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

20

21

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

21

22

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

22

23

Questions?

24

Instruments & Dialogue

24

25 25

Types of Measures

Survey – quantitative and/or qualitative Focus Group – qualitative Interviews – qualitative Document Review – quantitative and/or qualitative

26 26

Examples

27

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

27

28

Self-efficacy scale on an MTF survey

28

Yopp, D., Burroughs, B., Luebeck, J., ….& Swackhamer, L. (2010). Construct Reliability and Validity of Selected EMC Instrumentation. Montana State University & RMC Research Corporation

29

Self-efficacy scale items

Confidence in Teaching Mathematics Ability to Teach Mathematics Support

• f Mathematics Teaching

What is your level of confidence: What is your ability level: How supported do you feel: for learning mathematics at the college level? to respond to difficult mathematics questions from your students? for working with fellow teachers during the regular school week on math curriculum and/or instruction? for teaching mathematics at the middle school level or above? to gauge student comprehension of a mathematics lesson you just taught? for working with knowledgeable peers to increase your mathematics content knowledge? in your mathematics content knowledge? to craft good mathematics questions for your students? to learn new things about mathematics pedagogy in your present job? in your mathematics pedagogical content knowledge? for adjusting your mathematics lesson to the proper level for individual students? from colleagues to try out new ideas in teaching mathematics? for demonstrating effective math lessons to your peers? for using a variety of mathematics assessment strategies? to attend mathematics specific professional development sessions? that your mathematics content knowledge is above the level of your peers? for providing an alternative explanation

• r example when your mathematics

students are confused? from the school administration for teaching mathematics?

29

30 30

Self-efficacy scale development

18 items reduced to 14 3 factors (confidence, ability, and support)

Pilot Sample

• N = 171
• range 14 to 112
• M = 70.05, SD = 20.35
• verall scale α = .89
• confidence α = .83
• ability α = .89

MTF Noyce Participants

• N = 24
• confidence α = .71 - .82
• ability α = .82- .94

31 31

z

Chat

What qualities make a good survey item?

32 32

Perhaps…

Item wording

• no double barreled items
• interpretable
• appropriate level
• no jargon used
• content validity
• reverse worded items

Response scaling

33 33

z

Chat

What is a strength with using this scale? What is a challenge with using this scale?

Extremely Low 1 2 Below Average 3 4 5 Above Average 6 7 Extremely High 8

34 34

Would you change the scale? Why or why not? What would you change it to?

z

Chat

35 35

Would you change the scale? Why or why not? What would you change it to?

z

Chat

Webinar Polls

36 36

Perhaps…

Number of points

• 3-, 4-, 5-, 8-points, etc.

Anchors

• strength or frequency (low to high)
• agreement (disagree to agree)

Direction

• negative  positive
• neutral  positive

37 37

How might this scale be used in a capacity building project?

z

Chat

38 38

Perhaps…

Modifications

• as a needs assessment
• gather baseline data
• decide not to use it
• focus on one construct (e.g., confidence)
• use it with faculty, rather than potential

teachers

• pen-ended items to probe responses

Considerations

• ther efficacy scales more useful to you?

39

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

39

40 40

Motivation

Photo: https://flic.kr/p/dtuQ6g

TF/MTF

Interview: Based on your degree or experience as a classroom teacher, what do you suggest would motivate a teacher to want to participate in an MTF program to enhance their knowledge and skills to teach math and to mentor other teachers? Focus Group: In what ways has the project increased your motivation to attend the [IHE] to graduate with a grade 9-12 teaching license in secondary education?

Capacity Building

41 41

What are pros and cons of asking a question like this using a focus group or interview question?

z

Chat

42 42

Perhaps…

Pros

• response detail
• follow-up opportunities (e.g., tell me

more…) Cons

• not quantifiable
• requires more cognitive processing,

so it might be skipped

43 43

What elements would you include in an assessment of the program’s impact

• n motivation to teach STEM?

z

Chat

44 44

Perhaps…

Motivation for TFs/MTFs can be assessed with questions about:

• engagement
• planning
• instructional strategies

45

Construct Capacity Building Scholarship/ Stipend Program TF/MTF Events (e.g., Conferences) To what extent… Attitudes & Beliefs How does the program impact attitudes and beliefs? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact students’ interest in STEM? does the program impact students’ motivation to engage in STEM? do recruited students feel they have the ability to engage in STEM (self-efficacy)? do stakeholders view the project as a viable option for partner institutions (preparedness)? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact scholars’ interest in teaching STEM? does the program impact students’ motivation to teach STEM? does the program impact participants’ self-efficacy in learning STEM? do scholars feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what scholars are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the project as implementing a high quality teacher preparation program focused

• n STEM?

does the program impact TFs/MTFs’ interest in teaching STEM? does the program impact TFs/MTFs’ motivation to teach STEM? does the program impact participants’ self-efficacy in teaching STEM? do TFs/MTFs feel prepared to teach (e.g., STEM content, instructional practices, teaching in high needs schools)? do challenges exist in transferring what TFs/MTFs are learning into the classroom (and what are they?) do attitudes and beliefs about teaching and learning of STEM change? do stakeholders perceive the event as high quality? does the event impact participants’ interest in STEM? does the event impact participants’ motivation to engage in STEM? does the event impact participants’ self-efficacy in STEM? do participants feel better prepared to more effectively participate in and contribute to their program activities? do challenges exist in utilizing new ideas, knowledge, or information from events (and what are they)?

45

46 46

How would you operationalize what high quality means for projects?

z

Chat

47 47

Perhaps…

High quality

• goals are met / progress towards goals
• implemented with fidelity
• program is useful to stakeholders
• appropriate amount and type of content
• instruction elicits student learning

48 48

PI Interview: Please describe the progress the grant has made towards reaching each of its’ goals. I will read the goal and please describe related activities and any evidence you have seen of progress. (PROBE: Provide more specifics about the nature, intensity, frequency, and/or impact of these activities).

High quality in any project

GOAL A  GOAL B  GOAL C  GOAL D 

49

Recap

49

Brief review of Noyce grant types

 C a p a c i t y B u i l d i n g , S & S , T F / M T F , E v e n t

Common guidelines for what makes a good instrument

 C o n t e n t , C o n t e x t , P s y c h o m e t r i c P r o p e r t i e s

Rubric review

 W h a t d o y o u w a n t t o m e a s u r e a n d h o w ?

Instruments & Dialogue

 A t t i t u d e s & B e l i e f s  I n t e r v i e w s , F o c u s G r o u p s , S u r v e y s , D o c u m e n t R e v i e w  S c a l e s , C o n s i d e r a t i o n s , M o d i f i c a t i o n O p t i o n s

Let’s share!

50

Let’s share!

50

51 51

Instruments we used to pull our examples from are posted here:

Midwest Regional Noyce Community on LinkedIn

https://www.linkedin.com/groups/Midwest-Regional-Noyce-Community-6692728/about

Midwest Regional Connections website

http://scimath.unl.edu/noyce/conferences/

52

Email your example(s) and we’ll compile them to share John Sutton sutton@rmcres.com Brandie Ward ward@rmcres.com Emma Espel espel@rmcres.com

Imagine the possibilities if everyone shared one example!

53

Questions?