WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan GREAT EVALUATION PLAN The webinar will begin at 1 pm Eastern www.atecentral.net evalu ‐ ate.org 1
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Webinars Resource Blog ATE Survey Data Library www.evalu ‐ ate.org Materials Slides Evaluation Plan Checklist Recording and Other Resources evalu ‐ ate.org 2
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Introductions Mike Lori Lesiecki Wingate Behind the Scenes Emma Lyssa Sharon Cynthia Janet Shannon Wilson Becho Perk Gusky Williams Pinhorn Payne evalu ‐ ate.org 3
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan This material is based upon work supported by the National Science Foundation under grant number 1600992. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the presenters and do not necessarily reflect the views of NSF. Webinar Overview Lori Wingate 1 2 evalu ‐ ate.org 4
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan RESOURCE Evaluation Plan Checklist for ATE Proposals ADVANCED TECHNOLOGICAL EDUCATION www.nsf.gov/ate evalu ‐ ate.org 5
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Aviation Summer Camp PROJECT EXAMPLE AV 100 Intro to Aviation AV 150 Applied Mathematics for Aviation 1 evalu ‐ ate.org 6
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Project Description (15 pages) Evaluation Plan (1 ‐ 2 pages) evalu ‐ ate.org 7
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Evaluation Plan (1 ‐ 2 pages) Evaluator Evaluation Questions Data Communication and Use Timeline evalu ‐ ate.org 8
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Evaluation Plan Evaluator Identify the project’s evaluator Describe the evaluator’s qualifications Refer to the evaluator’s biosketch and letter of collaboration The funds to support an evaluator independent of the project or center must be requested… evalu ‐ ate.org 9
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Evaluators are professionals Photo credit: American Evaluation Association | www.eval.org A qualified ATE evaluator will have… Experience evaluating STEM education projects Strong research skills Strong communication skills and a service orientation Understanding of NSF and 2 ‐ year ‐ college contexts What counts as independent? Big, Wide World College Department Project E E E E OK if they work in a separate unit E Not OK if they do have another role on the project evalu ‐ ate.org 10
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan How to Find an Evaluator American Evaluation Association’s evaluator directory ATE evaluator map Local universities Recommendations from colleagues POLL: Which proposal has the best description of the evaluator? Proposal A Proposal B Proposal C Delores Stormborn will lead Lannister University’s Center Julia Snow will serve as this the project’s external for Evaluation will conduct project’s external evaluator. evaluation. She has a Ph.D. the project’s evaluation. This She leads the college’s in social psychology and is Center has been a leading faculty development center, CEO at The Stormborn evaluation service provider providing guidance to Evaluation Group. She has since 1975 and has several instruction and assessment. conducted 20 STEM prominent evaluators on its She serves as chair of the education evaluations, staff, as well as a cadre of college’s Student Success including several in the ATE capable graduate students. Committee, and has program. Dr. Stormborn’s When the project is funded, coordinated data collection biosketch and commitment we will work with the for several federal grants. letter are included in the evaluators there to further supplementary documents develop and implement the section of this proposal. project’s evaluation plan. evalu ‐ ate.org 11
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan RESOURCE Finding and Selecting an Evaluator for ATE Proposals RESOURCE Evaluator Biographical Sketch Template for NSF Proposals evalu ‐ ate.org 12
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Evaluation Plan Evaluation Questions List the key questions that the evaluation will address Include questions about both project implementation and outcomes Ensure that questions align with the project’s goals and activities evalu ‐ ate.org 13
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Logic Models PROJECT LOGIC MODEL MID ‐ TERM LONG ‐ TERM SHORT ‐ TERM ACTIVITIES OUTCOMES OUTCOMES OUTCOMES What the project DOES, CREATES, DELIVERS evalu ‐ ate.org 14
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan PROJECT LOGIC MODEL MID ‐ TERM SHORT ‐ TERM LONG ‐ TERM ACTIVITIES OUTCOMES OUTCOMES OUTCOMES CHANGES a project brings through its activities DIFFERENCE it makes for individuals, organizations, communities, society PROJECT LOGIC MODEL MID ‐ TERM LONG ‐ TERM SHORT ‐ TERM ACTIVITIES OUTCOMES OUTCOMES OUTCOMES Years 1 ‐ 3 Years 2 and 3 Y ears 2 & 3 Post ‐ project Develop and More current and More students Graduates gain offer Aviation incoming students graduate from employment as Summer Camp enroll in college’s aviation tech aviation for Grades 6 ‐ 12 aviation tech program technicians program Develop and Graduates offer Intro to transfer to More students Aviation course aviation persist in engineering program Develop and programs at four ‐ offer Applied year colleges Mathematics for Aviation course evalu ‐ ate.org 15
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan EVALUATION QUESTIONS ACTIVITIES Develop and 1. To what extent are the camp and courses offer aviation achieving project targets in terms of summer camp student numbers, diversity, and for grades 6 ‐ 12 satisfaction? Develop and offer Intro to 2. What are the strengths and weaknesses of Aviation course the camp and courses? Develop and offer Applied Mathematics for Aviation course ACTIVITIES Develop and offer aviation summer camp for grades 6 ‐ 12 Develop and offer Intro to Aviation course An expert advisory panel will provide Develop and formative feedback to ensure courses meet offer Applied academic standards and industry needs Mathematics for Aviation course evalu ‐ ate.org 16
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan EVALUATION SHORT ‐ TERM QUESTIONS OUTCOMES More current and 3. To what extent and how are incoming students project activities impacting enroll in college’s enrollment and persistence aviation tech program in the aviation program? More students persist in program MID ‐ TERM EVALUATION QUESTIONS OUTCOMES More students 4. To what extent and how are graduate from project activities impacting aviation tech the number, percentage, and program diversity of students graduating from the aviation program? evalu ‐ ate.org 17
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan LONG ‐ TERM OUTCOMES Graduates gain employment as aviation technicians Need to start collecting data on these outcomes, Graduates transfer to although outcomes won’t be aviation realized until after grant expires engineering programs at four ‐ year colleges PROJECT LOGIC MODEL MID ‐ TERM LONG ‐ TERM SHORT ‐ TERM ACTIVITIES OUTCOMES OUTCOMES OUTCOMES Years 1 ‐ 3 Years 2 ‐ 3 Years 3 & beyond Post ‐ project Develop and More current and More students Graduates gain offer Aviation incoming students graduate from employment as Summer Camp enroll in college’s aviation tech aviation for Grades 6 ‐ 12 aviation tech program technicians program Develop and EQ 4 Graduates offer Intro to transfer to More students Aviation course aviation persist in Outcome engineering program Develop and programs at four ‐ offer Applied EQ 3 year colleges Mathematics for Aviation course Outcome EQs 1 ‐ 2 Important to include any additional questions of Implementation interest to stakeholders! evalu ‐ ate.org 18
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan Why not just ask if goals were achieved? POLL POSSIBLE PROJECT GOAL: Offer an aviation summer camp for grades 6 ‐ 12, serving at least 50 students per year. evalu ‐ ate.org 19
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan POLL POSSIBLE PROJECT GOAL: Expand marketing of the college’s aviation technology program. POLL POSSIBLE PROJECT GOAL: Increase the pool of graduates who are prepared for careers in aviation. evalu ‐ ate.org 20
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan RESOURCE Evaluation Questions Checklist RESOURCE Logic Model Template for ATE Projects evalu ‐ ate.org 21
WEBINAR 8/22/2018 Give Your Proposal a Competitive Edge with a Great Evaluation Plan RESOURCE Logic Models: Getting Them Right and Using Them Well (webinar recording and handouts) evalu ‐ ate.org 22
Recommend
More recommend
