WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 SMALL PROJECT EVALUATION March 23, 2016 INTRODUCTIONS Lori Elaine Charlotte Russell Miranda Wingate Craft Forrest Cannon Lee evalu ‐ ate.org 1
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 This material is based upon work supported by the National Science Foundation under grant number 1204683. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the presenters and do not necessarily reflect the views of NSF. BEHIND THE SCENES Sharon Dennis Mike Janet Tim Gusky Faber Lesiecki Pinhorn Suchomski evalu ‐ ate.org 2
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 MATERIALS Slides Handout Recording www.evalu ‐ ate.org/webinars/2016 ‐ mar/ SE TTING THE STAGE 1 Tales from the Trenches Question Break EVALUATION SCOPE 2 Tales from the Trenches Question Break DATA COLLECTION & DIVISION OF LABOR 3 Tales from the Trenches Question Break WORKING WITH INSTUTIONAL RESEARCH 4 Tales from the Trenches Question Break ATE PROGRAM & MENTOR-CONNECT 5 Tales from the Trenches evalu ‐ ate.org 3
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 SE TTING THE STAGE EVALUATION the determination of something’s quality, value, or importance evalu ‐ ate.org 4
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 1. Ask important 2. Gather evidence questions about a that will help project’s processes answer those and outcomes. questions. EVALUATION 4. Use the 3. Interpret data information for and answer the accountability, evaluation improvement, and questions. planning. Gwen Generickson soon to be a new ATE project principal investigator Bio ‐ Inspired Solutions to Human Challenges $198,913 | 2016 ‐ 19 This is a fictional project. Any resemblance to actual persons or projects is purely coincidental. evalu ‐ ate.org 5
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 She’s been asked to submit a revised evaluation plan. Chat What’s the FIRST thing she should do to get started? evalu ‐ ate.org 6
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 Read ATE program solicitation: EVALUATION: All projects and centers carry out evaluative activities. The funds to support an evaluator independent of the project or center must be requested, and the requested funds must match the scope of the proposed evaluative activities. INTELLECTUAL MERIT: Is the evaluation plan clearly tied to the project outcomes? Is the evaluation likely to provide useful information to the project and others? The Project Description must begin with the subsection on Results from Prior NSF Support …. This subsection must contain specific outcomes and results including metrics to demonstrate the impact of the project activities. How much is it going to cost? evalu ‐ ate.org 7
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 ! CAUTION The following budget examples are for illustrative purposes only. The information should not be construed as recommendations or guidelines for evaluator costs or time commitments. evalu ‐ ate.org 8
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 Bio ‐ Inspired Solutions to Human Challenges Project Budget Category Cost Total direct costs Total direct costs Salaries & Fringe Benefits $103,500 before before external external Equipment $12,000 evaluation evaluation Materials $20,000 =$139,100 =$139,100 Travel $3,600 X 10% X 10% = $13,910 = $13,910 Other – Evaluation $13,910 Consultant Modified Total Direct Costs $153,010 Indirect Costs (30% ) $45,903 TOTAL PROJECT COST $198,913 Bio ‐ Inspired Solutions to Human Challenges Project Budget $13,910 over 3 years = $4,637 per year But m more e effort w will b be r required i in Y Year 1 1 to s set u up the eval the evalua uation on, so… , so… Year 1: $4,800 Year 2: $4,000 Year 3: $3,600 evalu ‐ ate.org 9
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 Bio ‐ Inspired Solutions to Human Challenges Project Budget Category Year 1 Year 2 Year 3 Consultant fees $4,800 $4,000 $3,600 Travel $500 $500 $500 TOTAL EVALUATION COST $5,300 $4,500 $4,100 Annual site visits are important! Annual site visits are important! Bio ‐ Inspired Solutions to Human Challenges Project Budget Category Year 1 Year 2 Year 3 Consultant fees $4,800 $4,000 $3,600 Travel $500 $500 $500 TOTAL EVALUATION COST $5,300 $4,500 $4,100 @ @ $100 per hour, how many days $100 per hour, how many days can the external evaluator devote can the external evaluator devote to this project? to this project? evalu ‐ ate.org 10
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 Bio ‐ Inspired Solutions to Human Challenges Project Budget Category Year 1 Year 2 Year 3 Consultant fees $4,800 $4,000 $3,600 Travel $500 $500 $500 TOTAL EVALUATION COST $5,300 $4,500 $4,100 Year 1: $4,800 = 48 hours or 6 days Year 2: $4,000 = 40 hours or 5 days Year 3 : $3,600= 36 hours or 4.5 days Answer the Poll How I can use my evaluation resources efficiently? evalu ‐ ate.org 11
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 ENHANCING EVALUATION EFFICIENCY Match the scope of the evaluation to the scope of the project. Involve the project team in monitoring the project and implementing the evaluation. Use institutional data to the extent possible. TALES FROM THE TRENCHES evalu ‐ ate.org 12
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 EVALUATION SCOPE ENHANCING EVALUATION EFFICIENCY Match the scope of the evaluation to the scope of the project. Involve the project team in monitoring the project and implementing the evaluation. Use institutional data to the extent possible. evalu ‐ ate.org 13
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 1. Ask important 2. Gather evidence questions about a that will help project’s processes answer those and outcomes. questions. EVALUATION 4. Use the 3. Interpret data information for and answer the accountability, evaluation improvement, and questions. planning. evalu ‐ ate.org 14
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 EVALUATION QUESTIONS should be evaluative pertinent reasonable specific answerable and the set of questions should be complete Engineering technicians: High demand, low supply evalu ‐ ate.org 15
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 SCI 152: Bio ‐ Inspired Solutions to Human Challenges Biology | Design | Economics | Engineering | Materials Science Buteo College 60% female Engineering 15% female Technology & Preengineering evalu ‐ ate.org 16
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 Project Goal: Attract female students to the course and motivate them to pursue degrees in engineering technology or preengineering Complete course curriculum Develop promotional video Raise awareness among Outreach to female students with college personnel undeclared majors evalu ‐ ate.org 17
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 PROJECT LOGIC MODEL SHORT-TERM MID-TERM ACTIVITIES OUTPUTS OUTCOME S OUTCOME S ACTIVITIES what a project does 52 PROJECT LOGIC MODEL SHORT-TERM MID-TERM ACTIVITIES OUTPUTS OUTCOME S OUTCOME S Develop course curriculum Develop overview video Make presenta ‐ tions to faculty, advisors, and admissions counselors Outreach to undeclared, female students 54 evalu ‐ ate.org 18
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 PROJECT LOGIC MODEL SHORT-TERM MID-TERM ACTIVITIES OUTPUTS OUTCOME S OUTCOME S OUTPUTS tangible products of activities PROJECT LOGIC MODEL SHORT-TERM MID-TERM ACTIVITIES OUTPUTS OUTCOME S OUTCOME S Develop course Course materials curriculum Promotional video Develop overview video Make presenta ‐ tions to faculty, advisors, and admissions counselors Outreach to undeclared, female students evalu ‐ ate.org 19
WEBINAR Small Project Evaluation: Principles and Practices 3/23/2016 PROJECT LOGIC MODEL SHORT-TERM MID-TERM ACTIVITIES OUTPUTS OUTCOME S OUTCOME S OUTCOMES changes brought about through project activities and outputs PROJECT LOGIC MODEL SHORT-TERM MID-TERM ACTIVITIES OUTPUTS OUTCOME S OUTCOME S Develop course Course materials Students enroll in curriculum course Promotional video Student interest Develop overview video in pursuing STEM degree increases Make presentat ‐ ions to faculty, advisors, and admissions counselors Outreach to undeclared, female students evalu ‐ ate.org 20
Recommend
More recommend
