WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 1
Lori
Wingate
Miranda
Lee
INTRODUCTIONS
Janet
Pinhorn
Mike
Lesiecki
Tim
Suchomski
INTRODUCTIONS Mike Janet Tim Lori Miranda Wingate Lesiecki - - PDF document
WEBINAR 8/17/2016 Logic Models: Getting Them Right and Using Them Well INTRODUCTIONS Mike Janet Tim Lori Miranda Wingate Lesiecki Pinhorn Suchomski Lee www.evalu ate.org 1 WEBINAR 8/17/2016 Logic Models: Getting Them Right and Using
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 1
Lori
Wingate
Miranda
Lee
INTRODUCTIONS
Janet
Pinhorn
Mike
Lesiecki
Tim
Suchomski
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 2 This material is based upon work supported by the National Science Foundation under grant number 1204683. Any opinions, findings, and conclusions
material are those of the presenters and do not necessarily reflect the views of NSF.
Slides
www.evalu‐ate.org/webinars/2016‐aug/
MATE RIALS
Recording Handout
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 3
ADVANCE D TE CHNOLOGICAL E DUCATION (ATE )
www.nsf.gov/ate
ATE PROGRAM
CE NTE RS RE SE ARCH PROJE CTS
National Centers Regional Centers Support Centers Planning Exploratory Research Full‐Scale Research Program Dev. & Improvement Educational Materials Dev. Professional Dev. for Educators Leadership Capacity Building Teacher Preparation Small Grants for Institutions New to ATE Business & Entrepreneurial Skills Development Conferences & Workshops ATE Coordination Networks Planning GrantsWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 4 17 6 Proposals due Today External evaluator should be lined up
ATE Proposal Development and E valuation Planning
CRE ATING A LOGIC MODE L
INTE GRATING A LOGIC MODE L INTO A FUNDING PROPOSAL
Question Break Question Break
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 5
Lori Wingate
CRE ATING A LOGIC MODE L
WHAT IS A LOGIC MODE L AND WHY SHOULD WE CARE
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 6
INPUTS ACTIVITIE S OUTPUTS SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S LONG-TE RM OUTCOME S NSF funding Existing resources on logic models MATEC’s hosting services and technical expertise Community College Liaison Panel member feedback Advertise webinar Develop, rehearse, and revise webinar Webinar recording Presentation slides Resource handout Participants know how to create a logic model Participants know how to integrate logic models into funding proposals Participants develop good logic models and include them in their proposalsE XAMPLE : LOGIC MODE L OF THIS WE BINAR
Deliver webinar Participants’ projects are funded ATE’s mission is advanced INPUTS NSF funding Existing resources on logic models MATEC’s hosting services and technical expertise Community College Liaison Panel member feedbackINPUTS
resources and assets that will be utilized by the project
Focus on special, important assets Be specific
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 7
ACTIVITIE S Advertise webinar Develop, rehearse, and revise webinar Deliver webinarACTIVITIE S
what a project does
Focus on delivery and development
Omit project management/ administrative activities Be specific If there are several activities, group related ones together to illustrate structure
OUTPUTS Webinar recording Presentation slides Resource handoutOUTPUTS
tangible products of activities
Highlight tangible evidence of service delivery Include numbers only if known or certain thresholds are necessary to bring about outcomes
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 8
Participants know how to create a logic model Participants know how to integrate logic models into funding proposalsImmediate changes brought about by project activities and outputs
SHORT-TE RM OUTCOME S
SHORT-TE RM OUTCOME S Focus on changes in individual‐ level knowledge, skills, attitudes,
MID-TE RM OUTCOME S
Changes that occur due to short‐term outcomes
Articulate link between immediate, individual
to improve‐ ments in broader context (long‐term
Focus on target audience, not project
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 9
LONG-TE RM OUTCOME S Participants’ projects are funded ATE’s mission is advancedLONG-TE RM OUTCOME S
Changes in problematic situation that led to creation of project
Focus on improvements in broader contexts—beyond immediate service recipients Frame in terms of addressing the original need for the project
LONG-TE RM OUTCOME S Participants’ projects are funded ATE’s mission is advancedOUTCOME S
SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S Participants know how to create a logic model Participants know how to integrate logic models into funding proposals Participants develop good logic models and include them in their proposals Represent important changes or improvements Realistic Logically linked Appropriately sequenced Are not about the project itself
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 10
LOGIC MODE L USE S
Include in funding proposal Evaluation planning Project management Communication to stakeholders
Building a Logic Model for the Bio-Inspired Solutions to Human Challenges Project
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 11
Engineering technicians: High demand, low supply
SCI 152: Bio‐Inspired Solutions to Human Challenges
Biology | Design | Economics | Engineering | Materials Science
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 12
Project Goal: Attract students with undeclared majors to the course and motivate them to pursue degrees in engineering technology or preengineering
Make presentations to college personnel Outreach to students with undeclared majors Develop promotional video Complete course curriculumWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 13 Simulation Software Expert Panel Videography Internship Program
The main activities include (a) completion of the course curriculum; (b) creation of a short video about bio‐ inspired engineering; (c) presentations to admissions counselors, advisors, and faculty about the focus and purpose of the course; and (d) outreach to students with undeclared majors. The project will leverage existing resources, including simulation soft‐ ware developed with funding from a prior NSF award, an advisory panel of industry experts, and the college’s videography internship program.Bio‐Inspired Solutions to Human Challenges
Project Abstract In spite of growing demand for technicians among local manufacturers, Chameleon Community College is experiencing under‐enrollment in engineering technology and pre‐engineering programs. To address this problem, the college is developing a general education science course about bio‐inspired engineering and design that will attract students with undeclared majors to these and other STEM programs.WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 14
What are the project’s main INPUTS?
The main activities include (a) completion of the course curriculum; (b) creation of a short video about bio‐ inspired engineering; (c) presentations to admissions counselors, advisors, and faculty about the focus and purpose of the course; and (d) outreach to students with undeclared majors. The project will leverage existing resources, including simulation soft‐ ware developed with funding from a prior NSF award, an advisory panel of industry experts, and the college’s videography internship program.Bio‐Inspired Solutions to Human Challenges
Project Abstract In spite of growing demand for technicians among local manufacturers, Chameleon Community College is experiencing under‐enrollment in engineering technology and pre‐engineering programs. To address this problem, the college is developing a general education science course about bio‐inspired engineering and design that will attract students with undeclared majors to these and other STEM programs. OUTPUTS SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S LONG-TE RM OUTCOME S INPUTS ACTIVITIE S NSF Funding Industry advisory board Simulation software Videography internship programWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 15
What are the project’s main ACTIVITIE S?
The main activities include (a) completion of the course curriculum; (b) creation of a short video about bio‐ inspired engineering; (c) presentations to admissions counselors, advisors, and faculty about the focus and purpose of the course; and (d) outreach to students with undeclared majors. The project will leverage existing resources, including simulation soft‐ ware developed with funding from a prior NSF award, an advisory panel of industry experts, and the college’s videography internship program.Bio‐Inspired Solutions to Human Challenges
Project Abstract In spite of growing demand for technicians among local manufacturers, Chameleon Community College is experiencing under‐enrollment in engineering technology and pre‐engineering programs. To address this problem, the college is developing a general education science course about bio‐inspired engineering and design that will attract students with undeclared majors to these and other STEM programs. OUTPUTS SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S LONG-TE RM OUTCOME S ACTIVITIE S Develop course curriculum Produce video Give presentations to faculty, advisors, and admissions counselors Outreach to students with undeclared majors INPUTS NSF Funding Industry advisory board Simulation software Videography internship programWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 16
What are the project’s main OUTPUTS?
The main activities include (a) completion of the course curriculum; (b) creation of a short video about bio‐ inspired engineering; (c) presentations to admissions counselors, advisors, and faculty about the focus and purpose of the course; and (d) outreach to students with undeclared majors. The project will leverage existing resources, including simulation soft‐ ware developed with funding from a prior NSF award, an advisory panel of industry experts, and the college’s videography internship program.Bio‐Inspired Solutions to Human Challenges
Project Abstract In spite of growing demand for technicians among local manufacturers, Chameleon Community College is experiencing under‐enrollment in engineering technology and pre‐engineering programs. To address this problem, the college is developing a general education science course about bio‐inspired engineering and design that will attract students with undeclared majors to these and other STEM programs. OUTPUTS SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S LONG-TE RM OUTCOME S ACTIVITIE S Develop course curriculum Produce video Give presentations to faculty, advisors, and admissions counselors Outreach to students with undeclared majors INPUTS NSF Funding Industry advisory board Bio‐inspired engineering lab Videography internship program Course materials Promotional videoWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 17
What are the project’s intended SHORT-TE RM OUTCOME S?
The main activities include (a) completion of the course curriculum; (b) creation of a short video about bio‐ inspired engineering; (c) presentations to admissions counselors, advisors, and faculty about the focus and purpose of the course; and (d) outreach to students with undeclared majors. The project will leverage existing resources, including simulation soft‐ ware developed with funding from a prior NSF award, an advisory panel of industry experts, and the college’s videography internship program.Bio‐Inspired Solutions to Human Challenges
Project Abstract In spite of growing demand for technicians among local manufacturers, Chameleon Community College is experiencing under‐enrollment in engineering technology and pre‐engineering programs. To address this problem, the college is developing a general education science course about bio‐inspired engineering and design that will attract students with undeclared majors to these and other STEM programs. OUTPUTS SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S LONG-TE RM OUTCOME S ACTIVITIE S Develop course curriculum Produce video Outreach to students with undeclared majors INPUTS Course materials Promotional video Student interest in pursuing engineering increases Students enroll in course Give presentations to faculty, advisors, and admissions counselors NSF Funding Industry advisory board Simulation software Videography internship programWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 18
What are the project’s intended MID-TE RM OUTCOME S?
The main activities include (a) completion of the course curriculum; (b) creation of a short video about bio‐ inspired engineering; (c) presentations to admissions counselors, advisors, and faculty about the focus and purpose of the course; and (d) outreach to students with undeclared majors. The project will leverage existing resources, including simulation soft‐ ware developed with funding from a prior NSF award, an advisory panel of industry experts, and the college’s videography internship program.Bio‐Inspired Solutions to Human Challenges
Project Abstract In spite of growing demand for technicians among local manufacturers, Chameleon Community College is experiencing under‐enrollment in engineering technology and pre‐engineering programs. To address this problem, the college is developing a general education science course about bio‐inspired engineering and design that will attract students with undeclared majors to these and other STEM programs. OUTPUTS SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S LONG-TE RM OUTCOME S ACTIVITIE S Develop course curriculum Produce video Make presentations to faculty, advisors, and admissions counselors Outreach to students with undeclared majors INPUTS Course materials Promotional video Number of graduates from engineering programs increases Student interest in pursuing engineering increases Students enroll in course Enrollment in engineering programs increases NSF Funding Industry advisory board Simulation software Videography internship programWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 19
What are the project’s intended LONG-TE RM OUTCOME S?
The main activities include (a) completion of the course curriculum; (b) creation of a short video about bio‐ inspired engineering; (c) presentations to admissions counselors, advisors, and faculty about the focus and purpose of the course; and (d) outreach to students with undeclared majors. The project will leverage existing resources, including simulation soft‐ ware developed with funding from a prior NSF award, an advisory panel of industry experts, and the college’s videography internship program.Bio‐Inspired Solutions to Human Challenges
Project Abstract In spite of growing demand for technicians among local manufacturers, Chameleon Community College is experiencing under‐enrollment in engineering technology and pre‐engineering programs. To address this problem, the college is developing a general education science course about bio‐inspired engineering and design that will attract students with undeclared majors to these and other STEM programs. OUTPUTS SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S LONG-TE RM OUTCOME ACTIVITIE S Develop course curriculum Produce video Make presentations to faculty, advisors, and admissions counselors Outreach to students with undeclared majors INPUTS Course materials Promotional video Students enroll in course Student interest in pursuing STEM degree increases Regional demand for engineering technicians met Number of graduates from engineering programs increases Enrollment in engineering programs increases NSF Funding Industry advisory board Simulation software Videography internship programWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 20
WHAT DO LOGIC MODE LS LOOK LIKE IN THE RE AL WORLD
Felix Alba, Salt Lake Community CollegeWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 21
Seth Goodman, Phoenix College Michael Lesiecki, Maricopa Community CollegesWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 22
Anonymous ContributorLogic Model Template for ATE Projects and Centers Link on resource handout – download from
www.evalu‐ate.org/webinars/2016‐aug/WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 23 Lori Wingate
INTE GRATING A LOGIC MODE L INTO A FUNDING PROPOSAL
LOGIC MODE L USE S
Include in funding proposal Evaluation planning Project management Communication to stakeholders
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 24
WHE RE DOE S A LOGIC MODE L GO IN AN ATE PROPOSAL
Cover Sheet Project Summary Project Description References Cited Budget & Budget Justification Current & Pending Support Facilities, Equipment & Other Resources Supplementary Documents
Put here if it will not displace critical information within the 15 pages Otherwise, put hereNSF PROPOSAL COMPONE NTS
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 25
Supplementary Documents
CAUTION!
Except as specified below, special information and supplementary documentation must be included as part of the Project Description (or part of the budget justification), if it is relevant to determining the quality of the proposed work.
TRANSLATIONIf it is important that reviewers see it, do not put it in Supplementary Documents. Reviewers are not required to review supplementary documents.
NSF GRANT PROPOSAL GUIDE
WHAT IS THE RE LATIONSHIP BE TWE E N A LOGIC MODE L AND A PROJE CT DE SCRIPTION IN A PROPOSAL?
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 26 Project Description
Otherwise, put hereResults of Prior Support Rationale Activities, Deliverables, Objectives & Goals Timetable Management Plan Roles & Responsibilities of the PI, co‐PI(s), Other Senior Personnel Plan for Sustainability Evaluation Plan Dissemination Plan
NSF PROPOSAL COMPONE NTS
Y
to your narrative description of:
ACTIVITIE S Develop course curriculum Produce video Give presentations to faculty, advisors, and admissions counselors Outreach to students with undeclared majorsProject Description
Bio‐Inspired Solutions to Human Challenges Vestibulum laoreet ultrices velit, non consequat augue scelerisque ac. Nam sollicitudin ullamcorper felis, quis suscipit nisi egestas in. Vivamus aliquam nisl quis libero dignissimACTIVITIE S
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean pretium nunc lacus, molestie volutpat dui lacinia ut. Etiam ut congue sapien. Morbi at egestas sem. Curriculum Development Sed porta, mauris eget elementum suscipit, turpis est congue ligula, eget rutrum justo ante a dui. Integer lacus tellus, pellentesque nec ultricies id, congue vitae est. Mauris non interdum nibh. Nullam ut cursus risus vestibulum. Mauris id magna orci. Praesent tristique metus et lorem condimentum, ut cursus risus vestibulum. Video Production Suspendisse commodo tempus. Nam augue mi, tempus sed orci non, efficitur lobortisWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 27
ACTIVITIE S Develop course curriculum Produce video Give presentations to faculty, advisors, and admissions counselors Outreach to students with undeclared majorsProject Description
Bio‐Inspired Solutions to Human Challenges Vestibulum laoreet ultrices velit, non consequat augue scelerisque ac. Nam sollicitudin ullamcorper felis, quis suscipit nisi egestas in. Vivamus aliquam nisl quis libero dignissimACTIVITIE S
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean pretium nunc lacus, molestie volutpat dui lacinia ut. Etiam ut congue sapien. Morbi at egestas sem. Curriculum Development Sed porta, mauris eget elementum suscipit, turpis est congue ligula, eget rutrum justo ante a dui. Integer lacus tellus, pellentesque nec ultricies id, congue vitae est. Mauris non interdum nibh. Nullam ut cursus risus vestibulum. Mauris id magna orci. Praesent tristique metus et lorem condimentum, ut cursus risus vestibulum. Video Production Suspendisse commodo tempus. Nam augue mi, tempus sed orci non, efficitur lobortisProject Description
Bio‐Inspired Solutions to Human Challenges Vestibulum laoreet ultrices velit, non consequat augue scelerisque ac. Nam sollicitudin ullamcorper felis, quis suscipit nisi egestas in. Vivamus aliquam nisl quis libero dignissimOBJE CTIVE S
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean pretium nunc lacus, molestie volutpat dui lacinia ut. Etiam ut congue sapien. Morbi at egestas sem. E nrollment Sed porta, mauris eget elementum suscipit, turpis est congue ligula. Mauris non interdumWEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 28
Project Description
Nam augue mi, tempus sed orci non, efficitur lobortis augue. Sed ut quam ut urna consectetur volutpat. Suspendisse potenti. Donec sagittis mollis elit, in ultrices lectus scelerisque blandit. Aliquam eget lectus maximus, tempor nisi a, dapibus ipsum. Suspendisse vitae tellus consectetur, fringilla ipsum non, cursus neque.GOALS
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean pretium nunc lacus, molestie volutpat dui lacinia ut. Etiam ut congue sapien. Morbi at egestas sem. E nrollment Sed porta, mauris eget elementum suscipit, turpis est congue ligula. Mauris non interdumOR…
Project Description
Nam augue mi, tempus sed orci non, efficitur lobortis augue. Sed ut quam ut urna consectetur volutpat. Suspendisse potenti. Donec sagittis mollis elit, in ultrices lectus scelerisque blandit. Aliquam eget lectus maximus, tempor nisi a, dapibus ipsum. Suspendisse vitae tellus consectetur, fringilla ipsum non, cursus neque.OUTCOME S
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean pretium nunc lacus, molestie volutpat dui lacinia ut. Etiam ut congue sapien. Morbi at egestas sem. Short-Term Outcomes Sed porta, mauris eget elementum suscipit, turpis est congue ligula. Mauris non interdumOR…
SHORT-TE RM OUTCOME S MID-TE RM OUTCOME S LONG-TE RM OUTCOMECAUTION!
This presentation style may not be familiar to all NSF reviewers who may expect to see a list of goals and objectives.WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 29
Narrative project description should include all elements of logic model. All major activities, deliverables, goals,
should appear in logic model.
ONE SIMPLE RULE
Project Description
Otherwise, put hereResults of Prior Support Rationale Activities, Deliverables, Objectives & Goals Timetable Management Plan Roles & Responsibilities of the PI, co‐PI(s), Other Senior Personnel Plan for Sustainability Evaluation Plan Dissemination Plan
NSF PROPOSAL COMPONE NTS
Align evaluation questions and data collection plan to logic model elements
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 30 Project Description
Otherwise, put hereResults of Prior Support Rationale Activities, Deliverables, Objectives & Goals Timetable Management Plan Roles & Responsibilities of the PI, co‐PI(s), Other Senior Personnel Plan for Sustainability Evaluation Plan Dissemination Plan
NSF PROPOSAL COMPONE NTS
Should be reflected in ACTIVITIE S section of logic model
Lori Wingate
BONUS!
More info to help with your ATE proposals
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 31
5-PAGE CHE CKLIST ON ATE PROPOSAL E VALUATION PLANS
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 32 Evaluation: Don’t Submit Your ATE Proposal Without It August 2015
WE BINAR ON INTE GRATING E VALUATION INTO ATE PROPOSALS
Required and recommended elements of descriptions of prior NSF support
RE SUL TS FROM PRIOR NSF SUPPORT CHE CKLIST
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 33 Small Project Evaluation: Principles and Practices March 2016
WE BINAR ON SMALL PROJE CT E VALUATIONS
Including demonstrations ofTips for a strong NSF proposal evaluation plan
by Leslie Goodyear
WWW.E VALU-ATE .ORG/ BLOG
WEBINAR Logic Models: Getting Them Right and Using Them Well 8/17/2016 www.evalu‐ate.org 34
www.evalu‐ate.org
THANK YOU!