through Implementation Evaluation and Logic Model Initiative CSP - - PowerPoint PPT Presentation

Grantee Logic Models: Application through Implementation

Evaluation and Logic Model Initiative CSP Project Directors’ Meeting April 2018

2

This presentation was produced in whole with Federal funds from the U.S. Department of Education under contract with WestEd, number ED-OII-15-C-

• 0051. Patricia Kilby-Robb is the Contract Officer’s Representative (COR) for this

project, patricia.kilby-robb@ed.gov. The content of this presentation does not necessarily reflect the views or policies of the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.

3

Objectives

• 1. Introduce Logic Model Review

tool

• 2. Identify strategies to

strengthen use and implementation of logic models

4

Agenda

• Logic model review
• Introduce and discuss logic

model review tool

• Identify strategies to

strengthen logic models

• Group activity

5

Who is WestEd?

is a research, development, and service agency that works with education and other communities to promote excellence, achieve equity, and improve learning for children, youth, and adults. has monitored several U.S. Department of Education (ED) discretionary grant programs including the Charter Schools Program, Magnet Schools Assistance Program, and Voluntary Public School Choice.

6

Evaluation and Logic Model Initiative

• Convene semi-regular expert panel to review logic

models, identify criteria, develop rubric and review tool

• Validated review tools with panel and CSP staff
• Reviewed all grantee logic models against rubric
• Providing on-going resources, TA, and guidance

7

Expert Panel Members

Candice Bocola – Harvard/WestEd Thomas Brush – University of Indiana Ann Doucette – Claremont University/Evaluator’s Institute Melvin Mark – Penn State University Joanna Smith – University of Oregon Elena Polush – Iowa State University

8

Evaluation and Logic Model Initiative

New this year:

• Reviewed FY17 grantees
• Developed

implementation/usage and end of grant phases of tool

9

FY16 Notice Inviting Applicants

Application Requirements: (d) Logic model: Provide a complete logic model (as defined in this notice) for the project. The logic model must address the role of the grant in promoting the State-level strategy for expanding the number

• f high-quality charter schools through startup subgrants, optional

dissemination subgrants, optional revolving loan funds, and other strategies.

10

FY17 Notice Inviting Applicants

(g) Quality of the Management Plan and Theory of Action (up to 15 points): (1) The quality, including the cohesiveness and strength of reasoning, of the ‘‘logic model’’ (as defined in this notice), and the extent to which it addresses the role of the grant in promoting the State-level strategy for using charter schools to improve educational

• utcomes for students through CSP subgrants for planning, program

design, and initial implementation and other strategies;

11

Definition in NIAs

Logic model (also referred to as a theory of action) means a well- specified conceptual framework that identifies key components of the proposed process, product, strategy, or practice (i.e., the active ‘‘ingredients’’ that are hypothesized to be critical to achieving the relevant outcomes) and describes the relationships among the key components and outcomes, theoretically and operationally. (34 CFR 77.1)

12

What is in a logic model?

Resources

• Staff,

partners, funding,

• ther

Activities

• Project

work Outputs

• Products

Outcomes

• Short-,

mid-, long- term

Assumptions External Factors External Evaluation

14

Why use a logic model?

“…to help generate the clarity and specificity required for success and often demanded by funders and constituents.” “Logic modeling offers an organized way of defining your program goals, services, and measurable outcomes by producing:

• An inventory of what you have and what you need to operate your program
• A strong case for how and why your program will produce desired results
• A method for program management and assessment

15

What makes a good logic model?

• Clarity – Are the connections

among elements clear and logical? Is level of detail appropriate?

• Accuracy – Does the model

reflect the project? Is content complete?

• Logic – Does it make sense?

17

CSP Logic Model Expectations

• Stand alone document
• Adequately characterizes grant

project

• Used in on-going

implementation of grant project

18

Logic Model Review and Implementation Tool

Implementation Progress tool is designed to facilitate on-going usage of the logic model during life span of grant Initial Review tool is designed to systematically review the content of the logic models to ensure common features and relevant content

19

Roles and Responsibilities

Application Content Expert Panel Review Grantee Input CSP Review

• CSP provides application

content

• Expert Panel conducts the initial

review on application content

• Grantees makes revisions, as

needed

• Grantees’ annually provide

updates on progress

• CSP annually reviews grantee

progress, provides feedback

20

Logic Model Review Tool – Structural Components

• Are all components included (resources, activities,
• utputs, outcomes, assumptions, and external

factors)?

• Are there explicit relationships?
• Are outcomes defined within the grant time frame?

21

Logic Model Review Tool – Content Review

• Is there an alignment with the state strategy?
• Are the project objectives/ performance measures

included?

• Are projected outcomes feasible?
• Do outcomes demonstrate meaningful value/public

benefit?

22

Logic Model Review Tool – Rubric

• Alignment/relationship between grant goals, inputs,

activities, outputs, and outcomes

• Reasonable timeframe for activities and outcomes

within the period of the grant

• Connects grant project to state strategy
• Adequacy of resources

23

Logic Model Progress Tracker

• Designed to support use of logic models by grantees in

regular grant administration

• Intended to help grantees capture major project

implementation and changes

• Expected to be reviewed annually by Program Officers in

conjunction with APR reviews

24

Logic Model Progress Tracker

• What is current implementation status? Has there

been any changes? Any additional information?

• Resources
• Activities
• Outputs
• Outcomes
• External Factors

25

Getting from here to there

• Know where you start
• Review your current logic model
• Know what you can adjust (and what you can’t)
• These are a part of your approved application package
• Revisions to clarify and strengthen are preferred
• Major changes should be done in conjunction with Program

Officer

26

Tips for developing a stronger logic model

• Components in the right order
• Elements under the correct components
• Defined and detailed elements
• Inclusion of project objectives and/or performance measures

in outcomes

• Articulated relationships
• Defined assumptions, external factors

27

Tips for using logic models for project implementation

• Using a logic model should be part of an ongoing process of

discussion, commitment, and reflection as part of a grantee’s dedication to program improvement and accountability.

• The process serves as the starting point… and can later be

considered the “report card”… to understand whether or not you are meeting objectives.

• They should be revisited on regular basis and should closely

reflect the goals of your [grant project] and evaluation plan.

28

Group Activity

Split the room by grant year.

• FY15 and FY16 grantees
• FY17 grantees

Then…(see handout)

• Review initial logic model feedback and

identify ways to strengthen logic model OR

• Review current logic model and identify

implementation status and changes

FY15 FY16 FY17

29

Homework

FY15 and FY16:

• Summarize current

implementation status

• Identify changes (as needed)

FY17:

• Clarify timeframes
• Identify assumptions and

external factors

• Talk with Program Officers

30

Wrap-up/Discussion

• Questions?
• What’s next?

WestEd.org

Contact us!

Sara Allender – sallend@wested.org John Flaherty – jflaher@wested.org Sharon Herpin – sherpin@wested.org