Key Questions Who will be best served by different types of data, - - PowerPoint PPT Presentation

Grantmaking 101: Developing

Assessment Measures that provide meaningful information for grant makers and nonprofit leaders

Paul Watson, MSHS William T. Oswald, Ph.D. Global Action Research Center

January 17, 2019

www.theglobalarc.org

Key Questions

 Who will be best served by different types of data, with discussions on the perspectives of nonprofits, funders, and community stakeholders?  How you go about developing assessment measures that accurately reflect the work of your grantees, while respecting your grantees’ time and resources?  How do you use different qualitative and quantitative data points to paint a more holistic picture of impact.  What are some different approaches to collecting the data you need? Pros and cons of different data types and collection methods.

Workshop Agenda

Some Basics Authentic Partnerships

Theory of Change Logic Model Appreciative Inquiry

Con Conne nect cting ing Outc Outcome

• mes

s to to Goa Goals ls

Operational Definitions We Do What We Measure Assessing Product v Process Assessing Connection between Goal and Deliverables

Basic Assumptions

• 1. Its all about the funder-grantee relationship
• 2. Numbers don’t lie people do
• 3. Numbers have no meaning except what we give them
• 4. We do what we measure
• 5. There is an uneasy alignment between our work and models of

assessment

• 6. Good data is everywhere

Why We Do Assessments

BECAUSE

we want to capture what is happening in ways that inform us about the connection across our

INVES INVESTM TMEN ENTS TS

(the funding)

OUTP OUTPUTS UTS

(what we do/produce)

OUT OUTCOME COMES

(how it affects those we touch)

IMPACTS

(how it changes conditions) There are many ways to capture what we want to know – there is no one method that fits all situations. How data is collected and who collects it impacts the quality of the data. Be very clear as to why you want the data and how you will use it before you design your assessment.

Authentic Partnerships Require Trust

Appreciative Inquiry & Becoming a Learning Organization

Appreciative Inquiry (AI) is a change management approach that focuses on identifying what is working well, analyzing why it is working well and then doing more of it. The basic tenet of AI is that an organization will grow in whichever direction that people in the organization focus their attention. A Learning Organization…is one where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning to see the whole together.

How to move beyond “Bean Counting” and move goals forward while getting a wholistic assessment of the impact of the foundation’s in inves estme tment nt

Building a Trusted Partnership

The five main characteristics that Peter Senge identified are said to innovate a learning organization. These are:

• 1. Systems thinking,
• 2. Personal mastery,
• 3. Mental models,
• 4. Building shared vision,
• 5. Team learning.

Requires:

• 1. Information is Shared and

Accessible

• 2. Learning is Emphasized and

Valued

• 3. Mistakes and Failures are not

Punished

• 4. People are Expected to Learn

Constantly

By building relationships that feed the conditions for creating Learning Organization

Collectively setting of benchmarks and jointly assessing progress towards those benchmarks on a regular basis

Logic Model

The LOGIC MODEL, because of its focus on “outcomes” provides a good road map for accomplishing goals and identifying the resources needed to achieve the goals.

Theory of Change

Long- Term Outcome Preconditions Preconditions Preconditions Preconditions Preconditions Preconditions

Intervention Intervention Intervention Intervention Intervention

Indicators Indicators Indicators Indicators Indicators Indicators Indicators A B

Connecting Outcomes to Goals

Numbers Don’t Lie – People Do

Numbers have no meaning except that which we give them

Operational Definitions

Concept Indicators

Measure Measure

Indicators

Measure Measure Wha hat i t is t s the he impact y impact you ar

• u are

e hoping to ac hoping to achie hieve? e? What will tell you that you’ve reac eached or ar hed or are mo e moving t ving towar ard d your des

• ur desir

ired impact? ed impact? Ho How w wi will y ll you ass

• u assess

ess the the exist xistence ence and st and strength of ength of the the Ind Indica icator tors s

Ope Operational tional Def Definition inition: To test an idea, one must gather data. This means relating your concept to concrete, measurable events in the world. To bridge that gap, scientists need

• perational definitions. These are definitions that specify how to measure or detect

something.

We Do What We Measure

Concept Indicators

Measure Measure

Indicators

Measure Measure

Civic Engagement

Interest in public affairs

Increase in the number

• f people voting

Increase in the number

• f people attending

specifically identified public meetings

Connected to the community

The number of people volunteering at the local schools Self-report on "feelings

• f connected" to

community

Deliverables

Shape the work

Informs

What is the relationship between the deliverables & the goal?

Appreciative Inquiry within a Learning Organization constantly assesses alignment and direction of movement ~AND~ Changes the nature of the relationship, laying the ground work for an authentic partnership

• Voter turn out will increase by 5%
• Attendance at Town Council meetings will increase by 10

attendees

• Pre-Post Survey of at least 100 residents shows increase

in feelings of connectedness to the neighborhood

Assessing Product v Project

Caution: Programs/Projects create products AND impact process

Products are easily measured – Process is not

Products are the number of voters, attendance at meeting, etc. Process relates to changes in how decisions are made and who makes them – a change in the

power dynamics of the community.

Notice that two of the three deliverables on the previous screen are “outputs” or “products” only one attempts to assess process – and even that does not assess actual change in community power. If it is true that

We do what we measure?

How do we shape our measures and deliverables in ways that are true to the project’s goal?

Exercise

Concept Indicators Measures Deliverables

Questions:

• 1. Assume all deliverables are met:
• How did/will the project achieve or move

toward its goal? (What is the Chain of Change?)

• If there is change – how sustainable is it?
• 2. How is the assessment benefiting:
• The project
• The Foundation
• The Targeted Population
• The people working on the project

We Do What We Measure

Concept Indicators

Measure Measure

Indicators

Measure Measure

Civic Engagement

Interest in public affairs

• f people voting

• f people attending

Connected to the community

• f connected" to

Deliverables

Informs

• Voter turn out will increase by 5%
• Attendance at Town Council meetings will increase by 10

• Pre-Post Survey of at least100 residents shows increase

Operational Definitions

Concept Indicators

Measure Measure

Indicators

Measure Measure Wha hat is is the he im impa pact ct y you

• u ar

are e hoping to ac hoping to achie hieve? e? What will tell you that you’ve reac eached or hed or ar are mo e moving to ving towar ard d your

• ur desir

desired impact ed impact? ? How w will y ill you

• u ass

assess ess t the he existence xistence and str and strengt ength h of

• f the

he Ind Indica icator

• rs

s

Operati tional Defi finiti tion: To test an idea, one must gather data. This means relating your concept to concrete, measurable events in the world. To bridge that gap, scientists need

• perational definitions. These are definitions that specify how to measure or detect

something.

Assessing Our Assessments

Closure

Name one thing you will take with you – that you will think about as you go back to work