Grantmaking 101: Developing Assessment Measures that provide meaningful information for grant makers and nonprofit leaders Paul Watson, MSHS William T. Oswald, Ph.D. Global Action Research Center January 17, 2019 www.theglobalarc.org
Key Questions Who will be best served by different types of data, with discussions on the perspectives of nonprofits, funders, and community stakeholders? How you go about developing assessment measures that accurately reflect the work of your grantees, while respecting your grantees’ time and resources? How do you use different qualitative and quantitative data points to paint a more holistic picture of impact. What are some different approaches to collecting the data you need? Pros and cons of different data types and collection methods.
Workshop Agenda Some Basics Authentic Partnerships Theory of Change Logic Model Appreciative Inquiry Con Conne nect cting ing Outc Outcome omes s to to Goa Goals ls Operational Definitions We Do What We Measure Assessing Product v Process Assessing Connection between Goal and Deliverables
Basic Assumptions 1. Its all about the funder-grantee relationship 2. Numbers don’t lie people do 3. Numbers have no meaning except what we give them 4. We do what we measure 5. There is an uneasy alignment between our work and models of assessment 6. Good data is everywhere
Why We Do Assessments BECAUSE we want to capture what is happening in ways that inform us about the connection across our INVES INVESTM TMEN ENTS TS (the funding) OUTPUTS OUTP UTS (what we do/produce) OUT OUTCOME COMES (how it affects those we touch) IMPACTS (how it changes conditions) There are many ways to capture what we want to know – there is no one method that fits all situations. How data is collected and who collects it impacts the quality of the data. Be very clear as to why you want the data and how you will use it before you design your assessment.
Authentic Partnerships Require Trust
Appreciative Inquiry & Becoming a Learning Organization How to move beyond “Bean Counting” and move goals forward while getting a wholistic assessment of the impact of the foundation’s inves in estme tment nt Appreciative Inquiry (AI) is a change management approach that focuses on identifying what is working well, analyzing why it is working well and then doing more of it. The basic tenet of AI is that an organization will grow in whichever direction that people in the organization focus their attention. A Learning Organization … is one where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning to see the whole together.
Building a Trusted Partnership By building relationships that feed the conditions for creating Learning Organization The five main characteristics that Requires: Peter Senge identified are said to 1. Information is Shared and innovate a learning organization. Accessible These are: 2. Learning is Emphasized and Valued 1. Systems thinking, 3. Mistakes and Failures are not 2. Personal mastery, Punished 3. Mental models, 4. People are Expected to Learn 4. Building shared vision, Constantly 5. Team learning. Collectively setting of benchmarks and jointly assessing progress towards those benchmarks on a regular basis
Logic Model The L OGIC M ODEL , because of its focus on “outcomes” provides a good road map for accomplishing goals and identifying the resources needed to achieve the goals.
Theory of Change A Indicators Long- Term Outcome Indicators Indicators Preconditions Preconditions Preconditions Intervention Intervention B Indicators Preconditions Preconditions Preconditions Indicators Indicators Indicators Intervention Intervention Intervention
Connecting Outcomes to Goals
Numbers Don’t Lie – People Do Numbers have no meaning except that which we give them
Operational Definitions Wha hat i t is t s the he impact y impact you ar ou are e Concept hoping to ac hoping to achie hieve? e? What will tell you that you’ve reac eached or ar hed or are mo e moving t ving towar ard d Indicators Indicators your des our desir ired impact? ed impact? Ho How w wi will y ll you ass ou assess ess the the exist xistence ence and st and strength of ength of the the Measure Measure Measure Measure Ind Indica icator tors s Ope Operational tional Def Definition inition: To test an idea, one must gather data. This means relating your concept to concrete, measurable events in the world. To bridge that gap , scientists need operational definitions . These are definitions that specify how to measure or detect something .
We Do What We Measure Concept Civic Engagement Connected to the Indicators Indicators Interest in public affairs community Increase in the number Measure Measure Measure Measure Self-report on "feelings The number of people Increase in the number of people attending volunteering at the of connected" to of people voting specifically identified community local schools public meetings What is the relationship between Shape Informs the deliverables & the the Deliverables goal? work • Voter turn out will increase by 5% • Attendance at Town Council meetings will increase by 10 attendees • Pre-Post Survey of at least 100 residents shows increase in feelings of connectedness to the neighborhood Appreciative Inquiry within a Learning Organization constantly assesses alignment and direction of movement ~AND~ Changes the nature of the relationship, laying the ground work for an authentic partnership
Assessing Product v Project Caution: Programs/Projects create products AND impact process Products are easily measured – Process is not Products are the number of voters, attendance at meeting, etc. Process relates to changes in how decisions are made and who makes them – a change in the power dynamics of the community. Notice that two of the three deliverables on the previous screen are “outputs” or “products” only one attempts to assess process – and even that does not assess actual change in community power. If it is true that We do what we measure? How do we shape our measures and deliverables in ways that are true to the project’s goal?
Exercise Concept Indicators Measures Deliverables
Assessing Our Assessments Questions: Operational Definitions Wha hat is is the he im impa pact ct y you ou ar are e Concept hoping to ac hoping to achie hieve? e? 1. Assume all deliverables are met: What will tell you that you’ve reac eached or hed or ar are mo e moving to ving towar ard d Indicators Indicators your our desir desired impact ed impact? ? • How did/will the project achieve or move How w will y ill you ou ass assess ess t the he existence xistence and str and strengt ength h of of the he Measure Measure Measure Measure toward its goal? (What is the Chain of Indica Ind icator ors s Operati tional Defi finiti tion: To test an idea, one must gather data. This means relating your Change?) concept to concrete, measurable events in the world. To bridge that gap , scientists need operational definitions . These are definitions that specify how to measure or detect something . • If there is change – how sustainable is it? We Do What We Measure 2. How is the assessment benefiting: Concept Civic Engagement • The project Connected to the Indicators Indicators Interest in public affairs community Increase in the number Measure Measure Measure Measure Self-report on "feelings The number of people • The Foundation Increase in the number of people attending volunteering at the of connected" to of people voting specifically identified local schools community public meetings What is the relationship between Shape Informs • The Targeted Population the deliverables & the the Deliverables goal? work • Voter turn out will increase by 5% • Attendance at Town Council meetings will increase by 10 • The people working on the project attendees • Pre-Post Survey of at least100 residents shows increase in feelings of connectedness to the neighborhood
Closure Name one thing you will take with you – that you will think about as you go back to work
Recommend
More recommend
