WELCOME to the webinar “ How to evaluate interventions in complex dynamic environments? ” 28 February 2012 This Live Webinar will start at 11:30 AM, New York time . All microphones & webcams are disabled and we will only enable microphones during the Q&A portion. Therefore, you will not hear any sound/noise till the beginning of the webinar.
Series of 17 live webinars on “Equity - focused Evaluations” Interact live with 28 world-level evaluators This series of webinars addresses the challenges and opportunities in evaluating the effects of policies, programmes and projects to enhance equitable development results, with a special focus on the effects to the most excluded, marginalized and deprived groups.
“Evaluation for Equitable Development Results” Available in MyM&E will be available early 2012
The book will be available at MyM&E Virtual Library www.mymande.org
Webinars on Equity-focused Evaluation 2011 Colin KIRK Evaluation to accelerate progress towards 6 September 2011 equitable development 9:30 AM NY time Penny HAWKINS Belen SANZ Human rights and Gender equality in evaluations 21 September 2011 Flaminia MINELLI 9:30 AM NY time Marco SEGONE How to design, implement and use equity- 4 October 2011 oriented evaluations 11:30 AM NY time Michael BAMBERGER Saville KUSHNER Case study evaluation as an intervention for 11 October 2011 promoting equity 9:30 AM NY time Bob WILLIAMS Systems approach (CSH) to address ethical 14 November 2011 issues 3:00 PM NY time Martin REYNOLDS Patricia ROGERS Program theories and LogFrames to evaluate 22 November 2011 pro-poor and equity programs 4:00 PM NY time Richard HUMMELBRUNNER Michael Quinn PATTON Developmental Evaluation 6 December 2011 11:30 AM NY time
Webinars on Equity-focused Evaluation 2012 Methodological guidance in evaluation for Social Donna MERTENS 24 January 2012 Justice 9:30 AM NY time Values-Engaged Evaluation Jennifer GREENE 15 February 2012 1:00 PM NY time How to evaluate interventions in complex Michael Quinn PATTON 28 February 2012 dynamic environments? 11:30 AM NY time Cultural Responsiveness in Applied Research Rodney HOPSON 15 March 2012 and Evaluation Settings 2:00 PM NY time Katrina BLEDSOE Evaluation of the ILO’s strategy to eliminate April 2012 Francisco GUZMAN discrimination in employment and occupation Juha UITTO Evaluating equity-focused public policies. The April 2012 case of Brazil and Mexico Oscar GARCIA Katherine HAY A healthy discomfort? Development, equity and May 2012 evaluation Sanjeev SRIDHARAN Ten Questions that Evaluations of Health Equity Initiatives should answer Julian BARR Evaluation of climate change interventions for June 2012 excluded populations Ken CHOMITZ
The recording will be available at www.mymande.org 1 2
The recording will be available at www.mymande.org
Interact with Questions and Answers Type here
Keynote Speaker Michael Quinn PATTON, Founder and Director, Utilization-Focused Evaluation, and former President of the American Evaluation Association
Agenda 11:30 – 11:35 Welcome and introduction Marco Segone, Systemic Management, UNICEF Evaluation Office 11:35 – 11:55 Michael Quinn PATTON, Founder and Director, Utilization- Focused Evaluation, and former President of the American Evaluation Association 11:55 – 12:25 Questions and Answers Moderator: Stewart Donaldson, Dean & Chair of Psychology School of Behavioral & Organizational Sciences, Claremont Graduate University 12:25 – 12:30 Wrap-up: Penny Hawkins, Evaluation Office, The Rockefeller Foundation
How to evaluate interventions in complex dynamic environments: Developmental Evaluation Michael Quinn Patton
Challenge: Matching the evaluation process and design to the nature of the situation 13
Seeing Through A Complexity Lens 14
Complex Nonlinear Dynamics • Nonlinear: Small actions can have large reactions. “ The Butterfly Wings Metaphor” • Emergent: Self-organizing, Attractors • Dynamic: Interactions within, between, and among subsystems and parts within systems can volatile, changing • Getting to Maybe: Uncertainty, unpredictable, uncontrollable 15
Getting to Maybe: How the World Is Changed? Frances Westley, Brenda Zimmerman, Michael Q. Patton Random House Canada,2006 16
Chapter 4 Situation Recognition and Responsiveness: Distinguishing Simple, Complicated and Complex 17
Conceptual Options • Simple • Complicated • Complex 18
Situation Analysis Matrix: Mapping the Territory Close to Degree of Far from Close to Certainty 19
Simple Space Simple Close to Plan, control Certainty Far from Close to 20
Simple Complicated Complex Following a Recipe A Rocket to the Moon Raising a Child z The recipe is essential z Recipes are tested to assure replicability of later efforts z No particular expertise; knowing how to cook increases success z Recipes produce standard products z Certainty of same results every time 21
Technically Complicated Close to Technically Complicated Simple Plan, control Experiment, coordinate expertise Certainty Far from Close to 22
Simple Complicated Complex Following a Recipe A Rocket to the Moon Raising a Child • Formulae are critical and necessary z The recipe is essential • Sending one rocket z Recipes are tested to assure replicability of increases assurance later efforts that next will be ok z No particular • High level of expertise; knowing how expertise in many to cook increases success specialized fields + coordination z Recipes produce standard products • Rockets similar in z Certainty of same critical ways results every time • High degree of certainty of outcome 23
Socially Complicated Socially Complicated Build relationships, create common ground Close to Simple Technically Complicated Plan, control Experiment, coordinate expertise Certainty Far from Close to 24
Socially complicated Implementing human rights agreements, like gender equity or outlawing child labor Environmental Initiatives Many different and competing stakeholders Diverse vested interests High stakes 25
Socially complicated situations pose the challenge of coordinating and integrating many players 26
Know When Your Challenges Are In the Zone of Complexity Socially Complicated Zone of Build relationships, Complexity create common ground Close to Technically Complicated Simple Experiment, coordinate expertise Plan, control Certainty Far from Close to 27
Simple Complicated Complex Following a Recipe A Rocket to the Moon Raising a Child • Formulae have only z The recipe is essential Sending one rocket a limited application increases assurance z Recipes are tested to that next will be ok • Raising one child assure replicability of later efforts gives no assurance of success with the next z No particular High level of expertise in expertise; knowing how • Expertise can help many specialized to cook increases but is not sufficient; success fields + coordination relationships are key z Recipes produce Rockets similar in critical standard products • Every child is ways unique z Certainty of same High degree of certainty results every time of outcome • Uncertainty of outcome remains 28
Complicated Complex Simple Following a Recipe A Rocket to the Moon Raising a Child • Formulae have only a • Formulae are critical limited application z The recipe is essential and necessary • Raising one child z Recipes are tested to • Sending one rocket gives no assurance of assure replicability of increases assurance success with the next later efforts that next will be ok • Expertise can help z No particular • High level of but is not sufficient; expertise; knowing how expertise in many relationships are key to cook increases specialized fields + success coordination • Can’t separate parts from the whole z Recipe notes the • Separate into parts quantity and nature of and then coordinate • Every child is unique “parts” needed • Rockets similar in • Uncertainty of z Recipes produce critical ways outcome remains standard products • High degree of certainty of outcome z Certainty of same results every time 29
“A Leader's Framework for Decision Making” by David J. Snowden and Mary E. Boone, Harvard Business Review, November, 2007: Wise executives tailor their approach to fit the complexity of the circumstances they face. 30
Wise evaluators tailor their approach to fit the complexity of the circumstances they face. 31
Prospective Example Leadership Program Example Simple elements Complicated elements Complex elements 32
Simple outcomes • Increase knowledge and skills of participants Evaluation: Pre-post data and documentation of learning 33
Complicated Impacts • Change participants’ organizations Evaluation: Case studies of organizational change 34
Complex Vision • Infuse energy into the moribund not-for-profit (voluntary) sector • Make the sector more dynamic • Create network of leaders who actively engage in change 35
Recommend
More recommend
