Achieving outcomes in complex environments Toby Lowe Summary - - PowerPoint PPT Presentation

achieving
SMART_READER_LITE
LIVE PREVIEW

Achieving outcomes in complex environments Toby Lowe Summary - - PowerPoint PPT Presentation

Achieving outcomes in complex environments Toby Lowe Summary Outcomes-Based contracting/Performance Management makes it harder to produce good outcomes Systems produce outcomes take responsibility for the health of your


slide-1
SLIDE 1

Achieving

  • utcomes in

complex environments

Toby Lowe

slide-2
SLIDE 2

Summary

  • Outcomes-Based contracting/Performance

Management makes it harder to produce good outcomes

  • Systems produce outcomes – take

responsibility for the health of your systems

  • Embrace complexity – people’s lives are

complex

slide-3
SLIDE 3

Overview

  • What happens when people implement

Outcomes-Based Performance Management?

  • What can you do differently?
  • Working in complexity
slide-4
SLIDE 4

What happens when people implement OBPM? What’s the evidence?

slide-5
SLIDE 5

A quick summary of the evidence…. “The overall conclusion from international experience of implementing an outcomes approach is that the journey is long and the results are disappointing.”

Wimbush, Erica (2011): ‘Implementing an outcomes approach to public management and accountability in the UK—are we learning the lessons?’, Public Money & Management

slide-6
SLIDE 6

What happens? Gaming

OBPM turns everyone’s job into the production of data, not meeting client need.

slide-7
SLIDE 7

Targets for results “frequently distort the direction of programs, diverting attention away from, rather than towards, what the program should be doing.”

Burt Perrin, “Effective Use and Misuse of Performance Measurement”, American Journal of Evaluation, 1998

slide-8
SLIDE 8

“Ossification, a lack of innovation, tunnel vision and suboptimization”

S van Thiel and F. L. Leeuw “The Performance Paradox in the Public Sector”, Public Performance and Management Review, 2002

slide-9
SLIDE 9

The Work Programme: “the available evidence to date suggests that providers are engaging in creaming and parking, despite the differential payment regime.”

Work Programme evaluation: Findings from the first phase of qualitative research on programme delivery, DWP, 2012

slide-10
SLIDE 10

The breadth of evidence

  • Young People – Australia (Keevers et al 2012)
  • Healthcare – UK (Bevan and Hood 2006),
  • Employment programmes (Perrin 1998 – USA

and Canada, Soss et al 2011 USA, Newton 2012 – UK)

  • Public services - Europe and USA, (Van Thiel

and Leeuw 2002)

  • Education - USA (Rothstein 2008)
slide-11
SLIDE 11

Latest evidence

Systematic review of Outcomes-Based Contracts by Australian Government: Emma Tomkinson 2016

  • Outcomes-contracts delivered the contracted
  • utcomes
  • But contracted outcomes weren’t what

people wanted

  • Other outcomes got worse or stagnated
slide-12
SLIDE 12

Measurement Problem:

OBPM doesn’t measure genuine impact, it measures proxies

slide-13
SLIDE 13

Proxy measures

“If we had a thousand measures, we could still not fully capture the health and readiness of young children. We use data to approximate these conditions and to stand as proxies for them.”

Mark Friedman, Results Based Accountability Implementation Guide

#KittensAreEvil

slide-14
SLIDE 14

=

#LittleHeresies

Proxy Measure Experience

  • f outcome

= distortion effect of Proxy when used as a Performance Measure

slide-15
SLIDE 15

“Outcome” measurement

slide-16
SLIDE 16

The attribution problem:

Interventions don’t create outcomes

slide-17
SLIDE 17

Programme Logic Model

Robert Schalock & Gordon Bonham “Measuring outcomes and managing for results”, Evaluation and Program Planning, 2003

slide-18
SLIDE 18
slide-19
SLIDE 19

Effects of proxy measures + Lack of control = Gaming

slide-20
SLIDE 20

What else can we do? Embrace the complexity

  • f the work
slide-21
SLIDE 21
slide-22
SLIDE 22

Findings - Headline

Complexity-friendly commissioning is based

  • n a different attitude to:

Motivation Learning System health – quality of relationships

slide-23
SLIDE 23

Motivation

Motivation is intrinsic, not extrinsic.

slide-24
SLIDE 24

Learning

Learning drives improvement:

  • Positive error culture
  • Reflection on practice
  • Measurement
slide-25
SLIDE 25

System Health

Taking responsibility for the health

  • f the system as a whole:
  • Networks
  • Building positive relationships,

nurturing trust

slide-26
SLIDE 26

Principles

Context matters “X would be completely ineffective if was done in [another city], but they might benefit from this model of infrastructure.” (Charitable Funder) “Humility is needed – recognising we don’t know the answer as individual organisations” (Charitable Funder) Outcomes matter – as motivation and a focus for learning “We do not, we never want to get into the measuring impact thing, and the attribution thing. It’s all nonsense I think…. I think for us it’s about saying ‘Did it work, did it do what we thought it would? If not why not?... It’s about learning rather than measuring.” (Charitable Funder).

slide-27
SLIDE 27

Cultures

Developing trust “The two things that are most important that I would say are, the first is trust and the second is respect. Without those two things, you can’t do anything.” (Public Sector Commissioner) It’s not cosy – it’s challenging “He [funded organisation] might also be very critical of me face to face. So he can say very critical things…. That’s okay” (Public Sector Commissioner) Learning and listening “I think creating a culture that models generosity and listening leads to critical thinking and asking questions.” (Charitable Funder)

slide-28
SLIDE 28

Findings: practice

slide-29
SLIDE 29

Trust

Trust provides the confidence to let go of the illusion of control. I trust this organisation to do the right thing when the world changes. Question: what is a good set of reasons for a commissioners and providers to trust one another?

slide-30
SLIDE 30

Trust

Some potential answers:

  • Because they collaborate well
  • Because they know their role(s) within the

system(s) in which they operate

  • Because they share and reflect on their

practice with others

  • Because they use data intelligently to learn
slide-31
SLIDE 31

Complex systems produce outcomes How can you make your places work better, as systems?

slide-32
SLIDE 32

Examples

slide-33
SLIDE 33

Plymouth Council: adults with complex needs

  • Realised outcomes-based commissioning

wasn’t working

  • Appreciative enquiry – whole system

approach

  • Built trusting relationships & co-production
  • Single alliance contract to 26 organisations
  • New network infrastructure – e.g. wicked

problems forum

  • Saved over £500k
  • Award winning!
slide-34
SLIDE 34

Buurtzorg

Letting go of illusion of control Devolved power to the frontline =

  • Reduced home care costs
  • Increased client and worker

satisfaction

slide-35
SLIDE 35

I want to work like this…

  • Read the report:

https://blogs.ncl.ac.uk/tobylowe/

  • Check out the case studies
  • Join our Learning Group

13th November - Newcastle

  • Help us to experiment
slide-36
SLIDE 36

Thanks for listening

Toby Lowe E: toby.lowe@newcastle.ac.uk Twitter: @tobyjlowe