Evaluation An overview of tools that can be used to improve - - PowerPoint PPT Presentation

Lean Thinking and Monitoring and Evaluation

An overview of tools that can be used to improve programme efficiency, document impact and improve programme effectiveness.

Objectives

2/15/2017 Disclaimer: 3

Key message

• Lean methodology tools are interlinked and they will help improve

the efficiency and effectiveness of your organisation.

• Use some of the tools as and when required (i.e. you do not have to

use them all at once) Areas that will be covered

• What is lean thinking?
• How has it been used by Tshikululu Social Investments NPC?
• What are some of the tools associated with this approach?
• When do you use it? Who is it applicable to? Why?
• Note: I will be referring to customers later in this presentation-

customers are beneficiaries or programme participants in the NPO

• context. In a school context they could be referred to as learners.

What is lean thinking?

• Business philosophy that encourages efficiency, waste

reduction and quality

• Linked with (social) entrepreneurship-bootstrapping approach

for coming up with MVPs and scalable enterprises

• Resemblance software development cycles
• Lean data movement pioneered by Acumen+
• Data collection: Focuses on rapid reporting-collect only what is

necessary

• Can be used for Programme Design, Programme Monitoring

and can aid the Evaluation Process.

2/15/2017 Disclaimer: 4

Comparable cycles

2/15/2017 5

Results Based Monitoring Cycle Deming Cycle (PDCA Cycle-Lean Cycle) Lean product cycle Faster learning feedback loops Better uptake of learning to improve programme delivery

Lean Thinking to Facilitate Learning and Fewer Indicators

• Food Sovereignty Programme
• Situation: New strategy implemented and

reviewed data collection efforts

• Problem: Programme Partners were uncertain

about baseline data collection efforts and what it meant in relation to the funding the strategy

• Action: Workshop to clarify strategy and M&E
• Result: Networking, shared learning,

agreement on M&E Plan

2/15/2017 Disclaimer: 6

2/15/2017 Disclaimer: 7

Lean Thinking to plan better

• Reviewed a Socio-Economic Development

Strategy to find opportunities to improve its alignment with community needs

• Situation: Felt that the work that was being done

was good – but not entirely reflective of intent

• Problem: Short-time horizon to change strategy

and implementation.

• Action: Use fishbone analysis and A3 report.

Understand beneficiary journey.

• Result: Better aligned strategy, philosophical fit

with intent, more engaged client, partners and beneficiaries.

Lean Thinking for new product design and rapid reporting

• Strategy development process and gather

insights

• Situation: Required to implement a rapid social

investment approach

• Problem: Short-time horizon to report and

assist in BEE Scorecard

• Action: Develop a strategy and reporting

format that meets time and client requirements (used existing implementation data and interviewed participants-focused questions, small sample)

• Result: Fast turnaround on strategy

implementation, met Clients reporting needs

2/15/2017 Disclaimer: 8

Making sense of Lean tools

• Value Proposition Canvas: understanding the job a programme

participant needs to complete and the barriers and rewards associated with completing the job

• Customer Journey Canvas: Understanding the motivations and

behaviours associated with completing a job.

• Business Model Canvas: a tool used to understand how

service/product offerings can be operationalised to become a viable business model.

• A3 Reporting: One page report that provides a summary of a

problem, how it was solved and the implementation plan to track progress

• Ishikawa Diagram: A mind map that allows for identifying the

potential causes of a problem.

2/15/2017 Disclaimer: 9

• Provides a simplified view of a customers needs.
• Useful for Programme Designers.
• Provides some direction into understanding the needs of customers and how they are

best served.

Emotional Rational Trigger Trigger Trigger Conversion Motivation Information Inspiration 1 2 3 4

• Provides insight into the emotions, rational thinking and motivation of customers.
• Useful for Programme Implementation Staff and Senior Leadership team.
• Useful when designing a programme, or collecting data to better understand the fit between

programme theory and programme delivery.

Programme Partner Story Boards – Adapted version

• f the Value

Proposition Canvas

Storyboarding to understand value proposition

13

Tools to develop a deeper understanding of the customer experience

Personification Mood Board Most Significant Change (MSC)

Who will you work with? What will you do? What is your product/ service? How do you interact with your customer? Who is your customer? What do you need to deliver your product/ service? How will you communicate and distribute your product/ service? What are the costs associated with delivering your product? How will you make money

• ff your product/service?
• Provides and overall picture of how a business operates
• Useful for all stakeholders involved with programme delivery.
• Mainly those in a Marketing, Fundraising, Operations, Finance and Senior Leadership role.
• Useful when developing/
• reviewing strategy

Needs Assessment Stakeholder Map Programme Activities Theory of Change Programme Processes Programme Plan Communi- cation/ Engagement Plan Programme Budget Programme Sustainability

• r Revenue Model

Stakeholder Map

Logic Model

Business Model Canvas also integrates various aspects of Monitoring and Evaluation that may be challenging to explain for stakeholders involved in implementing a programme and for those Evaluating a programme.

Needs Assessment

Logic model

2/15/2017 Disclaimer: 17

A3 Chart and Ishikawa Diagram

• Provides a framework for analysing and solving problems.
• Its an effective communication tool.
• Useful for all levels of staff within any organisation.
• Primarily used in process improvement within an organisation. Could also be used for strategy

development.

Objectives

2/15/2017 Disclaimer: 18

Areas that were covered

• What is lean thinking?
• Business philosophy focused on efficiency and continuous improvement.
• How has it been used by Tshikululu Social Investments NPC?
• What are some of the tools associated with this approach?
• Business Model, Value Proposition, Customer Journey Canvasses. A3

Reporting and Ishikawa Diagram.

• When do you use it? Who is it applicable to? Why?

Key message

• The tools are useful for making sense of how interdependent parts of
• rganisational operations interlink.
• They are useful in understanding that the needs of the customer

(beneficiary/participant).

• It can improve the efficiency and effectiveness of your organisation.

Useful Reading Resources and Training

Useful Reading Resources Search for Lean Data Case Studies or Lean Data Collection Root Capital: https://www.rootcapital.org/ Acumen+ http://acumen.org/ Stanford Social Innovation Review: https://ssir.org/topics/category/measurement_evaluation Useful Resources (Free Training)

• Lean Principles for Social Impact (Online and Free) https://novoed.com/lean-

social-change

• Lean Data Approaches to Measure Social Impact (Online and Free)

https://novoed.com/lean-data

• Strategyzer

2/15/2017 Disclaimer: 19

References

• Baxter, R. (2014). Generating value through A3 thinking. Be an A3 thinker. Retrieved from:

https://vgpblog.wordpress.com/2014/12/27/generating-value-through-a3-thinking/

• Dichter, S., Adams, T. & Ebrahim, A. (2016). The power of lean data. Stanford Social Innovation Review, Winter 2016.
• Liker, J.K. & Meier, D. (2005). The Toyota Way Field Book: A practical guide for Implementing Toyota’s 4 Ps. New York,

NY: McGraw-Hill.

• Miller, J., Bogotova, T. & Carnohan, B. (2011). Improving Performance in Service Organizations: How to Implement a

Lean Transformation. Chicago, IL: Lyceum Books.

• Rigani (2016). How to enhance your customer’s experience. Retrieved from: http://rigani.com.au/how-to-enhance-

your-customers-experience/

• Singal, K.R (2014). Quality concepts and ISO 9001:2008 QMS awareness: an effort to create awareness. Retrieved

from: http://iso9001-2008awareness.blogspot.nl/2014/04/pdca-cycle.html

• Unger, R. & Chandler, C. (2012). Project objectives and approach for UX design: Know which stars to navigate by.

Retrieved from: http://www.peachpit.com/articles/article.aspx?p=1856033&seqNum=2

• Wittine, D. (2014). RTB implementing for Results-Based Management. Retrieved from:

http://www.rtb.cgiar.org/blog/2014/01/16/rtb-implementing-pilot-for-results-based-management/

2/15/2017 Disclaimer: 20