RICKIE Launch A Monitoring, Evaluation and Research Framework for - - PowerPoint PPT Presentation

A Monitoring, Evaluation and Research Framework for Family Violence Regional Integration Committees

RICKIE Launch

20 July 2020 Framework available at: https://www.thelookout.org.au/family-violence- workers/regional-integration

Stren engthen ening Family V Violen ence R e Regional Int nteg egration – A Mo Monitori ring, R , Research rch a and Evaluation Framew ework f for F Family V Violen ence Regional I Integ egration C Committees ees

10.30

Welcome and acknowledgements Alison Macdonald, DV Vic Emma Morgan, DV Vic

10.40

Project Introduction Project Reference Group Members FVRIC PSA Alison Maclean & FVRIC Auspice Robyn Trainor

10.55

Framework Presentation and Development Process Project Research Team The University of Melbourne Cathy Humphreys, David Rose and Gemma McKibbin

11.20

Moderated Q&A Alison Macdonald, DV Vic

11.30

Wrap Up and Next Steps Project Reference Group Member FVRIC PSA John Duck

Acknowledgement of Country

3

Acknowledgements

The F Fam amily V ly Viol

• lence P

Philan anthrop

• py C

Collab labor

• rat

atio ion P Proj

• ject

The F Famil ily Vio Viole lence S Sec ector

• r C

Capacity B Buil ilding Pr Program

This project is part of The Family Violence Sector Capacity Building Program, generously funded by Gandel Philanthropy, The William Buckland Foundation, Give Where You Live Foundation, State Trustees Australia Foundation, The Victorian Women’s Benevolent Trust and the Johnstone Gumption Fund and the Jump Start Fund, sub-funds of Australian Communities Foundation

RICKIE Project introduction

Project Reference Group Members FVRIC PSA Alison Maclean & FVRIC Auspice Robyn Trainor

What are Family Violence Regional Integration Committees? (AM)

Established in 2006 Local governance structures that exist to improve integration of the family violence system and responses to victim survivors and perpetrators Broad membership, reach and influence Priority areas include:

• Governance and system leadership
• Integration of Reform Initiatives
• Workforce development
• Service data collection and local area profiling

Some of our current work (AM)

Regional integration and vertical integration Supporting the Family Violence reform agenda

• MARAM – mapping agency responsibilities and

training

• Information Sharing, including the production
• f supporting videos and workbooks
• Supporting changes associated with the

Gender Equity legislation

• Exploring the use of the Community Services

Quality Governance Framework to support regional integration

• Research
• Student Placements

Data (AM)

Are we improving the lived experience for people who experience family violence?

The RICKIE Project (RT)

Strategic planning remains an incredibly important mechanism for supporting the core functions of regional integration committees, with four key goals identified in the RICKIE Project proposal:

• Determining FVRIC research and evaluation needs
• Reviewing data set access and identify any gaps in data
• Undertaking regionally disaggregated data analysis
• Developing a Family Violence Regional Integration Committee Research and

Evaluation Framework

FV Integration (RT)

Family Violence Regional Integration and System Leadership Committees were developed in 2006, to maintain a strong focus on regional service and system coordination and provide a structure for service and system integration. Family violence integration is about coordinating responses to family violence through the implementation of policy and practice, to ensure that community and service responses are tilted towards the safety of women and children and the accountability of perpetrators. It is also about working together with a shared understandings, common frameworks and building family violence workforce capability and capacity.

There is plenty of ‘data’ in the system: funded agencies, non-funded agencies, Victoria Police, and the Court system all gather, collect and to varying degrees communicate data and information about family violence. However there is no common system of data collection (RCFV, Chapter 39, reference to Recommendations 203, 204, 205 and 206)

WORKFORCE DEVELOPMENT DATA SYSTEM REFORMS SYSTEM ADVOCACY BUILD SERVICE AND SYSTEM INTEGRATION COLLABORATION AND PARTNERSHIP

Capability and capacity

Build data monitoring and evaluation capability and capacity within SFVIAC and RIC’s Implement a common data dashboard and structure for monitoring and evaluation

Monitoring

Map and monitor local and regional data trends and themes Develop partnerships and data to build a regional foundation for social planning and decision making

Evaluation

Build regional capacity for data access, sharing and analysis Tailor data access and capture to evaluate and represent regional needs and contribute to reforms

Service and systems

Develop data supported and informed decision making and monitoring processes regionally and state-wide Build capacity and structures to contribute to an evidence base and system planning

Research

Establish strong ethical partnerships to guide evaluation and research to contribute to the evidence base Provide

• pportunities to

include and represent lived experience to contribute to systems change

Framework presentation and development process

Project Research Team The University of Melbourne Cathy Humphreys, David Rose and Gemma McKibbin

MREF – Process and Outcome (CH)

Introduction to ‘the team’ Capacity Building + MREF

• utput

Interviews and workshops (Gemma McKibbin) MREF (David Rose) Next steps

Introductions – Our team (CH)

Dr Gemma McKibbin Prof Cathy Humphreys Prof Stuart Ross A/Prof David Rose Dr Kristin Diemer David Gallant Dr Arno Parolini Wei Tan

14

Building on previous work (CH)

Healey, L. and Humphreys, C. and Wilox, K. (2013) Regional Governance Continuum Matrix of Practice for Domestic and Family Violence-Sexual Assault Partnerships: Background to the Practice Tool, Topic Paper 21, Australian Domestic Violence Clearinghouse, UNSW. https://www.nifvs.org.au/wp- content/uploads/2016/03/Governance-and-interagency-responses-Improving- practice-for-regional-governance-%E2%80%93-a-Continuum-Matrix-2013.pdf The FVRICs are well placed to “support and report on implementation of data system reforms, data collection and analysis” RCFV Vol 6 p124 Kwok (2018). Family Violence Regional Integration Project: Strengthening the case for the future of regional integration

The RICKIE Project (CH)

Two Primary Aims: 1) Capacity building and co- design of the Monitoring, Evaluation Research Framework (MERF) 2) Delivering the MERF No magic bullet ‘here is all your data analysed’

Developing the Framework (GM)

• Action research
• Semi-structured

interviews

• Three workshops – the

data parties!

17

Interviews (GM)

• 12 Semi-structured interviews
• Some group, some individual
• 26 participants - including PSAs, RIC Chairs and other key stakeholders
• Interview schedule covered:
• What monitoring and evaluation questions do you have for your RIC? What do you want to know?

Why do you need to answer those questions?

• In an ideal world, what monitoring and evaluation data would your RICs have access to? And in what

form? Would all RICs have access to this data? (as raw data, processed data or KPI reports)

• What are the primary sources of monitoring and evaluation data that your RIC currently has access to?

And how does this come to the RIC? Do you get this regularly or irregularly? Is the data you receive easy or difficult to interpret?

• What data does your RIC need access to but cannot get? Why can’t you get it? What’s the impact of

not having these data?

• How might your RIC use and data collected to improve responses to family violence across the service

system?

• SWOT analysis of the qualitative data

18

Interview findings (GM)

Strengths

• Getting access to

(limited) data sets

• Strengthening

governance arrangements and strategic priorities

• Trying out new ways
• f using data like the

Data Press in Central Highlands

• Developing this

shared MERF

Weaknesses/challenges

• Access to

comprehensive and relevant data sets

• Current boundaries

for DHHS and DSS reporting are not fully aligned with FVRICs

• The reported data

from government

• rganisations is not

always accessible to FVRICs

• Skills and knowledge

in understanding, analysing and applying data

Opportunities

• Using data to:
• understand gaps,

patterns and pathways in the region

• inform system’s

advocacy and local area planning

• evaluate the

effectiveness of linkages between services in the region

• answer research

questions at a local level

• Collaborating with

FSV re gaining access to Orange Door data

Threats

• Challenges related to

implementation of the (welcome) reform agenda

• Impact of funding

arrangements and competitive tendering

• Difficulty of

maintaining regional, rural and remote lenses

• Lack of funding

support for data projects

19

Sophisticated data work (GM)

A strength: The portal - in the data press there's the collection of the data from the services and that's been one element of the project. The portal actually takes all of that data and makes it mappable against publicly available datasets from [Australian Bureau of Statistics] and [Crime Statistics Agency] so that if I want to look at a particular LGA, I want to look at referrals from [region] services and I want to map it against AOD rates (PSA)

20

Essence of what was wanted (GM)

An opportunity: “I think it would be something really meaningful if we could somehow track and capture how people enter and leave and enter and leave the system, and how they come into - where they're referred from, how many times they've been into the system and what's making this experience different.” (PSA)

21

Workshops/Data parties highlights (GM)

Workshop one

• Overview of the RICKIE project and established a

consensus about the activities, outputs and outcomes of the FVRICs

• Beginnings of a program logic
• Acknowledgement of different sets of data literacy

Data party one

• Feedback from interviews
• Data ecology wheel
• Jess Cadwallader presented the Data Press

Data party two

• Masterclass on the use of Tableau Public
• Discussion of collaborations with Crime Statistics Agency,

DV Vic, University of Melbourne to move project forward

• Discussion of role to assist PSAs to access data and

provide a quarterly report of the indicators PSAs identify as helpful in their region

22

The MERF (DR)

Aims to address monitoring, evaluation and research functions- recognising all driven by data Three focus areas:

• Community/Service system monitoring
• Outcome evaluation
• Process/FVRIC activity monitoring

Initial phase utilises publicly available data as well as local/regional data where available Multi- year strategy recognising need for

• ngoing development of data literacy/ capacity

to support development of greater sophistication in MERF in later years

Three year plan (DR)

Moving forward CH

• Allison and Robyn

‘championing taking the MERF forward

• Strategy re CSA
• Further resourcing
• Further development of

data literacy and capacity building

• Taking forward

conversations re the Orange Door

Moderated Q&A

Alison Macdonald, DV Vic

Wrap Up and Next Steps

Project Reference Group Member FVRIC PSA John Duck

Wrap up and next steps (JD)