RWANDA RUSA Louis HUMUZA James LUDWIG DE NAEYER NSHIMIRUMUREMYI - - PowerPoint PPT Presentation

Measur suring ing Result lts s and nd Evalu luating ating Impa pact: t:

Turn rnin ing g Promise ises s in into Evid idence ence

Cape Town, South Africa December 2009

RWANDA

Human Development Network Development Impact Evaluation Initiative Spanish Impact Evaluation Fund Africa Region

RUSA Louis HUMUZA James LUDWIG DE NAEYER NSHIMIRUMUREMYI Christophe JENIFER STURDY BASINGA PAULIN

2

• 1. Background

 Results Based Financing on raise in the

development agenda, specially in the health sector

 Importance of linking payment to

performance

 How to measure performance routinely so

that payment can be linked to accurate performance

 Scaling of RBF

Background : RWANDA

 PBF at the health facilities nationally

implemented

 Concerns about the accuracy of the data being

reported by health facilities

 Measurement of the performance rewarded by

PBF



High temptation of cheating

 Experimentation setting for an evaluation

experiment on data verification

 Results to be used :

 Improve reporting system in Rwanda  Behavior change by implementing penalties  Generate knowledge on data verification strategy in

context of RBF (what cost effective strategy)

 Inform other RBF projects

Unanswered questions

 What is the “right” indicators to monitor for cheating?

 Amount of Incentive ?  Frequency ?

 What is the proportion of planned and unplanned

cheating

 What are the right methods to detect cheating:

 Record verification at facility level?  Patient tracing at the community level?  Combination of methods?

 How can we use the results of the data verification to

inform the system

 What are the appropriate penalties that can be

applied to correct perverse behavior



What intensity and frequency of data check and validation is sufficient to ensure quality?

5

• 3. Study objectives

 General objective :

 Strengthen and Improve accuracy of the

information used to reward performance

 Specifics objectives:

 Improve data quality  Ensure accuracy of the data  Pay for true performance  Promote culture of accountability

6

• 5. Intervention

 Checking the consistency of the data

system at health facility level:

 Data reliability check  Data accuracy check : tracing of the patient

across different services (reception, consultation, laboratory, pharmacy). Check the date of payment in the registry book (health insurance if appropriate)

 Tracing patients: ghosts clients

 Trace patients at the community

7

• 5. Identification Strategy/Method

Random assignment of facilities into 5 arms:

 Control group : Data reliability check: monthly

report check with registry

 T1: Data accuracy check. Patient tracing

across different services : Once a year

 T2: Data accuracy check. Patient tracing

across different services : Twice a year

 T3: Patient tracing in the community : once a

year

 T4: Patient tracing in the community : twice a

year

8

• 6. Sample and data

 Selection of the indicators to be checked

 Total possible : 24 (13 PMA and 11 HIV)  Choice based on level of incentive?  Tracer indicators (maternity indicators:

prenatal & deliveries?)

 Ethical consideration (HIV services, modern

contraceptive)

 Power calculation to determine the

number of patients / facility to be sampled

 Patient record  Community

 Study tools development

Methodology

 Data monitoring program:

 Matching of facilities based on the quality

score of data registry

 Block randomization of health facilities in 5

study arms

 Baseline in all arms on data quality  Public announcement of the data quality check

to everybody in the coming 3 years

 Intervention : auditing  Follow up study

9

Identification Strategy/Method

CONTROL

DATA ACCURACY CHECK ACROSS RECORDS

Low frequency (1/year) High frequency (2x/ year)

PATIENTS TRACING IN THE COMMUNITY

Low frequency (1/year) High frequency (2x/ year)

10

11

• 7. Time Frame/Work Plan

 2010 -2013?

12

• 8. Sources of Financing

 HRBF ?