How to Find the Poor: Field Experiments on Targeting Abhijit - - PowerPoint PPT Presentation

How to Find the Poor: Field Experiments on Targeting

Abhijit Banerjee, MIT

Why is targeting hard?

• Targeting entails a different set of challenges in developing

countries because governments lack reliable data on incomes

• Several methods used to address this problem entail a

tradeoff between information and local preferences:

▫ Proxy-means testing (PMT): government collects data on hard-to- hide-assets to proxy for consumption ▫ Community-based targeting: allow local community discretion to decide who is poor ▫ Self-selection: allow people to apply –and make the application process costly

 May be then do a PMT

Which of these works best?

• The Indonesian government wanted to know
• Not an easy question:
• Different programs are targeted differently but they are also

different in fifty other ways

• Also within each model what is the optimal design?

▫ Community targeting by the whole community

 Or just by the elites?

▫ How costly should the application process be?

 One extreme is National Rural Employment Guarantee Act (NREGA)  Another extreme is a one time application cost

• Evidence from experiments

Proj ect 1: PMT vs. Community Targeting

• This study examined a special, one-time real

transfer program operated by the government of Indonesia

▫ Beneficiaries would receive a one-time, US$3 transfer (PPP$6) ▫ Goal of program was to target those with per-capita consumption less than PPP$2 per day

• Sample consists of 640 sub-villages (rural and

urban) across 3 provinces in Indonesia

The PMT Method

• Government chose 49 indicators, encompassing the

household’s home (wall type, roof type, etc), assets (own a TV, motorbike, etc), household composition, and household head’s education and occupation

• Use pre-existing survey data to estimated district-

specific formulas that map indicators to PCE

• Government enumerators collected asset data door-

to-door

• PMT scores calculated, and those below village-

specific (ex-ante) cutoff received transfer

The Community Method

• Goal: have community members rank all households in

sub-village from poorest (“paling m iskin”) to most well-

• ff (“paling m am pu”)
• Method:

▫ Community meeting held, all households invited ▫ Stack of index cards, one for each household (randomly ordered) ▫ Facilitator began with open-ended discussion on poverty (about 15 minutes) ▫ Start by comparing the first two cards, then keep ranking cards

• ne by one
• Also varied who was invited (elites or everyone)

Hybrid

• Hybrid combined community with PMT

verification of very poor

• The idea was that the community could pre-screen

those to be put in to the PMT

Time Line

Baseline Survey

• Nov to Dec 2008

Targeting

• Dec 2008 to Jan 2009

Fund Distribution, com plaint form s & interviews with the sub- village heads

• Feb 2009

Endline Survey

• late Feb and early

Mar2009

Distribution of Per Capita Cons.

• PMT centered to the left
• f community methods—

better performing on average

• However, community

methods select slightly of the very poor (those below PPP$1 per day)

• On net, beneficiaries

have similar average consumption

(1) Full population Sample: Community treatment 0.031* (0.017) Hybrid treatment 0.029* (0.016) Observations 5753 Mean in PMT treatment 0.30

• Using the $2 per

capita expenditure cutoff per day, 3 percentage point (or 10 percent) increase in mistargeting in community and hybrid over the PMT

MISTARGETivk= α + β1COMMUNITY

ivk + β2HYBRIDivk + γk + εivk

Community S atisfaction: Endline

Is the method applied to determine the targeted households appropriate? (1=worst,4=best) Are you satisfied with P2K08 activities in this sub-village in general? (1=worst,4=best) Are there any poor HH which should be added to the list? (0=no, 1 = yes) Community treatment 0.161*** 0.245***

• 0.189***

(0.056) (0.049) (0.040) Hybrid treatment 0.018 0.063 0.020 (0.055) (0.049) (0.042) Observations 1089 1214 1435 Mean in PMT treatment 3.243 3.042 0.568 Number of HH that should be added from list Number of HH that should be subtracted from list Number of complaints in the comment box Community treatment

• 0.578***
• 0.554***
• 1.085***

(0.158) (0.112) (0.286) Hybrid treatment 0.078

• 0.171
• 0.554**

(0.188) (0.129) (0.285) Observations 1435 1435 640 Mean in PMT treatment 1.458 0.968 1.694

Understanding the differences

• Summary of results so far:

▫ Community methods did slightly worse based on PPP$ poverty line. No differences in average consumption ▫ Hybrid and community do exactly equally well ▫ Villagers overwhelmingly happier with community

• Why might PMT be different?

1. Elite capture – No

• 2. Effort (community gets tired) – Yes
• 3. Different concept of poverty – Yes
• 4. Different information (community less accurate that

PMT) – Unlikely

• 1. Elites
• Elite subtreatment:

▫ In half of community/ hybrid villages, only local small group

• f elites (neighborhood leader + a few others) invited to

meetings ▫ In other villages, whole community invited

• Results:

▫ No change in average mistargeting rates ▫ No change in mistargeting rates for households who are family members of elites ▫ Only result is that in community treatment, elite connected households even less likely to be on list than PMT – this “reverse discrimination” is eliminated to some degree in elite treatment

• 2. Community Effort
• To rank 75 households, must make 365 pairwise

comparisons

• Community members may get fatigued throughout

the meeting

• To investigate this, we randomized the order in

which households were considered

• We find that those ranked early in the meeting were

ranked more accurately – in fact, more accurately than PMT

• 3. Does the Community Have a

Different Concept of Poverty?

• Communities could be identifying who is poor, but have

a different view of what constitutes poverty

• We therefore estimate:

▫ RANKCORRvkw= α + β1 COMMUNITY

vk + β2 HYBRIDvk +

γk + εvkw

where RANKCORR is the rank correlation in each village between targeting rank list and:

▫ Consumption (ug) ▫ Community survey ranks (uc) ▫ Sub-village head ranks (ue) ▫ Self assessments (us)

Impact on different welfare metrics

Consumption (ug) Community survey ranks (uc) Subvillage head survey ranks(ue) Self Assessment (us) Community

• 0.065**

0.246*** 0.248*** 0.102*** (0.033) (0.029) (0.038) (0.033) Hybrid

• 0.067**

0.143*** 0.128*** 0.075** (0.033) (0.029) (0.038) (0.033) Observations 640 640 640 637 Mean in PMT treatment 0.451 0.506 0.456 0.343

• Community methods have lower correlation between targeting rank

list and consumption, but higher correlation between targeting rank list and self-assessment

• Hybrid closer to community preferences than PMT, but less so than

community method

Unpacking the local welfare metric

• To investigate this, we regress survey ranks and

community ranks in a wide variety of household characteristics, conditional on per-capita consumption

Key findings

• Household equivalence scales:

▫ Community accounts for household economies of scale ▫ Community treats kids as more costly than adults (e.g., greater “burden”)

• Networks for smoothing shocks

▫ Community counts elite-connected households as better off than indicated by consumption ▫ Community counts those who have high share of savings in banks as better off, even though total savings doesn’t affect rank ▫ Those with family outside the village also counted as richer

Key differences

• Discrimination

▫ No discrimination against ethnic minorities

• “Laziness” or “deservingness”

▫ Those with only primary education treated as poorer, conditional on actual consumption ▫ Widows, disabled, and those with serious illness all treated as poorer, conditional on actual consumption

• 4. Is community information worse?

Survey rank Survey rank (1) (2) Rank per capita consumption within 0.132*** 0.088*** village in percentiles (0.014) (0.012) Rank per capita consumption from 0.368*** PMT within village in percentiles (0.014) Individual PM Variables YES

• Unlikely: in fact, community members have additional

information about consumption over the PMT

• Controlling for PMT score, a one percentile increase in

consumption rank is associated with a 0.132 percentile increase in individual household ranks of the community

Conclusions

• Decentralizing to the community:

▫ Increased local satisfaction with the outcomes ▫ Produced only slightly worse targeting based on per- capita consumption

• Reason is primarily that local communities have a

different welfare function than the central government

PMT vs. S elf-selection + PMT

• One way to do so is to impose program

requirements that are differentially costly for the rich and the poor

• Welfare programs with labor requirements (WPA,

NREGA)

▫ Food schemes with lower quality food ▫ Unemployment schemes with weekly requirements to visit unemployment office during work hours

• Many self selection mechanisms rely on the idea

that “time” is more costly for the rich than the poor

Other Factors that May Worsen S election

• Introducing realistic features into the model (e.g.

concave utility) may result in worse targeting, if poor have higher money costs, but lower utility costs

• Behavioral arguments: self control (Madrian and

Shea, 2001); stigma (Moffitt, 1983); information (Daponte, Sanders and Taylor, 1999)

• Thus, whether self-section improves targeting is

ultimately an empirical question

S etting for Proj ect 2:

• Experiment Takes place in the context of

Indonesia’s Conditional Cash Transfer Program, PKH

▫ Must be very poor, defined as < 80% of poverty line ▫ High stakes: household annual benefits between Rp. 600,000 (US$66) and Rp. 2,200,000 (US$245) per year (11% consumption for a typical beneficiary)

• We examine the expansion of the program to 400

new villages in 3 provinces in Indonesia

Experimental Design

• To qualify for PKH: means test is determined by a

PMT survey based on assets and demographics

• Randomize targeting method:

▫ Automatic Enrollment System (200 villages): Status Quo in Indonesia ▫ Self-Targeting (200 villages)

 Households must come to application site to apply, then take asset test  Randomly varied travel time, opportunity cost of signing up

Explaining the Program

Application Process

Timeline

Baseline Survey

• Consumption
• Travel costs

to locations

• Variables for

PMT formula Targeting and Intervention

• Government

conducts targeting

• PKH funds

begin to be distributed Endline

• Satisfaction
• Process

questions: e.g. wait time during self- targeting

Who S elf-S elects?

• Self Targeting may differ than an automatic

enrollment system on two dimensions

1. Observable Characteristics: Households that would fail PMT test anyway may be less likely to come— save government money since no longer have to interview them 2. Unobservable Characteristics: Noise in PMT test, and so richer households may select out of being tested—can result in a poorer group of beneficiaries

• Decompose consumption into PMT score and

residual

Observable Characteristics

Unobservables

Comparing across treatments

• Selection on both observable and unobservable

components

• But, still in self-targeting, only 60% of those who are

eligible show up!

• How does this compare against:

▫ Status Quo of Automatic Enrollment on those who have been selected to be interviewed ▫ Hypothetical Universal Automatic Enrollment

• Self Targeting leads to a poorer distribution of

beneficiaries

• ST reduces both exclusion and inclusion error:

▫ 16 percent of those who are in the bottom 5 percent receive benefits in ST , as opposed to 7 in AE (sig at 10% level) ▫ Households in top 50 percent of consumption are more than twice as likely to receive benefits (sig at 1% level)

Taking S tock of Experimental Results

• Self-targeting leads to a poorer distribution of

beneficiaries, both because the poor are more likely to receive benefits and the rich are less likely

• Compare against hypothetical where everyone is

surveyed for automatic enrollment leads to similar results:

▫ ST leads to a poorer distribution of beneficiaries (although significance depends on specification) ▫ Probability of poor being selected is similar under both systems, but wealthier people more likely to receive it under AE

Costs of Alternative Approaches

• ST places a greater total cost on households:

$70,000 compared to $9300 in AE and $32,403 for universal AE

• In sample: Administrative costs for ST is about

$171,000: it is about 4.5 times that cost for AE and about 13 times that cost for AE

• Assuming we treat costs by households and

administrative costs the same, ST leads to a better distribution of beneficiaries at total lower costs

What is the Marginal Effect of Increasing the Ordeal?

• Difference between one sign up cost and NREGA
• Experimental Evidence: ST Villages were allocated

to the following two sub-treatments:

▫ Application Site Close or Far

• Increasing distance does not improve self-

selection—just reduces massively application rates, even for the poorest

Conclusions

• These two projects investigated alternative approaches

to identifying poor households

• Found that:

▫ Community targeting did about the same as PMT in terms

• f identifying people based on per-capita consumption, but

much better in terms of local poverty metrics. ▫ Self-targeting did a much better job at differentiating between poor and rich than automatic PMT, although it does impose costs on applicant households

• However all the approaches miss a very large proportion
• f the poor
• Making the ordeal harder does not help.