Family Trust Beneficiary Visit Observations - Evaluation Analysis - - PowerPoint PPT Presentation

Sarva Mangal Family Trust

Beneficiary Visit Observations - Evaluation

Analysis and Recommendations Ronak Shah – SMFT Diplomat Summer 2016

Project Summary

From June 12th to July 12th of 2016, I visited sites of interest to the Sarva Mangal Family Trust across five states

• f India. In each case, SMFT is aiming to either strategically

scale its investment at these sites, or fund analogous programs in other parts of the country. The purpose of the site visits was to gain a more granular perspective of each program’s assets, challenges, and

• pportunities for growth and development.

Education in India

Misconceptions Realities Many students in villages and slums are not enrolled in school. Enrollment in India is now at nearly 100%, across caste, creed, religion, and region Students don’t attend school because parents think work is more important. Most families prioritize education over labor, but often must migrate for their own work. Only wealthy families send their kids to private schools, because all private schools are expensive. Nearly half of Indian students are enrolled in private schools, including a quarter of India’s 65% rural population. Private schools teach better than public schools, and students in private schools learn more. Affordable private schools generally have similarly dismal results to government schools. Instructional quality is the issue in all schools.

Project Lens

Programmatic Analysis Critical Inquiry Prioritized Impacts

Agenda Programmatic evaluations include analysis, recommendations, and a brief question & answer period

12:00 – 12:10 Introduction 12:10 – 12:30 Pratham Arora Center for Education 12:30 – 12:50 Pratham Hybrid Learning 12:50 – 1:10 Learning and Migration Program 1:10 – 1:30 Market Aligned Skills Training 1:30 – 1:50 Digital Equalizer 1:50 – 2:10 Akshaya Patra 2:10 – 2:30 Milaap

Pratham Arora Centers for Education

Maharashtra

Analysis: Program Structure

• Vocational training targeting low-

income rural youth

• Accelerated residential and non-

residential courses for rapid job placement and vertical mobility

• Automotive, bedside assistance,

construction, beauty, electrical, hospitality, plumbing, and tailoring

• Currently free or operating on a

learn-now-pay-later system

Analysis: Program Structure

• Serves youth ages 18-35
• Costs ₹10,000 per youth
• 2-3 months of training
• Daily Schedule:
• 8 hours of practical training
• 1 hour of classwork
• 4 hours of self-learning
• Transitioning to new payment scheme:

80% sponsored, 20% trainee-paid

Analysis: Program Structure

• Vocational Awareness: Digital

resources designed to build understanding of available jobs

• Vocational Camps: One-day

crash courses taught in villages

• Entrepreneurial Initiatives:

Counseling and resources for trainees who want to start their

• wn ventures with their skills

Analysis: Facilities

• Uses repurposed buildings

located near relevant industries to increase job placement

• Access to vocational tools,

materials, and resources

• Residential facilities clean,

spacious, and comfortable

• Food is nutritious and often

prepared by the students

Analysis: Personnel

• PACE facilitators require a minimum of

2 to 3 years industry experience

• Facilitators are trained in group

learning and use of technology

• Observation and feedback are given to

guide facilitators away from theory and towards practical training

• Vocabulary and theory are relegated

to self-learning rather than lecture

Analysis: T echnology

• Groups of four to five students

share a tablet for self-learning

• Tablets are pre-loaded with a

library of videos and resources

• f vocabulary reviews, skill

demonstrations, English, and stories of alumni in the field

• Grouping allows peers to teach

each other and address any misconceptions classmates have

Analysis: Curriculum

• Theoretical curriculum focuses heavily
• n learning vocational vocabulary
• Content is often strong but also

uneven, with some content better than

• thers, some gaps in the curriculum

for some vocational verticals, and inconsistent resource development standards within and across tracks

• Unclear the extent that self-learning

impacts job placement and retention

Recommendation: Self-Learning Curriculum

• Goals of self-learning curriculum should be:
• Accelerate pace of vocational training
• Increase likelihood of job placement
• Increase rate of job retention
• Should be complete enough to replace classroom

theory, leaving more time for practical training

• Should be high-quality enough to prepare students

more effectively than an instructor could in a classroom

Recommendation: Self-Learning Curriculum

Why shorten the length of the program?

• Reduced drop-out rates for rural students
• Shortened path to much-needed compensation
• Competing programs are equally long or longer
• Less time pushes facilitators to focus on the practical
• Shorter programs can reach more students

Smart use of technology to develop a strong self-learning curriculum will shorten the length while raising quality

Recommendation: Self-Learning Curriculum

Recommendation: Self-Learning Curriculum

Questions?

Pratham Hybrid Learning Program

Maharashtra

Would you feel comfortable sending them

• ut of the village for further studies?

Meeting with mothers in a village near Aurangabad

Mothers, would you encourage your daughters to study through college before getting married? Of course! Now seeing the light of learning inside them, they should learn as much as they can. No, it’s not safe for them out there. Yes, if that’s where the opportunities are. Why isn’t it safe? I once left my family for a job in the city. I came back a few years later, and my husband had died. The village claimed I murdered him, and I spent two years in prison for it before I was acquitted. Now I don’t want my daughter to repeat my mistake.

Analysis: Program Structure

• Designed to target:
• Low school attendance
• Low literacy and math skills
• Inadequate school facilities
• Dated instructional methods
• Insufficient instructional rigor
• Implemented through low-cost

tablets and smart TVs with pre- loaded academic resources

Analysis: Program Structure

• Groups of 4-5 students per tablet
• Flexible grouping by ability
• Mobilizers and facilitators invest

families and coordinate program

• Videos, games, and quizzes in:
• English
• Math
• Science
• Miscellaneous (termed “Fun!”)

Analysis: Program Structure

• Four major components
• Curriculum Design
• Content Development
• Formative Assessment
• Data Analysis & Feedback
• Pratham already has strong teams

dedicated to Content Development and Data Analysis & Feedback

Analysis: Program Structure

Key learnings so far:

• Students take care of technology
• Students find tablets engaging
• New content regularly needed
• English most popular; math least
• English used as a basis for roleplay
• Some videos are often rewatched

Recommendations

Priority Area #1 Motivation Towards Rigor Priority Area #2 Curriculum Design Priority Area #3 Formative Assessment

Priority #1: Motivation Towards Rigor

Rigor can be motivate or defeat depending on design, but is necessary for high-impact learning. Pratham should use game design to push students to challenge themselves.

• Avatars: Growth Mindset
• Experience: Mistakes as Learning
• Achievements: Discrete Goals

ClassDojo uses avatars to motivate students to manage their behavior in class. ClassCraft employs a range of point systems that help students track their growth. These are flexible and can be customized to suit the task that is desired. Khan Academy uses badges to visualize skills and achievements.

Priority #2: Curriculum Design

An engaging curriculum should nudge students through pacing and sequencing so that their learning is complete, rigorous, and deep.

• Clarity: Students must know what

they are learning and why

• Rigor: Activities should push

students towards higher thinking

• Alignment: Content at all levels

should fit together

Priority #3: Formative Assessment

Teachers need to know if students have learned anything at the end of a lesson so they can guide them to a better understanding. These assessments do not need to feel like formal tests, and can take many forms:

• Quiz: Uses tech to collect raw data
• Game: Tech engages while assessing
• Activity: Facilitator assesses

students informally without tech Simple data tables can help facilitators

• r the H-Learning system guide

students more appropriately.

Questions?

American India Foundation Learning and Migration Program

Gujarat

Parents, how many of you ask your children what they learned in school that day?

School governance meeting near Sayla

Parents, how many of you have visited the classroom your child is learning in? In a room of sixty, no hands go up. Five hands go up this time. Four happened to be members of the School Management Committee You all need to be asking your children what they learned! This is the only way you know if they are learning, and it helps them practice and remember it too!

Analysis: Program Structure

Provides services for children in migration-prone communities so students have stable access to a high quality education

Analysis: Program Structure

• Establishes hostels for children

whose families migrate for work.

• Three staff per hostel during the

migration season – September through January

• Hostels are expensive and moving

under government control.

• Government typically opens them

too late for students to utilize

Analysis: Program Structure

• Implementing Learning Enrichment

Programs in local schools

• Subject-based skills intervention that

takes place before and after the regular school day for one to two hours

• Intended to serve as a model for

instructional best practice for the local school and the community that supports it, to ultimately improve the school itself

Analysis: Program Structure

• Funding and resourcing Learning

Resource Centers in villages

• Structures in villages with many

educational resources, including books, manipulatives, posters, games, and activities

• Standalone centers that typically

augment the capacity of a Learning Enrichment Program

Recommendations

• LAMP sees itself as a model for

effective instruction, not as a sustainable solution in itself

• LAMP cannot rely on the

government to implement its program with fidelity

• LAMP in Gujarat has recognized

latent assets in the village School Management Committees

Recommendations

• A twelve-member School Management

Committee is required for every local school, with 70% made up of parents

• SMCs have been underutilized and

disempowered, but still exist as assets

• LAMP has been mobilizing and
• rganizing SMCs in Gujarat and has

seen a significant boost in educational involvement in those communities

Recommendations: Community Ownership

Priority Area #1 Educational Programs Priority Area #2 Migrant Housing Priority Area #3 School Improvement

School Management Committee Ownership

Educational Programs

• Advocate for funding

LEPs and LRCs beyond three year AIF commitment

• Leverage Panchayat

to recruit students back from out-of- village private schools

Migrant Housing

• Meet with family and

friends of migrant families to house students seasonally

• Secure funding for

food and essentials during these months (family, donors, gov’t)

School Improvement

• Regularly observe

school instruction

• Provide feedback to

the local teacher

• Encourage parents to

do the same and to actively participate and volunteer

Questions?

American India Foundation Market Aligned Skills Training

Haryana

Why do you have two sewing machines in here instead of just one?

Meeting with a MAST alumna at her home

Why is part of your sewing machine on the floor instead of on its stand? There’s no light in the house, so I remove the machine and take it to the

• roof. I can get a few more hours of

daylight there. I have to work it by hand instead of with the pedal, but I can make more garments to sell.

Meeting with a MAST alumna in her chai shop

Once I was able to save enough money from the garments I was selling, I purchased a second sewing machine so I can teach other girls to sew while I’m making my own clothes. I can make some money that way too.

Analysis: Program Structure

• Vocational training to young adults who have

encountered barriers to employment

• 300 hours of coursework over the course of

three months – shorter if residential

• Vocational programs offered are based in

scans of regional market needs

• Focus on women, as in conservative states

many find it difficult to leave the home for long educational programs or for employment

Analysis: Comparing PACE and MAST

PACE MAST Cost of Program ₹10,000 ₹10,000 Cost to Beneficiary ₹2,000 (some still free or pay later) ₹0 Length of Program 2 to 3 months 2 to 3 months Hours of Instruction 700 hrs. + 200 hrs. self-learning 300 hrs. Residential Primarily residential Primarily non-residential Job Placement 75% to 80% 75% to 80% Program Scope 16 states, 80 centers 20 states, 177 centers Post-Placement Services One year One month Practice :Theory 8 : 1 (2 : 1 counting self-learning) 2 : 1

Analysis: Rewari MAST Centre

• Programs: cosmetology, electrical

work, garment making, information technology, and general workplace readiness including English language

• Workshop with ten sewing

machines available for graduates of garment making program

• Building located to reach more

candidates than previous location

Analysis: Rewari MAST Centre

• Post-tracking is short-term
• Women graduates expressed mixed

access to employment

• Capital is limited to self-employ

themselves in new enterprises, such as garment-making or teaching

• MAST garment workshop has

limited hours, prohibitive distance, and cultural barriers to usage

Priority Area #1: Microlending. AIF has used this tactic to help rickshaw drivers move from renters to co-owners of their vehicles. Priority Area #2: Sanghs. AIF organized the rickshaw drivers into collectives to broker loans, financing, and insurance in aggregate. Priority Area #3: Capacity-Building. AIF used its intermediary position to guarantee first loan defaults and expedite licensing and paperwork, at low-cost to all parties.

Recommendation: Skills to Enterprises

The example set through AIF’s success with its Rickshaw Sanghs can be applied to graduates of

• ther vocational training programs, like textiles.
• Microlending. Graduates need capital for home

sewing machines and startup materials.

• Sanghs. Graduates will have more marketing

power and lower operational costs in a collective. Capacity-Building. AIF can leverage its staff and resources to build the collective’s initial capacity, then release itself as the Sangh gains momentum.

Recommendation: Skills to Enterprises

Questions?

American India Foundation Digital Equalizer

T elangana

Analysis: Program Structure

• DE is the only program AIF takes to implementation,

rather than relying on NGO partners to carry out

• DE establishes computer labs in schools with ten

computers, a projector, and internet connectivity

• DE funds the center for three years, focusing on grades

6-10 in schools large enough to reach 400 students

• DE uses the center to train educators to use

technology in instructional planning and methods

Analysis: Program Structure

• In addition to digital training, DE
• bserves teachers weekly
• A DE coordinator manages 4 to 5

teachers and gives feedback on use of technology, engagement, and instructional methods

• Trainees are expected to produce
• ne DE lesson monthly and one

flipped lesson every 2 months

Analysis: Program Impact

• DE leverages recognition to

encourage teacher innovation, helping 118 teachers be recognized through the STiR program

• DE implements SwachVidyalaya in

its schools, assessing and reporting school environment conditions

• DE uses a hub-and-spoke model in

some centers to scale its impact

($108 / Rs. 7,200 per teacher)

Analysis: Program Impact

• Despite its large reach, staff, and

budget, the impact of the core technology is unclear

• The number computers is small

compared to class sizes

• Frequency that teachers must use

DE strategies is very low and does not increase with time

• Computers engage and attract

students, teachers, and donors

Recommendations: Invest in Karimnagar

During the site visit, a ribbon-cutting was held. The district collector was invited but did not initially attend. A reporter from The Hindu was there and called her, but still she

• refused. The reporter then drove to where she was, picked

her up, and brought her to the ribbon-cutting. Initially, she said she would only stay for five minutes to listen to me as I was from out of town. In fact, she stayed for twenty minutes and verbally committed to the government funding 10% of Digital Equalizer efforts in Karimnagar, with the reporter from The Hindu present.

Recommendations: Invest in Karimnagar

• Politically, socially, and economically,

Karimnagar is a strong opportunity to invest in the Digital Equalizer

• Teachers should be expected to

implement DE pedagogy more often as the three year period continues

• Computer labs in hubs should have

a 2:1 student to computer ratio using the average class size

Recommendations: Invest in Karimnagar

Digital Equalizer T eaching Strategies Use of Flipped Lesson Plan Frequency Duration Frequency Duration Once every month 6 months (6 times) Once in two months 6 months (3 times) Once in three weeks 6 months (8 times) Once in six weeks 6 months (4 times) Once in two weeks 6 months (12 times) Once every month 6 months (6 times) Once every week 6 months (24 times) Once in three weeks 6 months (8 times) Twice a week 6 months (48 times) Once in two weeks 6 months (12 times) Daily 6 months (120 times) Once every week 6 months (24 times)

Questions?

Akshaya Patra

Karnataka

Analysis: Program Structure

• Centralized and decentralized

kitchens prepare mid-day meals for 1.4 million children daily

• Meals designed to meet

government nutritional and hygienic standards

• Centralized kitchens ship meals
• n insulated trucks throughout

mostly urban school districts

Analysis: Meal Comparison

Akshaya Patra Meal Valley School Meal

Components Rice (1 cup) Sambar (1 cup) Curds Components Rice Dal Chapati Sabji Salad Curds Primary 450 calories 12g protein Secondary 700 calories 20g protein

Analysis: Centralized or Decentralized?

Centralized Kitchens Decentralized Kitchens

• More effective hygiene protocols
• Higher volume of production
• Economy of scale in urban areas
• Efficient and frequent oversight
• Local employment opportunities
• Regional menu responsiveness
• More practical in rural areas
• Perishable foods more available
• Hard to implement in remote areas
• Less availability of fresh fruit in meals
• Homogenized menus for all schools
• Caste and gender privileged work
• Less operational consistency
• More infrequent oversight
• Inadequate kitchen infrastructure
• Less hygienic standards

Recommendations: Survey Landscape

Determine if decentralized kitchens or a centralized kitchen better fits the geography:

• How dense are the government schools?
• What is the operational and hygienic

infrastructure for preparing meals locally?

• What are the respective transit costs?
• Is there a need to employ woman in local mahila

mandals or from scheduled castes that may have barriers to work in a centralized kitchen?

Questions?

Milaap

Tamil Nadu

Analysis: Program Structure

• Milaap coordinates and supports

microlending by partnering with nearly 50 local banks and NGOs

• Local partners administer, distribute,

and manage lending on the ground

• Milaap markets and fundraises for

these loans using an online interface which transparently tells borrower stories and shows full lending accounts

Analysis: Program Structure

Analysis: Enterprise Lending

• Microfinance traditionally focuses
• n lending for purchases that

directly generate income

• Borrowers use loans buy raw

materials or tools to make a product they can sell

• Borrowers may also use loans to

gain a skill or a technology so that they can sell a service

Analysis: Alternative Lending

• Milaap and its partners also offer

non-enterprise loans such as those depicted in the graphic to the right

• These loans comprise the majority of

Milaap’s lending and build quality of life as a part of overall livelihood

• These are sometimes paired with

enterprise loans, with different interest rates for each loan

Analysis: Comparing Lending Methods

Local Microfinance NGOs Milaap Small Banks

• Geographic targeting
• Programmatic targeting
• Investor has determining

capacity over programs

• Flexible and nimble
• Scaled across the country
• Diverse finance portfolio
• Capacity for innovative loans
• Focus on vulnerable people
• Vetted grassroots partners
• Foothold within regions
• Incentives for microfinancing
• Public reporting mandatory
• Sustainable without outside

charitable support

• Vetting done on one’s own
• Limited operational support
• Limited geographic scale
• Limited programmatic scope
• Beneficiaries depart as their

capacity allows autonomy

• Investor leaves determining

power to funding partners

• Avoids vulnerable people
• Risk-averse lending
• Beholden to investors
• Investor leaves determining

power to bank discretion

Recommendations: Paired Lending

• Paired lending allows for income

generating opportunities while also addressing quality of life

• Paired loans are also a way of connected

Milaap’s partners with successful vocational training programs SMFT already works with in appropriate geographies

• Once a partnership is established, SMFT

can work closely with the ground-level partner to shape a program

Conclusion

Vocational Training

• PACE and MAST operate at the same cost for the same time, but

PACE delivers over twice as many instructional hours and monitors graduates for a longer period

• In geographies near their lending partners, Milaap may be able to

coordinate ₹2,000 microloans for borrowers near PACE locations to receive vocational training, and may be able to pair these with enterprise development loans

• Increasing the availability of PACE centers will have a greater

impact than increasing the tablet to student ratio, as the current ratio allows for collaboration and group learning.

Comparing PACE and MAST

PACE MAST Cost of Program ₹10,000 ₹10,000 Cost to Beneficiary ₹2,000 (some still free or pay later) ₹0 Length of Program 2 to 3 months 2 to 3 months Hours of Instruction 700 hrs. + 200 hrs. self-learning 300 hrs. Residential Primarily residential Primarily non-residential Job Placement 75% to 80% 75% to 80% Program Scope 16 states, 80 centers 20 states, 177 centers Post-Placement Services One year One month Practice :Theory 8 : 1 (2 : 1 counting self-learning) 2 : 1

Education

• Instructional design and quality should be the primary criteria in

education programs supported by SMFT

• Pratham’s education programs already have a strong foothold, and

would be strengthened by curriculum development and instructor

• bservation and feedback
• Education programs like LAMP will only be sustainable if AIF can

establish a ground presence, which may be possible by mobilizing School Management Committees

• Digital Equalizer’s greatest strength is its observation and

feedback system, and this should be emphasized and supported.

Technology

• Technology is being used for two interrelated goals: digital

literacy and instructional improvement

• For digital literacy, one-to-one ratios are preferable, as

students need to be able to interact and problem solve on their own. The best program to invest in this is DE.

• For instructional improvement, the key lever is the quality and
• rganization of the content, so that digital content accelerates

the pace of learning, be it in primary education or in vocational training. This is more critical than the tech ratio.

Questions?

About the Analyst

Ronak Shah is a middle school science teacher in Indianapolis, Indiana. He also has experience teaching math, reading, writing, and social studies, all at the middle school level. He teaches in inner city schools where a majority of students are from low income families. He received his Masters in Teaching at Marian University in Indianapolis, after completing a B.S. in Cognitive Science and a B.A. in Conflict Resolution at Indiana University, Bloomington, where he won numerous awards, including the prestigious Wells Scholarship and the Stahr Distinguished Senior

• award. In addition to teaching, Mr. Shah is a Teach Plus Policy Fellow,

conducting research and writing op-eds on policy issues such as assessment, discipline, and school accountability.

• Mr. Shah recognizes that student success depends not only on classroom

teaching but also on holistic factors such as family support and availability

• f healthy meals. Since many of his students lack these basic needs, Mr.

Shah started an after school program at his school teaching students about healthy cooking and urban gardening This work has been featured in the documentary Food First about the Indianapolis food system.