ADVANCING YOUR UNDERSTANDING OF THE PROGRAM PERFORMANCE REPORT: - - PowerPoint PPT Presentation

▶
advancing your understanding of the program performance
SMART_READER_LITE
LIVE PREVIEW

ADVANCING YOUR UNDERSTANDING OF THE PROGRAM PERFORMANCE REPORT: - - PowerPoint PPT Presentation

Oct 02, 2022 153 likes •600 views

ADVANCING YOUR UNDERSTANDING OF THE PROGRAM PERFORMANCE REPORT: CHANGES AND STRATEGIES TO USE IN REPORTING 2017 AIDD Technical Assistance Institute 1 SESSION AGENDA What is the PPR and Why do we do it? Changes to the PPR (By Section)

slide-1
SLIDE 1

ADVANCING YOUR UNDERSTANDING OF THE PROGRAM PERFORMANCE REPORT:

CHANGES AND STRATEGIES TO USE IN REPORTING

2017 AIDD Technical Assistance Institute

1

slide-2
SLIDE 2

SESSION AGENDA

  • What is the PPR and Why do we do it?
  • Changes to the PPR (By Section)
  • PPR activity (Why should we be the only ones

having all of the fun?)

  • More changes, tips and tricks
  • Networking Activity
  • Q&A

ITACC: 2017 AIDD Technical Assistance Institute

2

slide-3
SLIDE 3

THE PPR IN A NUTSHELL

The annual Program Performance Report - Due by Jan 1

  • 1. Documents compliance
  • 2. Updates analysis of state services, supports
  • 3. Summarizes progress on your Plan
  • 4. Describes the effectiveness of your strategies

ITACC: 2017 AIDD Technical Assistance Institute

3

slide-4
SLIDE 4

Notes from slide 3: The program performance report is an annual accounting of the Council’s work for a specific fiscal year. The PPR is due by January 1st of each year

4

slide-5
SLIDE 5

THE PPR IN A NUTSHELL

  • 1. Documentation of compliance:
  • Reporting the appropriate use of funds
  • Describing your work on required activities
  • Self-Advocacy
  • DD Network collaboration
  • Targeted disparity

ITACC: 2017 AIDD Technical Assistance Institute

5

slide-6
SLIDE 6

THE PPR IN A NUTSHELL

  • 2. Update of analysis of state services, supports
  • Adequacy of health care and other services,

supports, and assistance

  • People living in ICFs
  • People receiving HCBS services
  • Federal and state initiatives impacting people
  • Other sections in the plan that apply

ITACC: 2017 AIDD Technical Assistance Institute

6

slide-7
SLIDE 7

THE PPR IN A NUTSHELL

  • 3. Summary of progress on your Five Year State Plan:
  • Activities conducted during the year
  • Outcomes of those activities
  • Progress toward meeting Goals and Objectives
  • Impact on the lives of people with DD

ITACC: 2017 AIDD Technical Assistance Institute

7

slide-8
SLIDE 8

THE PPR IN A NUTSHELL

  • 4. Analysis of the effectiveness of your strategies:
  • What worked and why did it work?
  • What didn’t work and why didn’t it work?

This information can help you improve your Five Year State Plan

PPR State Plan

ITACC: 2017 AIDD Technical Assistance Institute

8

slide-9
SLIDE 9

SECTIONS OF THE PPR

  • Section 1: Identification
  • Section 2: Comprehensive Review
  • Section 3: State Plan Implementation
  • Section 4: Progress Report
  • Section 5: Fiscal Information
  • Section 6: Measures of Collaboration

ITACC: 2017 AIDD Technical Assistance Institute

9

slide-10
SLIDE 10

Notes from slide 8: Section 1: No changes Section 2: Minor additions/changes Section 3: A good amount of changes Section 4: Some changes Section 5: Changes Section 6: No substantial changes This presentation will focus heavily on the sections that have the most changes (Sect. 3 & 4)

10

slide-11
SLIDE 11
  • SECT. 3: STATE PLAN IMPLEMENTATION
  • Provides context of what

is being reported in the PPR.

  • Includes:
  • Introduction
  • Evaluation of State

Plan Implementation

ITACC: 2017 AIDD Technical Assistance Institute

11

slide-12
SLIDE 12

Notes from slide 9: Introduction: Overview of the entire section and PPR (Sect. 3 & 4). Evaluation of State Plan implementation: includes sections on evaluation activities, results of evaluation activities and efforts, and lessons learned and future work of the council based on evaluation efforts. Input on national priorities

12

slide-13
SLIDE 13
  • SECT. 3A: INTRODUCTION

Executive Summary

  • Overall efforts
  • Targeted areas of emphasis
  • Strategies used to implement

activities

  • Significant accomplishments
  • Barriers to implementation
  • Needs for State Plan amendments
  • Cultural Diversity
  • Efforts to address needs of individuals

with ID/DD and their families from culturally diverse communities

TIP:

  • Write this section

last!

  • Be mostly broad
  • Include a few

details (Ex. Aggregate #’s)

  • Do not duplicate
  • ther sections

ITACC: 2017 AIDD Technical Assistance Institute

13

slide-14
SLIDE 14

Notes from slide 10: Question: If I provide a summary of the overall report, am I writing the report twice? Answer: No! This should be a much broader more abbreviated version of your larger PPR

  • report. It should hit on larger ‘highlights’ of the whole year and report, not specifics for
  • bjectives. Think aggregate – ‘large picture’ overview. We recommend writing this section last!

14

slide-15
SLIDE 15
  • SECT. 3B: EVALUATION ACTIVITIES
  • Describe the Evaluation Activities undertaken

during the FY to assess overall progress of State Plan implementation

  • Examples of activities:
  • Monthly meetings going over progress on

activity plans

  • Council evaluation committee
  • Building evaluation into contracts
  • Comparing pre-post data (qualitative and

quantitative)

ITACC: 2017 AIDD Technical Assistance Institute

15

slide-16
SLIDE 16

Notes from slide 11: Question: Where do we come up with our evaluation activities? Answer: The first place we suggest starting is with the evaluation section of the State Plan – here you would have outlined all of the different approaches your council has outlined for evaluating progress on your state plan, secondly check out some of the evaluation tools that will be coming out of ITACC in the near future, third reach out to other councils and see what activities they are undertaking to help evaluate their state plan

16

slide-17
SLIDE 17
  • SECT. 3B: EVALUATION RESULTS
  • This section is a summary of the evaluation data you

have collected over the last FY

  • Types of data included: Qualitative and Quantitative
  • How do I know what to summarize in this section?
  • Review your Evaluation Plan
  • Review your logic model
  • Review your FY17 Work Plan in your State Plan
  • Review the AIDD PMs in your FY17 Work Plan
  • Review of grantee quarterly reports

ITACC: 2017 AIDD Technical Assistance Institute

17

slide-18
SLIDE 18

Notes from slide 12: This section is a summary of the evaluation data you have collected over the last FY Types of data included: Qualitative and Quantitative

  • Quantitative or qualitative data on outputs from objectives/activities
  • Quantitative or qualitative data on outcomes from objectives/activities
  • Quantitative or qualitative data on customer satisfaction

How do I know what to summarize in this section?

  • Review your Evaluation Plan
  • Review your logic model
  • Review the Data Evaluation and Measurement sections of your FY17 Work Plan in your

State Plan

  • Review of grantee work plans
  • Review the AIDD PMs you agreed to report on in your FY17 Work Plan for each
  • bjective

18

slide-19
SLIDE 19
  • SECT. 3B: EVALUATION RESULTS

Summarize the following information:

  • Data collected from Evaluation

activities

  • Progress on outputs and outcomes
  • Effectiveness of the strategies used
  • Results of customer satisfaction

data

  • Progress made on State Plan goals

TIPs:

  • Look at the data

collected and decide how best to summarize it

  • Make sure the data

you collected matches the data you agreed to collect in your FY17 Work Plan

  • Be prepared to

explain missing data

  • Summarize

additional data

ITACC: 2017 AIDD Technical Assistance Institute

19

slide-20
SLIDE 20

Notes from slide 13: Summarize the following information –

  • Information collected from all your evaluation activities
  • Progress made in achieving the outputs and outcomes of your state plan goals and objectives,

especially self-advocacy

  • Effectiveness of the strategies used to achieve the goals/objectives of the State Plan
  • Summarize results of customer satisfaction data collected
  • Determination of whether the projected Council results are being achieved
  • Findings regarding progress made toward achieving State Plan goals

TIPS: Look at the data collected and decide how best to summarize it Make sure the data you collected matches the data you agreed to collect in you FY17 Work Plan

  • If data is missing, be prepared to explain what happened, and if this changes what will be collected

in FY18

  • If you collected additional data beyond what you included in the Data Evaluation and

Measurement sections of your FY17 work plan, make sure to summarize it as well Question – If I report the results here, do I have to report any results in the progress narratives? Answer – Yes. All narrative must include a summary of evaluation activities to monitor progress and impact of council supported activities; data collected during the year, data sources (where you got the data), and data collection methods (how you got the data). And all narratives must include the AIDD performance measures for the activities being reported in the narrative.

20

slide-21
SLIDE 21
  • SECT. 3B: LESSONS LEARNED
  • Report on lessons learned during the FY
  • How will the council use these lessons learned?
  • How will the council use the data gathered to

move council work forward?

  • Possibly to:
  • Adjust future annual work plans,
  • Amend current annual work plans
  • Amend the State Plan
  • Shift project focus/strategies

ITACC: 2017 AIDD Technical Assistance Institute

21

slide-22
SLIDE 22
  • SECT. 3C: NATIONAL PRIORITIES
  • Purpose: to provide AIDD input on possible national

priorities and areas of potential collaboration at the federal level

  • Where do these ideas come from?
  • Evaluation results of Council’s projects
  • Policy work/collaborations with other groups
  • Needs assessments data in your state
  • Brainstorming with your Council Executive Director,

staff, and members

  • Survey of the DD Network Partners in your state

ITACC: 2017 AIDD Technical Assistance Institute

22

slide-23
SLIDE 23

Notes from slide 15: Purpose – to provide AIDD input on possible national priorities in the disability field and possible areas of collaboration at the federal/national level A few suggestion regarding where these ideas come from?

  • Evaluation results of your Council’s projects
  • Policy work/collaborations with other groups
  • Need assessment data in your state
  • Brainstorming with your Council Executive Director (and Council staff)
  • Brainstorming with Council Members at a meeting or Retreat
  • Survey of the DD Network Partners in your state

The key is to take some time to generate ideas to share with AIDD based on quantitative and/or qualitative sources Questions –

  • Do we have to fill out this section?
  • Where do I get this information from?
  • Am I required to ask my Council? My Executive Director?
  • Where does the information go?
  • Does my Governor need to approve?

Answers –

  • No, this is not a required section, however, it’s a good idea to have input into out national priorities

and agenda!

  • You can get this information from a variety of places, but in general it can be gathered from evaluation

data, needs assessment data, and input from others through brainstorming or planning sessions or surveys.

  • AIDD does not require the data be collected in a certain manner and no, the Council does not need the

“approval” of the Governor.

23

slide-24
SLIDE 24

SECT 4. PROGRESS REPORT

  • Focuses on progress toward achieving outcomes for the
  • bjectives and goals
  • Section includes:
  • Areas of emphasis
  • Strategies used
  • Collaborators on projects
  • Detailed information for each objective
  • Overall description of objective
  • Expected vs achieved outputs and outcomes
  • Objective narratives
  • Progress toward achieving outcomes
  • Work with culturally diverse populations under objective
  • Goal narratives
  • State Plan Overviews (Year 4 and Year 5)

ITACC: 2017 AIDD Technical Assistance Institute

24

slide-25
SLIDE 25

Notes from slide 16: Parts of this section will need to be repeated and done for each goal and/or objective. There is a distinction between objective and goal narratives! Objective narrative – includes providing an overall description of this objective, which should at a minimum include background information on the objective to establish a context for the activities of the objective. Goal narrative - For each goal that you are reporting on in the PPR, provide an overall description of the progress you made in achieving that goal, it’s outcomes, what barriers you encountered and if state plan amendments are required based on the results from the FY you are reporting on. We will go into more detail about the goal narrative a little later in the presentation.

25

slide-26
SLIDE 26

ITACC: 2017 AIDD Technical Assistance Institute

SECT 4. PROGRESS REPORT

26

slide-27
SLIDE 27

SECT 4A: PROGRESS REPORT

  • No substantial change in narrative elements required
  • Content includes:
  • Description of activities implemented to achieve
  • bjective
  • Description of the stage of implementation
  • Some of the numbers reported in the AIDD PMs section
  • Summary of evaluation activities for the objective
  • List of expected vs achieved outcomes
  • An assessment of progress made on intended
  • utcomes
  • Stories of people who participated in Council programs
  • Stories of policy or legislative changes that happened

as a result of Council work

ITACC: 2017 AIDD Technical Assistance Institute

27

slide-28
SLIDE 28

Notes from slide 18: No substantial changes in required content Description of activities implemented toward achieving the objective, including -

  • how the identified strategy was used
  • how the activity was implemented
  • challenges to achieving the objective
  • unexpected benefits

Description of the stage of implementation (for systems change activities) –

  • planning
  • initiation
  • implementation
  • utcome/fully integrated

Output and outcome numbers that are reported in the AIDD PMs section Summary of evaluation activities for the objective, which includes –

  • data collected during the year
  • data sources
  • data collection methods

List the expected outcomes for the objective/activities An assessment of the extent to which progress was made on the intended outcomes for the objective Stories of people who participated in Council programs whose lives are better Stories of policy or legislative changes that happened as a result of Council work

28

slide-29
SLIDE 29
  • SECT. 4A: PROGRESS REPORT -

ACTIVITY

  • What makes a “good”

narrative?

  • What makes an even

“better” narrative?

  • Let’s discuss!

ITACC: 2017 AIDD Technical Assistance Institute

29

slide-30
SLIDE 30
  • SECT. 4A: GOAL NARRATIVE
  • For each goal provide a description of

progress made toward achieving this goal for the FY you are reporting on

  • Include the following type of

analysis/information:

  • progress being made in achieving the

intended outcomes;

  • the extent to which each goal was or

was not achieved for the reporting; year

  • barriers that impeded goal

achievement

  • changes needed to be made to your

State Plan

Tips:

  • Ask yourself the

following questions when writing:

  • Did the major

activities of the

  • bjective help

accomplish the goal?

  • Does the data for

each objective show progress toward meeting goal?

  • Are you closer to

your long term

  • utcomes for

that goal based

  • n your 5 year

logic model?

ITACC: 2017 AIDD Technical Assistance Institute

30

slide-31
SLIDE 31

Notes from slide 20: Helpful hints – For the FY, did the major activities of the objective(s) under the goal work toward accomplishing that Council goal? Does your evaluation data (outputs and outcomes, AIDD PMs) for each objective indicate that progress was made toward reaching the Council goal? Looking at your 5 Year Summary Logic Model, are you closer to or further from your long term

  • utcomes and Impact Statements for the goal under review?

31

slide-32
SLIDE 32
  • SECT. 4A: PROGRESS REPORT

Tools for writing the narrative parts of Progress Report:

  • Guidance provided in PPR

template

  • PPR Annual Progress Report

Narrative Review Rubric Tool

  • Five Year State Plan - Tier 2

Review Tool

  • Guidance for Completing the

Annual DD Council Program Performance Report (PPR)

TIP:

  • Look at the

Tier 2 Review Tool peer review questions for the FY17 work plan section!!

ITACC: 2017 AIDD Technical Assistance Institute

32

slide-33
SLIDE 33

Notes from slide 21: Tools for writing the narrative parts of your Progress Report Guidance provided in the PPR template PPR Annual Progress Report Narrative Review Tool Rubric State Council on Developmental Disabilities Five Year State Plan - Quality Review System- Tier 2 Review Tool Look especially at the peer review questions for the FY17 work plan Guidance for Completing the Annual DD Council Program Performance Report (PPR) document provided by ITACC

33

slide-34
SLIDE 34
  • SECT. 4A: PROGRESS REPORT
  • Report individual and family demographic

data BY Goal

  • Demographic data includes:
  • Race/Ethnicity
  • Category (individual with DD/Family)
  • Geographic location
  • Report performance measure data BY

Goal & Objective

  • Performance measure data includes:
  • Individual and Family output and
  • utcome measures
  • Systems output and outcome

measures

TIPs:

  • Provide

language to grantee’s around demographic data collection

  • Pair outputs &
  • utcomes

together

  • Build outputs

& outcomes into contract language

ITACC: 2017 AIDD Technical Assistance Institute

34

slide-35
SLIDE 35

Notes from slide 22: Demographic and performance measure guidance documents are forthcoming Question: What are outputs? Outcomes? Answer: Great question! Outputs are referring to some of the tangible “products”, #’s, and data that you get from your council efforts. For example: the # of people trained, # of websites created, # of trainings implemented, # of new policies created. Outputs often can be calculated and shown

  • numerically. They provide a quick snapshot of something that the council has done. Outcomes on the
  • ther hand are a longer term look at the benefits or the “so what” of the outputs. For example. You

trained 100 people, that’s great but so what? What is the larger impact, value, or results of training those 100 people? Outcomes for that output could be - 100 people are now more knowledgeable about supported decision making, or 100 people are now participating in advocacy events because of their participation in the training. Outcomes can be reported numerically too, but often also include some qualitative data collection and discussion as well.

35

slide-36
SLIDE 36
  • SECT. 5: FISCAL INFORMATION
  • What’s new? The entire section!
  • The fiscal information is for programmatic purposes
  • nly
  • Purpose is to identify obligation/liquidation status for

the 3 FFY of funds (each year reported)

  • Funds
  • Federal
  • State
  • Additional
  • Match information
  • Required
  • Met
  • Unmet

ITACC: 2017 AIDD Technical Assistance Institute

36

slide-37
SLIDE 37

FINANCIAL INFORMATION

SECTION V: COUNCIL Financial Information [Section 124(c)(5)(B) and 125(c)(8)]

Council is its own DSA: ____Yes ____No

Fiscal Information for Programmatic Purposes ONLY

Purpose: In this section, Councils are to identify the obligation and liquidation status for the 3 FFY of funds

ITACC: 2017 AIDD Technical Assistance Institute

37

1. Fiscal Year YR 1. Reporting Period MM/DD/YR - MM/DD/YR 1. Total Federal Fiscal Award for Reporting Year $ 1. State Funds Contributing to Council State Plan Activities $ 1. Additional Council Funds Used for Other Activities $ 1. Federal Share of Expenditures $ 1. Federal Share of Unliquidated Obligations $ 1. Unliquidated Balance of Federal Funds (=Total Federal Fiscal Year Award – Federal Share of expenditures – Federal Share of unliquidated obligations $ 1. Match Required $ 1. Match Met $ 1. Match Unmet $

slide-38
SLIDE 38
  • SECT. 6: MEASURES OF

COLLABORATION

Collaboration other than the planned collaboration

UCEDD

DD COUNCIL P&A

ITACC: 2017 AIDD Technical Assistance Institute

38

slide-39
SLIDE 39

NETWORKING ACTIVITY

ITACC: 2017 AIDD Technical Assistance Institute

39

slide-40
SLIDE 40

Notes from Slide 26: What are your best tips and strategies for organizing your data for the report? What are your best tips and strategies for planning the development of the report? What are your best tips and strategies for writing the report?

40

slide-41
SLIDE 41

RESOURCES

  • Summary of PPR changes document
  • 2017-2021 Draft Program Performance Report Template
  • Calendar of Federal Reports – What is due? When?
  • PPR Rubric
  • PPR Narrative self-assessment with quality indicator prompts

New resources coming soon! Guidance document for completing the PPR Webinars will be offered in the Fall!

ITACC: 2017 AIDD Technical Assistance Institute

41

slide-42
SLIDE 42

CONTACT INFO

Sara Newell-Perez AIDD Sara.newell- perez@acl.hhs.gov Jessica Pidgeon NYS DD Council 1(518) 486-7505 Jessica.pidgeon@ ddpc.ny.gov Joanna Cordry TX DD Council (512) 437-5410 Joanna.Cordry@tcdd.state.tx.us Alicia Cone TN DD Council (615) 253-1105 Alicia.cone@tn.gov

42