Services Research Helen Burstin, MD, MPH, FACP Chief Scientific - - PowerPoint PPT Presentation

The Growing Intersection Between Data Journalism and Health Services Research

Helen Burstin, MD, MPH, FACP Chief Scientific Officer, NQF

June 26, 2016

NQF Mission

The National Quality Forum leads national collaboration to improve health and healthcare quality through measurement.

Definitions of HSR

3

• AHRQ: Examines how people get access to health care,

how much care costs, and what happens to patients as a result of this care.

• Academy Health: HSR is the science/field of study that

asks:

▫ What works? ▫ For whom? ▫ At what cost? ▫ Under what circumstances?

• Greater recognition that “research” can take many

forms beyond classic academic paradigm

Data Journalism

4

• “Real data journalism comes down to a couple of

predilections: a tendency to look for what is categorizable, quantifiable and comparable in any news topic and a conviction that technology, properly applied to these aspects, can tell us something about the story that is both worth knowing and unknowable in any other way.“

• “Great data projects don't generally begin with great

data sets. They begin with great questions and the desire to find the hardest evidence available to answer those questions.“

» http://www.poynter.org/author/troy-thibodeaux/

National Quality Strategy on

5

Progress: Hospital Acquired Infections

6

NQF Evaluation Criteria

• Importance to measure and report

▫ What is the level of evidence for the measures? ▫ Is there an opportunity for improvement?

• Scientific acceptability of the measurement

properties

▫ What is the reliability and validity of the measure?

• Usability and Use

▫ Can potential audiences use performance results for both

accountability and performance improvement?

• Feasibility

▫ Can the measure be implemented without undue burden,

capture with electronic data/EHRs?

Importance of Measure Testing

8

Reliability testing

• Demonstrates the measure data elements are

repeatable, producing the same results a high proportion

• f the time when assessed in the same population in the

same time period and/or that the measure score is precise. Validity testing

• Demonstrates that the measure data elements are

correct and/or the measure score correctly reflects the quality of care provided, adequately identifying differences in quality.

Threats to Validity

9

• Measure focus is not a relevant outcome of healthcare
• r not strongly linked to a relevant outcome
• An unreliable measure cannot be valid
• Patients inappropriately excluded from measurement
• Differences in patient mix for outcome and resource use

measures not adjusted

• Measure scores generated with multiple data

sources/methods

• Systematic missing or “incorrect” data (unintentional or

intentional)

Rigor & Transparency

10

Measurement: Drive Selection and Change

11

Fung et al. adaptation of Berwick, James and Coye’s Conceptual Framework

NQF: Transparency

12

• Transparency is the extent to which performance results

about identifiable, accountable entities are disclosed and available outside of the organizations or practices whose performance is measured.

• Maximal transparency is achieved with public reporting

defined as making comparative performance results about identifiable, accountable entities to the public at large (generally on a public website).

• At a minimum, the data on performance results about

identifiable, accountable entities are available to the public.

• The capability to verify the performance results adds

substantially to transparency.

Requirements for Transparency of Endorsed Measures

13

• Use in Public Reporting:

▫ Disclosure of performance results to the public at large ▫ Performance results are used in at least one accountability

application within three years after initial endorsement and are publicly reported within six years after initial endorsement (or the data on performance results are available).

▫ If not in use at the time of initial endorsement, then a

credible plan for implementation within the specified timeframes is provided.

14

Consumer Reports, 2011

15

• It's sad but true that it's easier to make an informed

choice about cars and refrigerators than it is about health-care providers. To help correct that problem, we’ve teamed with the Society of Thoracic Surgeons (STS) to publish ratings of heart-surgery groups based on their performance data for bypass surgery.

• Using this information, consumers can see how surgical

groups compare with national benchmarks for overall performance, survival, complications, and other measures.

• No organization is better poised to provide this

snapshot of surgical outcomes than the STS, a nonprofit

• rganization that represents some 5,400 surgeons

worldwide who operate on the thorax, or chest. “

Improvement and Potential for Unintended Consequences

16

• Progress toward achieving the goal of high-quality,

efficient healthcare for individuals or populations is demonstrated.

• The benefits of the performance measure in facilitating

progress toward achieving high-quality, efficient healthcare for individuals or populations outweigh evidence of unintended negative consequences to individuals or populations (if such evidence exists).

Path Forward: Data and Measurement

17

• Push for greater transparency of the best possible data

for quality measurement and reporting - status quo is not acceptable

• Use standardized measures vetted through

multistakeholder process whenever possible to avoid focus on methods over end-goal

• Recognize that HSR data and methods must evolve to

meet this brave new world of apps and ready access to data

Onpoint Health Analytics

Craig Jones, Department of Vermont Health Access

18

Claims data from APCD

Data Use for a Learning Health System

Clinical Registry BRFSS Data CAHPS Data Corrections data

• Process data sets
• Check data quality
• Address data gaps
• Link data sets
• Analytics
• Reporting

Data extracts

Other?

• Utilization Measures
• Expenditure Measures
• Unit Costs
• Quality Measures
• Patient Experience Measures
• Comparative Evaluation
• Practice Profiles
• HSA Profiles
• PCMH + CHT Evaluation
• Opioid Program Evaluation
• Associations & Predictive Models
• Planning, Coordination, Quality
• Performance Payments

Provider Registry

HSR and Data Journalism

19

• As data are more available, the public and reporters who

work on their behalf will use them more

• Collaborate and maintain open dialogue to drive toward best

possible information for patients and providers

• When a reporter calls you for help with data and research:

▫ Be collaborative ▫ Be open to new ideas ▫ Be willing to share methods and ideas

20

Purpose of Measurement: Improve Quality & Safety

21

Helen Burstin, MD, MPH, FACP hburstin@qualityforum.org @HelenBurstin