A simple tool to 00100000 01001001 01101110 01110100 01100101 - - PowerPoint PPT Presentation

▶
a simple tool to
SMART_READER_LITE
LIVE PREVIEW

A simple tool to 00100000 01001001 01101110 01110100 01100101 - - PowerPoint PPT Presentation

Dec 24, 2022 276 likes •483 views

01110101 01101101 01110011 01100001 01101110 00100000 01101110 01100101 01110001 01110101 01100101 00100000 01101100 01101111 01110010 01100101 01101101 00100000 01110000 01110101 01101100 01110110 01101001 01101110 01100001 01110010 00100000

slide-1
SLIDE 1 01110101 01101101 01110011 01100001 01101110 00100000 01101110 01100101 01110001 01110101 01100101 00100000 01101100 01101111 01110010 01100101 01101101 00100000 01110000 01110101 01101100 01110110 01101001 01101110 01100001 01110010 00100000 01110100 01110101 01110010 01110000 01101001 01110011 00101110 00100000 01010000 01110010 01101111 01101001 01101110 00100000 01100001 00100000 01100011 01101111 01101110 01110110 01100001 01101100 01101100 01101001 01110011 00100000 01101100 01100101 01100011 01110100 01110101 01110011 00101110 00100000 01010110 01101001 01110110 01100001 01101101 01110101 01110011 00100000 01100001 01110101 01100011 01110100 01101111 01110010 00100000 01101100 01100101 01100011 01110100 01110101 01110011 00100000 01110101 01110100 00100000 01100101 01110011 01110100 00100000 01110000 01110010 01100101 01110100 01101001 01110101 01101101 00101100 00100000 01110110 01101001 01110100 01100001 01100101 00100000 01100011 01101111 01101110 01110011 01100101 01100011 01110100 01100101 01110100 01110101 01110010 00100000 01100101 01111000 00100000 01100001 01101100 01101001 01110001 01110101 01100001 01101101 00101110 00100000 01000001 01101100 01101001 01110001 01110101 01100001 01101101 00100000 01110100 01100101 01101101 01110000 01110101 01110011 00100000 01101110 01101111 01101110 00100000 01100101 01110010 01100001 01110100 00100000 01110001 01110101 01101001 01110011 00100000 01100110 01110010 01101001 01101110 01100111 01101001 01101100 01101100 01100001 00101110 00100000 01001001 01101110 01110100 01100101 01100111 01100101 01110010 00100000 01100001 01100011 00100000 01100101 01100111 01100101 01110011 01110100 01100001 01110011 00100000 01100101 01111000 00101100 00100000 01101001 01101110 00100000 01110110 01100101 01101000 01101001 01100011 01110101 01101100 01100001 00100000 01101101 01101001 00101110 00100000 01010001 01110101 01101001 01110011 01110001 01110101 01100101 00100000 01101101 01100001 01110100 01110100 01101001 01110011 00100000 01110011 01100011 01100101 01101100 01100101 01110010 01101001 01110011 01110001 01110101 01100101 00100000 01101101 01101001 00100000 01101110 01100101 01100011 00100000 01100011 01101111 01101110 01100100 01101001 01101101 01100101 01101110 01110100 01110101 01101101 00101110 00100000 01001101 01100001 01110101 01110010 01101001 01110011 00100000 01100110 01100101 01110101 01100111 01101001 01100001 01110100 00100000 01100001 01110010 01100011 01110101 00100000 01101110 01101111 01101110 00100000 01100100 01101001 01100001 01101101 00100000 01110010 01110101 01110100 01110010 01110101 01101101 00100000 01101100 01101111 01100010 01101111 01110010 01110100 01101001 01110011 00100000

A simple tool to increase confjdence in results

SALLY DUCKWORTH

Research Evaluation & Design Specialist

slide-2
SLIDE 2

Data quality is a concept that academics, scientists and IT specialists have been taking seriously for years.

slide-3
SLIDE 3

Data quality is the fjtness of data for an intended purpose.

Accurate Coherent Precise Complete Culturally responsive Ethical Valid Reliable Intuitive Depth

slide-4
SLIDE 4

Having quality data leads to confjdence in results.

slide-5
SLIDE 5

DATA

FINDINGS

CONCLUSION

RECOMMENDATIONS

And poor quality data is a house of cards

An expression dating back to 1645 meaning a structure or argument built on a shaky foundation that would collapse if a necessary but overlooked element is removed.

slide-6
SLIDE 6

The main way

  • f making data

quality explicit in formal evaluation products is by stating limitations.

slide-7
SLIDE 7

Limitations are

  • ccurrences that

the evaluator did not or could not control.

slide-8
SLIDE 8

Acknowledging limitations should not be seen as a weakness of the evaluator, but rather a signifjer of credibility. They are also an opportunity for refmective learning.

slide-9
SLIDE 9

Best practice for acknowledging limitations points to 3 things:

  • 1. Identifying limitations and their importance
  • 2. Discussing constraints
  • 3. Recommending further research and data collection
slide-10
SLIDE 10

We did a stocktake of evaluation reports and found that 28% did not refer to limitations

25

18 17 7 7

Evaluation reports published by 6 state service organisations in 2015 72% discussed data limitations 68% talked about how limitations constrain fjndings 28% recommended further research 28% did not make comments

slide-11
SLIDE 11

Then we asked senior evaluators* 3 questions:

  • 1. How important is the quality of data used in

evaluations?

  • 2. How does your organisation determine the quality
  • f data used in evaluations?
  • 3. Does your organisation have standards for

reporting the quality of data in evaluation products?

*6 in 4 state sector organisations

slide-12
SLIDE 12

We heard that data quality is important when conducting evaluation. And people have different expectations

  • f data quality

depending on the scope, size, duration and location of an evaluation.

$

slide-13
SLIDE 13

We also heard that data quality is determined by having trust and confjdence that evaluators are following ‘good professional practice.’

Sense checking Peer review Triangulation Response rates Co-interviewing Co-coding Highly qualifjed people Teamwork Completeness

slide-14
SLIDE 14

We learnt that organisations do not have standards for stating data quality but are more explicit (risk averse) when publishing externally.

slide-15
SLIDE 15

We found that limitations slide off accessible and real time products such as A3’s, correspondence and progress reports.

slide-16
SLIDE 16

Decision makers and other stakeholders are wanting things instantly and digestible. We can provide a one or two page thing, but you lose the nuances and limitations.

‘ ’

slide-17
SLIDE 17

Finally we learnt that limitations are seen as weakness of the evaluator or evaluation, rather than enhancing credibility or confjdence. We need to fmip this around and start talking about and celebrating good data quality to enhance the confjdence

  • f results.

C R E D I B I L I T Y

slide-18
SLIDE 18

A simple tool to increase the confjdence in results is to:

1

State the quality

  • f your data

DATA SOURCE QUALITY RATING COMMENT ON QUALITY

LITERATURE REVIEW HIGH QUALITATIVE INTERVIEW HIGH STOCKTAKE MEDIUM

Q U A L I T Y

slide-19
SLIDE 19

A simple tool to increase the confjdence in results is to:

1 2 3

State the quality

  • f your data

...Adjust it so it fjts the context ...And apply it to all evaluation products

DATA SOURCE QUALITY RATING COMMENT ON QUALITY LITERATURE REVIEW HIGH QUALITATIVE INTERVIEW HIGH STOCKTAKE MEDIUM

Q U A L I T Y

Q U A L I T Y Q U A L I T Y

slide-20
SLIDE 20

Data quality... ...let’s start a conversation

@LitmusNz #ANZEA2016 #dataqual