Electronic Standardize Testing for Advanced Practice Registered - - PowerPoint PPT Presentation

Electronic Standardize Testing for Advanced Practice Registered Nursing (APRN) Programs

Pamela Willson, PhD, RN, FNP, BC

Prairie View A&M University, Associate Professor Elsevier, Assessment HESI™, Director of Research

Objectives

 Summarize a standardized APRN end of program

examination’s predictive validity.

 Critique potential contextual implementation

strategies for including standardized examinations within APRN.

Background

 There are more than 158,000 practicing NPs.

Close to 9,203 new NPs are prepared each year at over 325 colleges and universities in the U.S.

 To assure a match between education and

clinical competencies, certification is a commonly legislated practice eligibility mandate.

 Graduate nursing faculty are implementing

standardize certification predictive testing, exit exam benchmarking, and remediation strategies to facilitate student success on national certification exams.

A major indicator of nursing programs’

effectiveness is the pass rate on national certification examinations among first-time candidates.

Twenty years of evidence supports use of

standardized examinations for undergraduate end of program assessment to facilitate knowledge development, ensure competence

• f their new graduates, and demonstrate
• rganizational and curriculum effectiveness.

Standardize examination use in graduate

education lags behind that of undergraduate education, providing limited evidence to guide faculty.

Purpose

 The aim of this study was to establish the

predictive validity of a standardized examination for APRN certification success in a national United States sample.

 To determine implementation strategies used

by their schools.

Evidence-based Testing Products

Should contain:

Reliable and valid test items Matched certification body test blueprints Established national benchmark or cutscore

Reliability

 Reliability Estimates—dynamically measured  Calculated for each HESI exam administered

 Item Difficulty  Item Discrimination

 Point Biserial Correlation Coefficient

 Kuder-Richardson 20 (KR-20)

Morrison, S., Adamson, C., Nibert, A., & Hsia, S. (2004) HESI Exams: An overview of reliability and validity. CIN: Computers, Informatics, Nursing. (22)4, 220-226.

Exam Blueprints

 HESI™ APRN Exam Blueprints

 HESI Standard Scoring Categories  NONPF Competencies  ANCC & AANP Exam Content Outlines  National Organization of Nurse Executives

 Exams Developed

 Family Nurse Practitioner  Adult Nurse Practitioner  Acute Care Nurse Practitioner*  Nursing Executive*  3Ps – Patho/Pharm/Physical Diagnosis*

* Piloting Exam

HESI APRN Exam

 Reliabilities and Average Item Uses

Family NP Exams KR-20 ~ 0.932 Item uses = ranges from 466 to 2394

 HESI Recommended Benchmarks

Recommended Score - 800 and above Minimally Acceptable Score - 750

Methods

 Electronic surveys were mailed to

Directors/Chairs of 35 graduate nursing schools that administered Elsevier HESI™ Advanced Practice Registered Nurse (APRN) Exam for Family and/or Adult APRN students during the 2008–2009 academic year.

 Data were collected regarding students’ APRN

national certification examination outcomes and the schools’ implementation strategies and policies.

Methods

 After obtaining Institutional Review Board

approval, a cohort of students was invited to participate in testing and report review.

 The Elsevier HESI Family Nurse Practitioner

Exam (FNP) consisting of 110 items was taken electronically and individualized score reports were printed at the conclusion of the exam.

 Faculty reviewed exam outcomes and reports

with the participants and action plans were developed.

Instruments

Elsevier HESI™ APRN FNP 100-item APRN Comprehensive Exams

10 pilot items Matched to AANP and ANCC test blueprints Identify specific areas for remediation Provide an objective measure of program

• utcome achievement

Contribute to overall curriculum evaluation

Instruments

 Validity Study Questionnaire

 14-item

 Policy  Benchmarks  Consequences  Remediation  Use of Score reports  Satisfaction with tool

 Student & Program

Survey

 13-item survey

Implementation strategies

 Count for grade/percentage  Timing  Score report

Program policy Remediation

 Sent out to 35 programs

9 programs – 141 students

National Survey Results

 Program types

3 FNP 3 ANP 3 FNP & ANP

 Three schools count exam for 10% of grade  One school had an Exit Testing Policy

Benchmark was set at 800 Consequences

Results

Certification outcomes were reported for

141 APRN students

 96 Family APRN students  45 Adult APRN students

 APRN HESI™ scores

 Ranged from 484 to 1039  Mode of 805  Mean of 796  Standard deviation 111.4

Summary Scoring Report

Study Results

Elsevier HESI™ APRN FNP Exam Results

Descriptive Statistics

N Minimum Maximum Mean Std. Deviation 141 484 1039 796.09 113.390

Exam Scoring Categories

 Number & Percent of Students by Scoring Category

(N = 141)

Scoring Category HESI Score N Percent Category A/B 900->1000+ 22 16% Category C 850-899 23 16% Category D 800-849 30 21% Category E/F 700-799 39 27% Category G/H < 699 28 20%

HESI™ Scores & Certification

 Number & Percent of Students by Scoring Category

(N = 141)

Scoring Category HESI Score N Predictive Rate Category A/B 900->1000+ 22 100% Category C 850-899 23 100% Category D 800-849 30 100% Category E/F 700-799 38 100% Category G/H < 699 28 56%

Identify Learning Needs

Competency Outcomes

Standardize APRN predictive exams provided valuable student competency data that allow faculty to address student knowledge gaps with remediation and clinical experiences.

Course & Program Evaluation

 In addition, standardize testing provides outcome

measures for curriculum evaluation, program accreditation, and program policy development.

Program Evaluation

Use to:

 Document the evidence for decision making  Track and Trend outcomes over several

cohorts of same and/or different specialty tracks

 Adds richness of details at the program and

course level for quality assurance and accreditation bodies

Summary

 A national multi-site United States sample of

graduate programs has found that electronic standardized end of program assessments are highly accurate at predicting APRN certification success.

 These findings are consistent with preliminary

benchmark setting studies for APRN HESI™ minimum scores of 750 or higher. Faculty’s implementation strategies, policy development, and methods for curricular evaluation were discussed.