Regulatory Quality Indicators: A Delphi Study Nancy Spector, PhD, - - PowerPoint PPT Presentation

Nancy Spector, PhD, RN, FAAN Director, Regulatory Innovations ATI National Nurse Educator Summit April 1 & 2, 2019, Savannah Georgia

Consensus on Nursing Education Regulatory Quality Indicators: A Delphi Study

The National Council of State Boards of Nursing (NCSBN) provides education, service and research through collaborative leadership to promote evidence-based regulatory excellence for patient safety and public protection.

Background of the study

• Most BONs approve nursing education

programs.

• BONs are asking for legally defensible

metrics to measure, other than first-time NCLEX pass rates.

• Results of this study will be analyzed

with those from a larger, quantitative study.

Background of the study

Charge: Establish a set of outcomes and associated metrics to recommend processes to assess nursing education programs.

• Review current literature on program approval

metrics and their relevance to public safety.

• Recommend factors in addition to first time

NCLEX pass rates that can be used to determine criteria for a legally defensible Board of Nursing’s approval/removal process.

Literature

Outcomes:

• 1. Licensure pass rates (accreditors; USDE)
• 2. Practice readiness* (AAMC, 2014; Benner et al.,

2010; Berkow et al., 2008; Dickison et al., 2019; Hayden et al., 2014; Kavanagh & Szweda, 2017; Spector et al., 2015) [*Definition: newly licensed nurses able to deliver consistent, competent and safe care in predictable situations, with guidance for more complex situations.]

Literature

Outcomes (continued): 3. Employment rates (accreditors; Feeg & Mancino, 2016; Matsudaira, 2016) 4. Graduation/retention/persistence rates (Cook & Hartle, 2011; HERI, 2011; Matsudaira, 2016;Randolph, 2013)

Literature

RQIs:

• 1. Quality clinical experiences (Benner et al.,

2010; Hungerford, 2019; Kavanagh & Szweda, 2017; NCSBN, 2006; Odom-Maryon et al., 2018).

• 2. Student/faculty/program characteristics

(Alexander, 2019; Benner et al., 2010; Hooper & Ayars, 2017; Libner & Kubela, 2017; Odom- Maryon et al., 2018; Pitt et al., 2012).

• 3. National nursing accreditation (Cuff & Perez,

2017; Spector et al., 2018; WHO, 2013)

Literature

Warning signs – opposite of RQIs 1. Lack of curricular rigor (Hooper & Ayars, 2017) 2. High number of complaints (Alexander, 2019) 3. Insufficient qualified faculty (Alexander, 2019; Hooper & Ayars, 2017) 4. Unwillingness to fail unsafe students (Killam et al., 2011; Luhanga et al., 2014)

Study Questions

• 1. What are the characteristics of nursing

programs that graduate safe and competent nurses (RQIs)?

• 2. What are the warning signs when a program is

beginning to fall below standards?

• 3. What are the outcome measures used to

determine if a program is graduating safe and competent students?

Method

Delphi approach:

• Developed during 1950s cold war – RAND

Corporation.

• Assumption: Group opinion more valid than.
• Used successfully for policy and education

questions.

Method

• Round I – Qualitative responses
• Round II – Rating the factors
• Rounds III & IV – Consensus on those items

where respondents disagree

Method

Sample selection:

• 1. Education consultants from BONs
• 2. Educators (list from NCLEX)
• 3. Clinical nurse educators (list from the

Association for Nursing Professional Development)

Method

Inclusion Criteria:

• 1. Education Consultants – selected all
• 2. Educators – taught students for the last 2

years; PN educator: BSN; RN educator: master’s

• 3. Clinical nurse educators – work with new

graduates

Method

Procedure:

• 1. IRB – exempted;
• 2. Piloted questions with n=10 education

consultants, educators, clinical nurse educators.

Method

Procedure: 3. Introductory email sent to sample, via Qualtrics (Utah), inviting to participate if they met the criteria; 4. If they were willing to take part and met criteria, clicked into a demographics survey. 5. Defined consensus as 67% agreement, with interquartile range of 1.0 or below.

Round I

• 1. What are the characteristics/quality

indicators of nursing education programs that graduate safe and competent nurses?

• 2. What are red flags (warning signs) that

indicate a program is falling below the standard of graduating safe and competent nurses?

• 3. What outcome measures could BONs use to

determine if nursing programs are graduating safe and competent nurses?

Round I Analysis

Content analysis carried out three ways for validation: 1) By hand; 2) Use of NVivo software; 3) Use of R, with Latent Dirichlet Allocation (LDA).

Round II

Sent out responses (content analyzed) from Round I, asking for RQIs, red flags (warning signs) and outcomes:

• How important are the RQIs (or red flags or
• utcomes)?
• Rating from 1=unimportant to 4=very

important

Round II Analysis

Statistical analysis conducted:

• SPSS (version 22.0) – simple descriptive

statistics, looking for percent who agreed that an item was “important” or “very important.”

• Means and SD calculated; Medians and IQR

calculated.

• One-way analysis of variance for differences

between the groups on ratings.

Results

Sample: 174/293 educators – 59% response rate 71/125 clinical educators – 57% response rate 50/62 education consultants – 81% response rate

Results

Demographics: Educators – 93% were female; 72% aged 55 or

• lder; 56% had doctorates; 95% taught for more

than 5 years. Education consultants – 96% were female; 72% aged 55 or older; 50% had doctorates; 54% had more than 5 years experience in regulation.

Results

Demographics: Clinical nurse educators – 96% female; 48% aged 55 or older; 19% had doctorates; 73% had more than 5 years experience.

Results

• Agreement ranges from 78 – 100%
• None of the ratings had a median rating below

3 (important)

• Interquartile ranges from 0-1

Round III not necessary!

RQIs

Evidence-based curriculum that emphasizes quality and safety standards for patient care – 100% Evidence-based curriculum that emphasizes critical thinking and clinical reasoning skills – 99.3% Faculty are able to role model professional behaviors – 99.3% Clinical experiences with actual patients that prepare students for the reality of clinical practice – 98.7% Program has a systematic process in place to address and remediate student practice errors – 98.7% Faculty teaching clinical courses demonstrate current clinical competence – 98.7%

RQIs

Consistent administrative leadership in the nursing program – 98.3% Collaboration between education and practice to enhance readiness for practice – 97.7% Ongoing systematic evaluation of the nursing program – 97.7% Institutional administrative support of the nursing program – 97.3% Consistently has a pattern of NCLEX pass rates that meet set standards – 96.3% Administrative support for ongoing faculty development – 96.3%

RQIs

Significant opportunities for a variety of clinical experiences with diverse populations – 95.7% Consistent full-time faculty, as opposed to reliance on adjunct faculty – 95% Quality simulation is used to augment clinical experiences – 93.3% Comprehensive student support services – 93% Program has national nursing accreditation – 84% Admission criteria that emphasize a background in the sciences – 90%

Red Flags

Lack of consistent and prepared clinical faculty – 100% Limited clinical experiences that do not prepare the students for practice – 99% Poor leadership in the nursing program – 98.7% Trend of NCLEX pass rates is inconsistent or decreasing – 96.7% Complaints to the nursing program or board of nursing from employers, students or faculty – 94.7% Pattern of faculty attrition – 94%

Red Flags

Pattern of nursing program administrator attrition – 92.7% Unwillingness of healthcare institutions to host clinical experiences for the nursing program’s students – 92% Pattern of student attrition – 85% Curriculum is based on “teaching to the NCLEX” – 81.7% Over-reliance on simulation to replace clinical experiences with actual patients – 80%

Outcomes

NCLEX pass rates of the nursing program – 97.7% Relationship the nursing program has with its clinical partners – 97% Employer satisfaction with the graduates’ readiness for practice – 94.7% Graduate preparedness to practice for an interprofessional environment – 93.7%

Outcomes

Graduates’ satisfaction with the nursing program – 85.4% Graduation rates of students in the nursing program – 80.4% Consistency of graduate employment rates with regional data on nurse employment rates – 79.3% History of board of nursing discipline with the graduates of the nursing program – 78%

Discussion

Support for RQIs:

• 1. Quality clinical experiences (Benner et al.,

2010; NCSBN, 2006; Kavanagh & Szweda, 2017; Hungerford, 2019; Odom-Maryon et al., 2018;).

• 2. Administrative leadership and faculty support

(Alexander, 2019; Benner et al., 2010; Hooper & Ayars, 2017; Libner & Kubala, 2017).

• 3. Faculty ratios and qualifications (Libner &

Kubala, 2017; Odom-Maryon, 2018)

Discussion

RQIs (continued):

• 4. Evidence-based curriculum (Benner et al.,

2010; Dickison et al., 2019; QSEN, 2019; Rusch et al., 2019). Warning Signs (besides those that are the

• pposite of RQIs):
• 1. Unable to find clinical experiences (Hooper &

Ayars, 2017).

• 2. Complaints to the Board from students,

faculty, employers (Alexander, 2019).

• 3. Teach to the NCLEX (Candela & Bowles, 2007;

Kavanagh & Swzeda, 2017; Barrett et al., 2017).

Conclusions and Implications

Strong agreement among faculty, clinical nurse leaders and regulators on:

• 1. Quality, hands-on clinical experiences – maybe

it’s time to focus on faculty development for providing quality direct care experiences?

• 2. Meaningful collaboration between practice and

education is needed – early on in program planning.

Conclusions and Implications

• 3. Administrator consistency and leadership is

essential.

• 4. A well-prepared, consistent faculty is essential.

These results will be integrated into the larger, 5- Year Annual Report Study for evidence-based recommendations. Stay Tuned!

“Perfection is not attainable, but if we chase perfection we can catch excellence.”

• Vince Lombardi

Questions?

nspector@ncsbn.org 312-525-3657