The Internationally Educated Nurses Assessment Program (IENCAP) - - PowerPoint PPT Presentation

Presentation for

The Partners in Education & Integration of Internationally Educated Nurses: 8th National Conference, Toronto, 2014

Sten Ardal, CEO, CEHPEA

The Internationally Educated Nurses Assessment Program (IENCAP)

5/5/2014

2

About Half of all Registered Health Professionals are Nurses

Social Workers Respiratory Therapists Psychologists Physiotherapists Physicians (Excluding Residents) Pharmacists Optometrists Occupational Therapists Nurses Midwives Medical Radiation Technologists Medical Laboratory Technologists Dietitians Dentists Dental Hygienists Chiropractors Audiologists & Speech–Language Pathologists

1 10 100 1,000 10,000 100,000

* There are1.4 Million healthcare workers in Canada, with 500,000 working in Ontario (2006 Census)

Graph in log units, Showing 240,000 Regulated Health Professionals in Ontario (CIHI, 2010 Data)

5/5/2014

3

14,000 (11%) of Ontario Nurses Are Internationally Educated

3,858,455 720,430 29,000 749,430

Post Secondary Education in Canada Post Secondary Abroad IEHPs employed in Regulated Health Professions

In Ontario, 19% of all Postsecondary Graduates studied abroad 4% of whom are IEHPs working in Regulated Health Professionals

(2006 Census) (HealthForceOntario)

Who is CEHPEA?

• We are the only organization in Ontario mandated to provide

assessment, evaluation and training programs for internationally educated health professionals

• We have state-of-the-art examination and education facilities in

downtown Toronto and are the largest assessment centre in Canada

5/5/2014

4

Vision & Value

• Our Vision:

*To facilitate access to a health care career for internationally

educated health professionals who want to reside and practice in Ontario, through the provision of evaluation and orientation services.

• Our Value Proposition:

*Canada’s trusted Leader in professional competency assurance

regarding internationally trained health professionals, supporting the strength of the healthcare system by ensuring that only qualified professionals enter the pathway to practice.

5/5/2014

5

Our Facility



45,000 Square Feet



Central Location



78 Examination Rooms



Audio-Visual Control Room



Large Classrooms



Breakout Rooms

5/5/2014

6

Our Expertise



Test Construction



Assessment Design



Clinical Exam Administration



Psychometric Analysis



Standardized Patient Program



Educational Programming



E-learning

5/5/2014

7

5/5/2014

8

Our Programs

• Internationally Educated Nurse Competency Assessment
• Nurse Practitioner Practice Assessment
• International Medical Graduates
• National Assessment Collaborative Objective Structured Clinical

Examination (OSCE)

• Advanced Specialty and Practice Ready Assessment
• Pre-Residency Program (PRP)
• Orientation to Training and Practice in Ontario (OTPC)
• Canadian Medicine Primer
• Physician Assistant Program
• College of Optometry Evaluating Examination
• Communication Competency Program

5/5/2014

9

Our Participants’ World (2013)

IMG IMG & IEN Snapshot

What is IENCAP

• In 2010 the Ministry of Health and Long Term Care provided

funding for CEHPEA to work with the CNO to develop a competency assessment

• To expedite a registration process for qualified IENs
• To provide information for those requiring upgrading for specific

competencies

• Based on a test blueprint derived from the national document

Competencies in the Context of Entry-level Registered Nurse Practice

5/5/2014

10

5/5/2014

11

Assessment in 10 Steps

• 1. Identify candidates
• 2. Set outcomes & purpose (eq eligibility for licensure exam)
• 3. Specify competencies: knowledge, skills, attitudes
• 4. Blueprinting

Design assessment formats to map to competencies Set performance levels & scales

• 5. Write written or performance (OSCE) items
• 6. Validate each assessment

Expert reviews with criterion based tools, Field tests, pilots

• 7. Standard setting
• 8. Launch
• 9. Provide candidate and assessment performance reports

10.Evaluate iteratively and refine

5/5/2014

12

IENCAP Format: MCQ & OCSE

• The IEN MCQ
• Multiple choice questions that primarily address

competencies not amenable to the OSCE

• The IEN Clinical Examination
• The IEN clinical examination is presented in OSCE format.

The examination consists of 12 ten-minute stations (each station consists of a 7 minute client-encounter followed by 3 minutes of oral questions)

5/5/2014

13

136 Candidates Require Staff of 270!

Candidates 34% Examiners 19% Standardized Patients 23% Support Staff 20% CEHPEA Staff 4% Candidates Examiners Standardized Patients Support Staff CEHPEA Staff

5/5/2014

14

Competencies

• Competencies assessed in the client-encounters are:
• History taking and data collection
• Physical examination
• Communication skills
• Planning
• Competent Professional Practice
• Ethical practice
• Safe practice
• Collaboration with client(s) health care team members
• Evaluation: Effectiveness of interventions
• Competencies assessed in post-encounter oral questions are:
• Planning/Implementation
• Evaluations of care outcomes/Revised treatment/management plan

5/5/2014

15

Over 1,000 IENs Already Assessed & Over 1,200 Scheduled

82 138 137 136 131 134 140 132 213 214 216 215 216 159

50 100 150 200 250

IENCAP Administrations

2013 2014 2015 The CNO manages over 10,000 applications from Ontario graduates annually, and over 5,000 from IENs.

5/5/2014

16

Top 10 Countries of Study

Philippines India Iran Nigeria China Nepal United States Israel England Jamaica Other

N = 2,235 (Shown are > 30)

70 Different Countries

5/5/2014

17

IENCAP Candidates Average 34 Years of Age

100 200 300 400 500 600 700 20-24 25-29 30-34 40-44 45-49 50-54 55+

Age Distribution

5/5/2014

18

Half of IENCAP Candidates Graduated Less Than 10 Years Ago (Median = 9)

50 100 150 200 250 5 10 15 20 25 30 35 43

Years Since Graduation

6-10 Years = 42%

5/5/2014

19

Practice Competency Domain Percentage of Items Correct Professional Responsibility and Accountability 83.33% Specialized Body of Knowledge 70.00% Ethical Practice 66.67% Service to the Public 83.33% Professional Self-Regulation 66.67% Total Test 66.67% CEHPEA Multiple Choice Question (MCQ) Score Report Examination Date: March, 28th, 2014 Candidate/CNO Application ID: 0000000 Last Name: DAYTE First Name: CANDY

5/5/2014

20

RATING SCALE

5 Meets Expectations 4 Meets Expectations with Suggestions 3 Borderline 2 Below Expectations 1 Not Acceptable

CEHPEA OSCE REPORT CERTIFICATE OF GENERAL ASSESSMENT Examination Date: March, 28th, 2014 Candidate/CNO Application ID: 0000000 Last Name: DAYTE First Name: CANDY

OSCE ASSESSMENT SUMMARY Type of Score Total No. of Scores

• Min. Score

Level

• Max. Score

Level Median Score Level Total Test Score 12 1.50 4.30 3.00

5/5/2014

21

CNO Competency Categories

• No. of CNO

Competencies Covered

CEHPEA Components of Performance

• No. of

Stations Evaluated Min. Score Max. Score Median Score Level

Professional Responsibility & Accountability**

4

Competent Professional Practice

12 1 4 3

Knowledge-Based Practice: Competent Application of Knowledge Ongoing Holistic Assessment

9

History & Data Collection

12 1 4 2.5

Physical Examination

5 1 4 3

Communication Skills

12 1 4 3

Collaborates with Clients to Develop Health Care Plans

11

Planning · Clinical Reasoning · Incorporation of Relevant Knowledge · Organization & Prioritization

12 1 4 3

Planning/Implementation (PEP)

12 1 4 3

Provides Nursing Care with Clients

20

Competent Professional Practice

12 1 4 3

Safe Practice

12 2 5 3

Collaboration

12 1 4 3

Planning/Implementation (PEP)

12 1 4 2.5

Ongoing Evaluation of Client Care

5

Evaluation · Effectiveness of Interventions

12 2 5 2.5

Evaluation (PEP) · Evaluation of care

• utcomes

· Revised treatment/management plan

12 2 5 3

Ethical Practice**

12

Ethical Practice

12 1 4 3

Service to the Public**

7

Collaboration

12 1 4 3

5/5/2014

22

Key Weakness(es) Stn.1 Stn.2 Stn.3 Stn.4 Stn.5 Stn.6 Stn.7 Stn.8 Stn.9 Stn.10 Stn.11 Stn.12 Total Inadequate knowledge 1 1 1 3 Inadequate skill 1 1 1 3 Provided misinformation to client 1 1 Could not focus on client’s problem 1 1 2 Language ability – fluency Language ability – comprehension 1 1 1 3

5/5/2014

23

Examiner(s) Comments

Station Number Station Focus

Comments

(Sample of Stations) Inadequate Knowledge Inadequate Skill Provided misinformation to client Could not focus on client’s problems Language ability – fluency Language ability - comprehension Station 4 History & Physical

Example of comment here

Station 6 Ethical Dilemma

Example of comment here Example of comment here

Station 9 History & Physical

Example of comment here Example of comment here Example of comment here

5/5/2014

24

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Alpha Reliability

Meta analysis of 188 OSCEs: Average 𝜷 of 0.66 (Brannick et al, 2011)

IENCAP is Reliable and Consistent

5/5/2014

25

5/5/2014

26

*S.ardal@cehpea.ca *www.cehpea.ca