The Internationally Educated Nurses Assessment Program (IENCAP) - PowerPoint PPT Presentation

The Internationally Educated Nurses Assessment Program (IENCAP) Presentation for The Partners in Education & Integration of Internationally Educated Nurses: 8th National Conference, Toronto, 2014 Sten Ardal, CEO, CEHPEA

About Half of all Registered Health Professionals are Nurses * There are1.4 Million healthcare workers in Canada, with 500,000 working in Ontario (2006 Census) Audiologists & Speech – Language Pathologists Chiropractors Dental Hygienists Dentists Dietitians Medical Laboratory Technologists Medical Radiation Technologists Midwives Nurses Occupational Therapists Optometrists Pharmacists Physicians (Excluding Residents) Physiotherapists Psychologists Respiratory Therapists Graph in log units, Social Workers Showing 240,000 Regulated Health Professionals 1 10 100 1,000 10,000 100,000 in Ontario (CIHI, 2010 Data) 2 5/5/2014

14,000 (11%) of Ontario Nurses Are Internationally Educated In Ontario, 19% of all Postsecondary Graduates studied abroad 4% of whom are IEHPs working in Regulated Health Professionals Post Secondary Education 3,858,455 in Canada 720,430 Post Secondary Abroad 749,430 (2006 Census) IEHPs employed in Regulated Health Professions (HealthForceOntario) 29,000 3 5/5/2014

Who is CEHPEA?  We are the only organization in Ontario mandated to provide assessment, evaluation and training programs for internationally educated health professionals  We have state-of-the-art examination and education facilities in downtown Toronto and are the largest assessment centre in Canada 4 5/5/2014

Vision & Value  Our Vision: * To facilitate access to a health care career for internationally educated health professionals who want to reside and practice in Ontario, through the provision of evaluation and orientation services.  Our Value Proposition: * Canada’s trusted Leader in professional competency assurance regarding internationally trained health professionals, supporting the strength of the healthcare system by ensuring that only qualified professionals enter the pathway to practice. 5 5/5/2014

Our Facility  45,000 Square Feet  Central Location  78 Examination Rooms  Audio-Visual Control Room  Large Classrooms  Breakout Rooms 6 5/5/2014

Our Expertise  Test Construction  Assessment Design  Clinical Exam Administration  Psychometric Analysis  Standardized Patient Program  Educational Programming  E-learning 7 5/5/2014

Our Programs  Internationally Educated Nurse Competency Assessment  Nurse Practitioner Practice Assessment  International Medical Graduates • National Assessment Collaborative Objective Structured Clinical Examination (OSCE) • Advanced Specialty and Practice Ready Assessment • Pre-Residency Program (PRP) • Orientation to Training and Practice in Ontario (OTPC) • Canadian Medicine Primer  Physician Assistant Program  College of Optometry Evaluating Examination  Communication Competency Program 8 5/5/2014

Snapshot Our Participants’ World (2013) IMG IMG & IEN 9 5/5/2014

What is IENCAP  In 2010 the Ministry of Health and Long Term Care provided funding for CEHPEA to work with the CNO to develop a competency assessment  To expedite a registration process for qualified IENs  To provide information for those requiring upgrading for specific competencies  Based on a test blueprint derived from the national document Competencies in the Context of Entry-level Registered Nurse Practice 10 5/5/2014

Assessment in 10 Steps 1. Identify candidates 2. Set outcomes & purpose (eq eligibility for licensure exam) 3. Specify competencies: knowledge, skills, attitudes 4. Blueprinting  Design assessment formats to map to competencies  Set performance levels & scales 5. Write written or performance (OSCE) items 6. Validate each assessment  Expert reviews with criterion based tools,  Field tests, pilots 7. Standard setting 8. Launch 9. Provide candidate and assessment performance reports 10. Evaluate iteratively and refine 11 5/5/2014

IENCAP Format: MCQ & OCSE  The IEN MCQ  Multiple choice questions that primarily address competencies not amenable to the OSCE  The IEN Clinical Examination  The IEN clinical examination is presented in OSCE format. The examination consists of 12 ten-minute stations (each station consists of a 7 minute client-encounter followed by 3 minutes of oral questions) 12 5/5/2014

136 Candidates Require Staff of 270! CEHPEA Staff 4% Candidates Candidates Support Staff 34% Examiners 20% Standardized Patients Standardized Support Staff Patients CEHPEA Staff 23% Examiners 19% 13 5/5/2014

Competencies  Competencies assessed in the client-encounters are: • History taking and data collection • Physical examination • Communication skills • Planning • Competent Professional Practice • Ethical practice • Safe practice • Collaboration with client(s) health care team members • Evaluation: Effectiveness of interventions  Competencies assessed in post-encounter oral questions are: • Planning/Implementation • Evaluations of care outcomes/Revised treatment/management plan 14 5/5/2014

Over 1,000 IENs Already Assessed & Over 1,200 Scheduled IENCAP Administrations 2014 2013 2015 250 216 215 216 214 213 200 159 150 140 138 137 136 134 132 131 100 82 50 0 The CNO manages over 10,000 applications from Ontario graduates annually, and over 5,000 from IENs. 15 5/5/2014

Top 10 Countries of Study N = 2,235 (Shown are > 30) Jamaica 70 Different England Other Countries United States Israel Nepal China Philippines Nigeria Iran India 16 5/5/2014

IENCAP Candidates Average 34 Years of Age Age Distribution 700 600 500 400 300 200 100 0 20-24 25-29 30-34 40-44 45-49 50-54 55+ 17 5/5/2014

Half of IENCAP Candidates Graduated Less Than 10 Years Ago (Median = 9) Years Since Graduation 6-10 Years = 42% 250 200 150 100 50 0 0 5 10 15 20 25 30 35 43 18 5/5/2014

CEHPEA Multiple Choice Question (MCQ) Score Report Examination Date: March, 28 th , 2014 Candidate/CNO Application ID: 0000000 Last Name: DAYTE First Name: CANDY Percentage of Practice Competency Domain Items Correct Professional Responsibility and Accountability 83.33% Specialized Body of Knowledge 70.00% Ethical Practice 66.67% Service to the Public 83.33% Professional Self-Regulation 66.67% Total Test 66.67% 19 5/5/2014

CEHPEA OSCE REPORT CERTIFICATE OF GENERAL ASSESSMENT Examination Date: March, 28 th , 2014 Candidate/CNO Application ID: 0000000 Last Name: DAYTE First Name: CANDY RATING SCALE 5 4 3 2 1 Meets Meets Expectations Borderline Below Not Expectations with Suggestions Expectations Acceptable OSCE ASSESSMENT SUMMARY Total No. of Min. Score Max. Score Median Score Level Type of Score Scores Level Level Total Test Score 12 1.50 4.30 3.00 20 5/5/2014

No. of CNO Evaluated Stations No. of Score Min. Score Max. Level Score Median CNO Competency Categories CEHPEA Components of Competencies Covered Performance Professional Responsibility Competent Professional 4 12 1 4 3 & Accountability** Practice History & Data Collection 12 1 4 2.5 Ongoing Holistic 9 Physical Examination 5 1 4 3 Assessment Communication Skills 12 1 4 3 Planning · Clinical Reasoning · Incorporation of Collaborates with 12 1 4 3 Relevant Knowledge Clients to 11 · Organization & Develop Health Prioritization Care Plans Planning/Implementation 12 1 4 3 (PEP) Knowledge-Based Competent Professional 12 1 4 3 Practice: Competent Practice Application of Safe Practice 12 2 5 3 Provides Nursing 20 Knowledge Care with Clients Collaboration 12 1 4 3 Planning/Implementation 12 1 4 2.5 (PEP) Evaluation · Effectiveness of 12 2 5 2.5 Interventions Ongoing Evaluation (PEP) 5 · Evaluation of care Evaluation of outcomes Client Care 12 2 5 3 · Revised treatment/management plan Ethical Practice** Ethical Practice 12 12 1 4 3 Service to the Public** 7 Collaboration 12 1 4 3 21 5/5/2014

Key Weakness(es) Stn.1 Stn.2 Stn.3 Stn.4 Stn.5 Stn.6 Stn.7 Stn.8 Stn.9 Stn.10 Stn.11 Stn.12 Total Inadequate knowledge 0 0 0 1 0 1 0 0 1 0 0 0 3 Inadequate skill 0 0 1 0 1 0 0 0 1 0 0 0 3 Provided misinformation to 0 0 0 0 0 1 0 0 0 0 0 0 1 client Could not focus on client ’ s 0 0 1 0 0 0 0 0 0 0 0 1 2 problem Language ability – fluency 0 0 0 0 0 0 0 0 0 0 0 0 0 Language ability – 0 0 0 0 0 0 1 1 1 0 0 0 3 comprehension 22 5/5/2014

Examiner(s) Comments Station Station Comments Number Focus Inadequate Inadequate Skill Provided Could not Language Language ability - Knowledge misinformation focus on ability – comprehension (Sample of Stations) to client client’s fluency problems Example of History & comment here Station 4 Physical Example of Example of comment here comment here Ethical Station 6 Dilemma Example of Example of Example of comment here comment here comment here History & Station 9 Physical 23 5/5/2014

IENCAP is Reliable and Consistent Alpha Reliability Meta analysis of 188 OSCEs: Average 𝜷 of 0.66 1 (Brannick et al, 2011) 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 24 5/5/2014

25 5/5/2014

* S.ardal@cehpea.ca * www.cehpea.ca 26 5/5/2014