Mysteries of the Real Estate Licensing Exam Mysteries of the Real - - PowerPoint PPT Presentation

Mysteries of the Real Estate Licensing Exam REEA Annual Conference June 2011

Tadas Dabsys, Vice President PS I S ervices LLC

Mysteries of the Real Estate Licensing Exam REEA Annual Conference June 2011

Tadas Dabsys, Vice President PS I S ervices LLC

2

ARELLO-Accredited Vendors

2

Applied Measurement Professionals

http://www.goamp.com

Pearson VUE

http://www.pearsonvue.com

PSI Services LLC

http://psionline.com

3

Benefits of ARELLO-Accreditation

3

Independent Evaluation Confidence and Protection Defensible Standards Reasonable Consistency in Quality Facilitates License Recognition www.ARELLO.org (ARELLO Guidelines for Accreditation)

4

Stakeholders

4

Applicants Public Regulators Educators Publishers Industry

5

Licensing Exam Purpose

5

Regulation Public Protection Competency Assurance Minimum Competence

6

Licensing Exam Development

6

Validity Reliability Fairness Legal Defensibility Standards

Standards for Educational and Psychological Testing

(AERA, APA, & NCME, 1999)

7

7

Subject Matter Experts

Qualifications Experience Training in Job Analysis

Test Development Experts

Psychometricians Item Writers

Credibility

8

8

Job Analysis Survey Procedure

Content Development Format Development

Sampling – Representative Sample Target Groups Sources for Addresses/Contact Information

Job Analysis

9

9

Demographic Information Professional Activities Knowledge

Rating and Respective Response Scales Summary Statistics Identification of Qualifying Professional Activities/Knowledge

Analysis of the Survey Results

10

10

Development of Content Specifications Establishing Content Base for Inclusion in Exams Linking Knowledge Areas to Professional Activities Development of Operational Definitions Test Outline and Content Weighting

Content Specifications

11

11

Activities and Tasks Subject/Content Area of the Task Knowledge Statement (KSAs)

Knowledge Skills Abilities

Linking Back to the Job Analysis

12

12

Based on Job Analysis and Content Specifications Written by Professionals Reviewed by Subject Matter Experts Checked Against References Sensitivity Review Continual Review and Updating Quality Assurance

Item Development

13

13

One Correct Answer Relevant and Appropriate Realistic Important to Related Task Limit Use of Negative Words Clear and Concise No Clues that Give Away the Answer Entry-Level Reading Level

Basic Item Writing Principles

14

14

Response Options:

Fit the Stem Logic Grammatically Equal in Length Unique Meaning Plausible - yet definitely incorrect Avoid “All the Above” and “None of the Above” Avoid Multiple Options

Basic Item Writing Principles

15

15

Balancing Cognitive Levels

Recall: Able to state the definition of a “latent defect.” Application: Applies knowledge of the definition of “latent defect” by recognizing and classifying certain described property conditions as meeting the definition of a latent defect. Analysis: Applies knowledge of the definition by recognizing certain described property conditions as meeting the definition and interpreting the information to determine what action should be made.

16

16

Sample Items

Recall - Terminology Item A land contract is also known as 1. an option. * 2. a contract for deed. 3. an executory contract. 4. an agency contract. Application - Practical Item A young couple have enough income to make monthly house payments easily, but they do not have a sufficient down payment to meet lender requirements. Which would their agent MOST LIKELY suggest? * 1. A land contract. 2. An assigned mortgage. 3. A reverse annuity mortgage. 4. A deed in lieu of foreclosure.

17

17

Test Question Review Examination Review Monitoring Web Sites Updating References REEA and ARELLO Satisfaction Surveys

Quality Assurance

18

18

p-value:

Percentage of candidates in a sample who answer an item correctly

pbis:

Correlation within a sample between performance

• n an item and performance on the overall exam
• ption stats: Percentage of candidates who answer each
• ption
• ption pbis:

Correlation between performance on the options and the overall exam

Item Statistics

19

Expertise in Content and Test Development

19

Producing Professional Results

Content Domain Job Analysis Questionnaire Committee Content Specifications Committee Review Empirical Results Item Development Relevancy Ratings Cutoff Score Test Form

20

20

Examination Education Candidate – Study Habits, Health Reference Material Environment Others

Variables Affecting Pass Rates

21

21

Candidate Score Reports

Diagnostics

State Reports

Partner Site

School Reports

Topic Level for Schools and State

Reporting

22

22

Internet Candidate Information Bulletin Content Outline Reference Materials Comments – During Exam ONLY Security Agreement – Confidentiality

Tips for Increased Potential for Success

23

23

I will not give or receive assistance while taking the test,

including the use of unauthorized notes.

I will maintain the confidentiality of the test. I will not have in my possession a cell phone, pager, or

• ther unauthorized materials

I understand that violating the confidential nature of the

licensing test can result in severe civil or criminal penalties, invalidation of test scores, reports to the Authorized Agency

Security Agreement

24

24

Suspicious Candidate Activity

Test for significant differences in pass rates Better performance on harder items High scores in low test times Suspicious improvement in scores

Forensic Data Analysis

25

Thank You Questions ? Comments !