Sharing and Comparing: Best Practices from Education & - PowerPoint PPT Presentation

Sharing and Comparing: Best Practices from Education & Credentialing Contexts Tony Alpert, SBAC Wayne Camara, ACT Gregory J. Cizek, UNC Liberty Munson, Microsoft Jamie Mulkey (moderator), Caveon

Why share?  Best practices that should be considered  Good to get outside of your domain  Helps to understand different needs for different environments  We all have the same goal in mind

Today’s session  Format  Business Drivers  Achievement Levels  Alignment  Test Length  Test Security

Business Drivers Liberty J. Munson

Learning for the Modern Era Enabling the next generation of innovators, entrepreneurs and developers 1 3 4 5 Creative Coding through Learning Credits Machine Learning Total reach Games and Apps Badges Microsoft Azure Harvard/MSFT CS50.AP Fundamentals Microsoft Professional New/Competitive Developers reached C# for Absolute Beginners Degree Programming in C# Hour of Code Skilled technologists building on Microsoft MOC On Demand platforms and services Minecraft 2 Preference for Microsoft platforms & services Spark Excitement Drive Deep Engagement Enable Capability Mastery Affinity Adoption Advocacy Empower Students Activate Cloud & Enable Customers Drive Usage, Deployment & Educators & Consumption Productivity Partners & Partners at Scale

An Overview of MS Certification Exams Variety of item types (e.g., multiple choice, Test center drag and drop, Computer Global deliveries or Ongoing delivery active screen, administered distribution online proctoring hot area, case studies, labs, text entry, code analysis, etc.)

Microsoft’s Certification Structure Certifications will include The five new expert To earn each of these achievement date that signifies certifications are: credentials: candidates investment in staying up to date on the technology • MCSE: Cloud Platform and • Earn Microsoft Certified • Every year, re-earn the Infrastructure (Windows Server Solutions Associate (MCSA) certification by passing an and Microsoft Azure) certification additional exam from the list of electives or retaking an exam • MCSE: Mobility – (Windows • Pass a single additional exam on a rapidly changing Client and Enterprise Mobility from a list of electives technology Suite) • Purpose for “recertification”? • MCSE: Data Management and Analysis (on-premises and • Breadth cloud-based Microsoft data • Continual competence products and services) • Engagement • MCSE: Productivity (Office 365, SharePoint, Exchange, and Skype for Business) • MCSD: App Builder (Web and Mobile app development)

Microsoft Learning Experiences Drive adoption of Microsoft technologies through training and certification

Cloud Platform & Infrastructure Path 410: 411: 412: MCSA Windows Installing and Configuring Administering Configuring Advanced Server 2012 Windows Server 2012 Windows Server 2012 Windows Server 2012 Services 740: 742: 741: MCSA Windows Installation, Storage, and Identity with MCSE Networking with Server 2016 Compute with Windows Server 2016 CLOUD PLATFORM & INFRASTRUCTURE Windows Server 2016 Cloud Platform & Infrastructure Windows Server 2016 Earned: 2017 Elective MCSA Linux on 533: Managing LFCS: Azure Microsoft Azure Linux Foundation Infrastructure Solutions Certified System Administrator Choose two from: MCSA Cloud 532: Developing Microsoft Azure Solutions Platform 533: Managing Microsoft Azure Infrastructure Solutions 534: Architecting Microsoft Azure Solutions 473: Designing and Implementing Cloud Data Platform Solutions 475: Designing and Implementing Big Data Analytics Solutions

Certification & Licensure  Purpose of certification/license  Practice tests, preparation materials  Protect public  Align to training?  Drive adoption  Accreditation  High, medium, low stakes  Security  Business goals  Braindump sites, proxy testers,  Reach, revenue, strategic cheating  Drive engagement  Proctoring  Development and sustainment costs  Identity verification  Balancing psychometric needs with business reality  Scoring  Global vs local distribution  Immediate vs delayed  Exam delivery providers (EDPs)  Role of marketing  Exam availability  Value of certification to candidate  Any time, any place delivery and employer  Online proctoring options  Competitive space

Business Drivers Wayne Camara

Business Drivers - Educational Assessments B2B – Large scale state testing B2C (SAT, ACT, GRE, etc.)  Cost to state/district  Access – ability to take test at convenience  Alignment to standards  Demand by external agency  Instructional sensitivity (college, scholarship service, (subscores & diagnostics) NCAA)  Growth over time (across  Security, turnaround time, grades) repeat testing  Turnaround time  Practice test – How to  Administrative flexibility – at improve total score school level, state  Standardization, fairness accommodations  Shorter testing time  Opt-out  Shorter testing time.

Business – Drivers in Digital vs paper B2C (SAT , ACT , GRE, etc.) B2B – Large scale state testing High security – organized efforts to expose content Low cost   High priority on standardization High priority for flexibility support (long   windows, scheduling, retesting) Global vs state specific  Multiple devices (Chromebook, tablets,  Large item banks and rotating forms  laptops) and comparability Access to test – local administration, local school  In school testing  Saturday administration (Summer date?)  Innovative item types – performance tasks  Paper and Digital versions existing side by side for  Released items and pools for practice  a long time with strict score comparability Rapid score turnaround vs testing late in  Device and mode should be no factor in score  school year Fee waivers – 15-18% drive costs  Reduced testing time  Demand for comparability prevents some  Performance tasks, TEI, highly desirable  innovation – modularity, TEI, innovation, shorter adaptive testing

Business Drivers - Educational vs Credentialing Business model Education: more customer sensitivity Credentialing: less competition and are often monopolies Cost Education: high sensitivity Credentialing: less sensitivity Test site Education: prioritizes local administration Credentialing: Use test center model, requiring travel greater travel/less convenience is a given Purpose/Use Education: Multiple Credentialing: Single Reporting Education: Student, class, teacher, school, state Credentialing: test taker Security B2B Education: Site, score, growth focus Certification: Individual, item bank, item pool Education: administrative flexibility, connectivity, device familiarity, innovative item types Digital issues Credentialing: standardization, single device, items which best represent construct (vs innovation for the sake of innovation)

What Credentialing Can Learn for Educational Business Drivers  Customer focus – more score information to help test taker improve, identify strengths and weaknesses (GMAT or LSAC vs GRE)  Security focused on score integrity vs IP  Cost containment – value add of enhancements vs cost to test taker  Fee waivers – when credential is a barrier to employment or advancement?  Transparency – data (subgroup, volumes, validity, reliability)  Investment in assessment – new forms, items  Test prep and practice materials – free and accessible

Achievement Levels Tony Alpert

17 Achievement Levels  Provide meaning and consequence to the scale  Build consensus regarding the requisite knowledge and skills that students are required to master  Shine a spotlight on inequities in the education system  Support state and federal accountability systems  Support state and federal retention laws (e.g. grade and ELL)

18 Achievement Levels  Involves Expert Judgement  Educators who understand the grade level content  Educators that are knowledgeable re: the diversity of students who take the test  Educators who who are knowledgeable re: grades either above or below  Build consensus regarding the content that is assessed

19 Achievement Levels  Moderated by:  Test format  Politics  Consequences of false positives and false negatives  Cost and logistics of true positives and true negatives  Resources available  For instructional days  Professional learning  Textbooks and curriculum supports  Test administration windows

Achievement Levels Wayne Camara

Achievement Levels EDUCATION CERTIFICATION EXCEEDS – What you can do MEETS – What you can do PASS APPROACHES – What you can do FAIL PARIALLY MEETS – What you can do EXCEEDS – MATH ALGEBRA MEETS – MATH ALGEBRA Algebraically solves linear Algebraically solves linear equations, linear inequalities and equations, linear inequalities and quadratics in one variable (at quadratics in one variable (at complexity appropriate to the complexity appropriate to the course), including those with course). coefficients represented by letters. Utilizes structure and rewriting as 21 strategies for solving.