Is Online Assessment Safe ? Is Online Assessment Safe ? Our Journey - PDF document

Is Online Assessment Safe ? Is Online Assessment Safe ? Our Journey • SMU started to have controlled environment for Online Exams in 2005 • What we have tried ? • 2005 ‐ OS, IP Address, User Profile • High Manual Effort 2006 – 3 rd Party Software like Software Secure • • High maintenance effort • Frequent testing whenever computing environment changes • Clashes with Anti ‐ Virus software • Slow technical support from overseas • 2007 – Self Made Exam Browser Integrated Information Technology Services (IITS) Jun 2011 1 Is Online Assessment Safe ? Benefits of Online Assessment • Easier to Setup Quiz & Changes • No printing/re-printing • Select questions based on categories and question sets • Keep track of Questions Usage • Easy to tell what questions have been used • Easy to shuffle questions and MCQ options • Harder for students to copy even if they sit close to each other • Less Marking • MCQ, T/F, Fill in the Blanks, Picture Selection auto-marking, only need to mark essays, plus do not need to deal with horrible handwriting • Quick Analysis and Structured Report • % of right and wrong, easy to tell strong and weak areas Integrated Information Technology Services (IITS) Jun 2011 2 Is Online Assessment Safe ? Our Users’ Requests • Closed Book • Block all access to Course Materials, Internet, Communications • Semi ‐ Open Book • Block access to Internet and Communications but not Course Materials • Open Book • Block communications but not others Communications (Email, Chat, Social Media and … etc) Integrated Information Technology Services (IITS) Jun 2011 3 eLearning Forum Asia 2011, NTU Singapore 1

Is Online Assessment Safe ? Is Online Assessment Safe ? What we have considered ? • Wireless Signal Jammer • License, public network (Wireless@SG), 3G • No precise control • Not portable • Controlled Computer Lab • Not scalable • Respondus Lockdown Browser • Only cater to Quiz and cannot overcome network connection break Integrated Information Technology Services (IITS) Jun 2011 4 Is Online Assessment Safe ? Challenges we face • New software can break the controls • Korean Toolbar can allow pop-ups in our Exam Browser • French OS can achieve Copy and Paste • Exam Browser need maintenance along the way • New Browser or Browser Version or Add-ons/Toolbars • New OS • Mac Laptop without Bootcamp • Some laptops just cannot have this client software Integrated Information Technology Services (IITS) Jun 2011 5 Is Online Assessment Safe ? Actions we take for Online Exams • Seating arrangement to position the ‘high risk’ group in front of instructor • Randomized the questions and/or answers • Invigilator is still needed, we provide easy way to check for Exam Browser • Change Password after exam started • Prevent re-entry if student close the Exam Browser and switch to normal browser Integrated Information Technology Services (IITS) Jun 2011 6 eLearning Forum Asia 2011, NTU Singapore 2

Is Online Assessment Safe ? Our Exam Browser Is Online Assessment Safe ? Special Features in our Exam Browser • Auto-Update to latest version • Easy to deploy latest changes • Auto-Save every 15 secs • Ensure answers not lost when network connection breaks • Ability to Backup and Restore student answers in Quiz • Administrator overwrite control to retrieve answers when necessary • Detect Virtual Machines, Remote Computer & Desktop Integrated Information Technology Services (IITS) Jun 2011 8 Is Online Assessment Safe ? How we support Exam Browser ? • Intranet download link - Users Self Help • Conduct a Mock Exam before Actual Exam • Manage expectations • Minimize support effort on actual day • One technical engineer per room or 2 rooms (Ratio 1 : 50 or 100) • Support starts 30 minutes before exam • Last minute software update, troubleshooting • Briefing to students and invigilator • Support extends 30 mins after exam • Check through submissions, troubleshooting Integrated Information Technology Services (IITS) Jun 2011 9 eLearning Forum Asia 2011, NTU Singapore 3