Educators & The Judiciary History Center: A Usability Study - PowerPoint PPT Presentation

Educators & The Judiciary History Center: A Usability Study Methodology Introduction Discussion Reginald (Keahe) Davis Literature Problem Results The University of Hawai ʻ i at M ā noa davisr@hawaii.edu www.jhchawaii.net https://keahe6.wixsite.com/jhctestsite

ALOHA K Ā KOU Methodology Introduction Discussion Problem Literature Keahe Davis Results • M.Ed. – Learning Design & Technology (LTEC) candidate at the University of Hawai ʻ i at M ā noa • Education Director - King Kamehameha V Judiciary History Center • I oversee tour programs for the Hawai ʻ i State Supreme Court and First Circuit Court of Hawai ʻ i • I create and coordinate professional development programs for Social Studies teachers • I act as administrator for the Center’s website

Discussion Results Methodology Literature Problem Ali ʻ i ō lani Hale Introduction

School Field Trips They enhance classroom study and provide an opportunity to explore concepts in an authentic setting that can engage the cognitive, affective and psychomotor domains of learning. Website Interaction Methodology Preparation on the part of the Introduction Discussion educator is important for such learning Problem Literature opportunities. Educators usually use Results the Center’s website to interact with the Center in advance of a tour. Content is Key While in the classroom, teachers need content resources that engage critical thinking skills. They also need PD opportunities to increase their knowledge base and improve their compensation. In it’s current iteration, the Judiciary History Center’s website is not very user friendly with regard to navigation and content, particularly for educators.

Methodology Introduction Discussion Problem Literature Results User Friendly Net Effect Muddled Design The purpose of this study is to The net effect of the redesign will The Center’s website provides evaluate a redesign of the Center’s increase student engagement with content resources and information site for ease of navigation and civics content. on PD opportunities but the access to content for users. information is not easily found.

Methodology Introduction Discussion Problem Literature Results Yang, Linder, and Moore, C., 2015 Cappel, James J., & Bolchini, 2012 Huang, Zhenyu, 2015 Museum users move Design-oriented Evaluation Good usability practices fluidly between online of Percieved website allow users to accomplish and in-person usability ( DEEP ) their goals quickly and engagement, museum Content easily, promote customer educators will need to Structure & Information goodwill, and send a work to make sure the Architecture positive message about the two spheres are Navigation image and professionalism seamlessly integrated Layout Consistency of the organization Visual Guidance

Research Questions & Goals 1. How did the redesign of the website affect the ease of use for educators seeking Methodology Introduction Discussion Problem Literature to schedule a tour of the Center or access Results available resources? 2. How did the redesign of the website affect educators’ interest in professional development available through the Center?

Content Analysis Methodology Introduction Discussion Problem Literature Results The first phase of the website redesign addresses the navigation of content for educators, in particular, the scheduling of field trips, the development of lesson plans, and information regarding professional development opportunities.

Content Analysis Methodology Introduction Discussion Problem Literature Results Before wireframing began, data was collected from Hotjar on the Center’s active site. With Hotjar polling enabled, visitors to the homepage were asked if they were able to find the information for which they were looking. Nearly a quarter of respondents (23.6%) were unable to find relevant information.

Participants Nine in-service public and private K-12 Social Studies teachers were recruited to participate in the study. 67% Have been teaching for over ten years Have been teaching for three to five years 33% Methodology Introduction Discussion Problem Literature Results All have experience with website navigation on various devices 100% 67% Found out about PD opportunities through school announcements or email Applied for PD opportunities via email 56% 44% Applied for PD opportunities via a website All are interested in keeping abreast of civics-related topics 100%

Evaluation Instruments Methodology Introduction Discussion Problem Literature Results 12/5/2018 Usability Study Pre-Test Survey Usability Study Post­Test Survey Usability Study Pre­Test Survey Thank you for agreeing to participate in this study of a website. Please keep in mind that the site is being tested, not you. You canʻt do anything wrong here. You can withdraw from this study at anytime. Thank you for agreeing to participate in this study of a website. Please keep in mind that the site is being tested, not you. You canʻt do anything wrong here. You can withdraw from the study at anytime. * Required * Required Usability Study Follow­Up Questions 1. Email address * 1. On a scale from 1 to 5 with 1 representing very difficult and 5 representing extremely easy, how would you rate your experience booking a field trip to the Center? * Mark only one oval. Usability Study Preliminary Questions 1 2 3 4 5 2. Are you a pre­service or in­service educator? * Mark only one oval. Very difficult Extremely easy Yes No 2. On a scale from 1 to 5 with 1 representing very difficult and 5 representing extremely easy, how would you rate your experience finding information to help you develop a lesson plan? * Mark only one oval. 3. If you are in­service, how long have you been teaching Social Studies? * Mark only one oval. 1 2 3 4 5 1­2 years 3­5 years Very difficult Extremely easy 5­10 years More than 10 years 3. On a scale from 1 to 5 with 1 representing very difficult and 5 representing extremely easy, how would you rate your experience finding information regarding professional development? * 4. As an educator, have you ever planned a Social Studies­related field trip in Hawaiʻi? * Mark only one oval. Mark only one oval. Yes 1 2 3 4 5 No Very difficult Extremely easy 5. As an educator, do you know where to find information on Social Studies­related field trips in Hawaiʻi? * 4. On a scale from 1 to 5 with 1 representing very difficult and 5 representing extremely easy, Mark only one oval. how would you rate your experience finding information on historic Aliʻiōlani Hale? * Yes Mark only one oval. No 1 2 3 4 5 Very difficult Extremely easy 5. After participating in this study, would you recommend this website to colleagues? * Mark only one oval. Yes No https://docs.google.com/forms/d/1M7Ge0np2fA6GnpRbsk5W3uQq9_ICpUT_uWm524pAk8Y/edit 1/3

Project Design Methodology Introduction Discussion Problem Literature Results • Initial wireframing of the revised site was done using Adobe XD • The redesigned site was created using Wix • The redesigned site is not yet visible to the general public • Google Forms and observations were used to collect data on participants in the study

Procedures The protocol sessions consisted of two parts: • The completion of four tasks with Screencastify enabled • A post-test survey using Google Forms Methodology A script was used to introduce the study, inform participants of Introduction Discussion Problem Literature procedures, advise them of any potential risk, and request consent from Results them to participate Four tasks and scenarios were developed to address four objectives: • Be able to easily book a field trip • Easily find information to help them develop lesson plans • Be able to easily find information about PD opportunities Be able to easily find information about Ali ʻ i ō lani Hale •

Qualitative & Quantitative Data 4 1 2 3 Methodology Introduction Discussion Problem Literature Results Screencastify DEEP Errors Post-survey Review recordings Consider the five Classify errors as Review post-test and notes taken dimensions of minor, major, and survey results and during the perceived usability critical plan and execute sessions changes

Booking Field Trips Methodology Introduction Discussion Problem Literature Results From the first to the third iteration there was an improvement in the ability to easily book a tour of the Center.

Lesson Planning Methodology Introduction Discussion Problem Literature Results From the first to the third iteration there was a slight improvement in the ability to find information to help with lesson planning.

Professional Development Inquiry Methodology Introduction Discussion Problem Literature Results Over the course of the study, the ability to find information on professional development remained relatively unchanged.