Educators & The Judiciary History Center: A Usability Study - - PowerPoint PPT Presentation

Educators & The Judiciary History Center: A Usability Study

Reginald (Keahe) Davis The University of Hawaiʻi at Mānoa davisr@hawaii.edu www.jhchawaii.net https://keahe6.wixsite.com/jhctestsite Introduction Problem Literature Methodology Results Discussion

Introduction Problem Literature Methodology Results Discussion

• I act as administrator for the Center’s website
• Education Director - King Kamehameha V Judiciary History

Center

• I create and coordinate professional development programs for

Social Studies teachers

• I oversee tour programs for the Hawaiʻi State Supreme Court and

First Circuit Court of Hawaiʻi

• M.Ed. – Learning Design & Technology (LTEC) candidate at

the University of Hawaiʻi at Mānoa

ALOHA KĀKOU

Keahe Davis

Introduction Problem Literature Methodology Results Discussion

Aliʻiōlani Hale

Introduction Problem Literature Methodology Results Discussion

School Field Trips

They enhance classroom study and provide an opportunity to explore concepts in an authentic setting that can engage the cognitive, affective and psychomotor domains of learning.

Website Interaction

Preparation on the part of the educator is important for such learning

• pportunities. Educators usually use

the Center’s website to interact with the Center in advance of a tour.

Content is Key

While in the classroom, teachers need content resources that engage critical thinking skills. They also need PD

• pportunities to increase their

knowledge base and improve their compensation.

In it’s current iteration, the Judiciary History Center’s website is not very user friendly with regard to navigation and content, particularly for educators.

Introduction Problem Literature Methodology Results Discussion

Muddled Design

The Center’s website provides content resources and information

• n PD opportunities but the

information is not easily found.

Net Effect

The net effect of the redesign will increase student engagement with civics content.

User Friendly

The purpose of this study is to evaluate a redesign of the Center’s site for ease of navigation and access to content for users.

Introduction Problem Literature Methodology Results Discussion

Yang, Linder, and Bolchini, 2012

Design-oriented Evaluation

• f Percieved website

usability (DEEP) Content Structure & Information Architecture Navigation Layout Consistency Visual Guidance

Moore, C., 2015

Museum users move fluidly between online and in-person engagement, museum educators will need to work to make sure the two spheres are seamlessly integrated

Cappel, James J., & Huang, Zhenyu, 2015

Good usability practices allow users to accomplish their goals quickly and easily, promote customer goodwill, and send a positive message about the image and professionalism

• f the organization

Introduction Problem Literature Methodology Results Discussion

Research Questions & Goals

• 1. How did the redesign of the website

affect the ease of use for educators seeking to schedule a tour of the Center or access available resources?

• 2. How did the redesign of the website

affect educators’ interest in professional development available through the Center?

Introduction Problem Literature Methodology Results Discussion

Content Analysis

The first phase of the website redesign addresses the navigation of content for educators, in particular, the scheduling of field trips, the development of lesson plans, and information regarding professional development

• pportunities.

Introduction Problem Literature Methodology Results Discussion

Content Analysis

Before wireframing began, data was collected from Hotjar on the Center’s active

• site. With Hotjar polling enabled, visitors to the homepage were asked if they were

able to find the information for which they were looking. Nearly a quarter of respondents (23.6%) were unable to find relevant information.

Introduction Problem Literature Methodology Results Discussion

Participants

Nine in-service public and private K-12 Social Studies teachers were recruited to participate in the study.

Have been teaching for over ten years

67%

Have been teaching for three to five years

33%

All have experience with website navigation on various devices

100%

Found out about PD opportunities through school announcements or email

67%

Applied for PD opportunities via email

56%

Applied for PD opportunities via a website

44%

All are interested in keeping abreast of civics-related topics

100%

Introduction Problem Literature Methodology Results Discussion

Evaluation Instruments

Usability Study Pre­Test Survey

• 1. Email address *

Usability Study Preliminary Questions

• 2. Are you a pre­service or in­service educator? *

• 3. If you are in­service, how long have you been teaching Social Studies? *

• 4. As an educator, have you ever planned a Social Studies­related field trip in Hawaiʻi? *

• 5. As an educator, do you know where to find information on Social Studies­related field trips in

Usability Study Post­Test Survey

Usability Study Follow­Up Questions

• 1. On a scale from 1 to 5 with 1 representing very difficult and 5 representing extremely easy,

• 2. On a scale from 1 to 5 with 1 representing very difficult and 5 representing extremely easy,

• 3. On a scale from 1 to 5 with 1 representing very difficult and 5 representing extremely easy,

• 4. On a scale from 1 to 5 with 1 representing very difficult and 5 representing extremely easy,

• 5. After participating in this study, would you recommend this website to colleagues? *

Introduction Problem Literature Results Discussion

Project Design

• Initial wireframing of the revised site was done using Adobe XD
• The redesigned site was created using Wix
• The redesigned site is not yet visible to the general public
• Google Forms and observations were used to collect data on participants in

the study

Methodology

Introduction Problem Literature Methodology Results Discussion

Procedures

The protocol sessions consisted of two parts:

• The completion of four tasks with Screencastify enabled
• A post-test survey using Google Forms

A script was used to introduce the study, inform participants of procedures, advise them of any potential risk, and request consent from them to participate Four tasks and scenarios were developed to address four objectives:

• Be able to easily book a field trip
• Easily find information to help them develop lesson plans
• Be able to easily find information about PD opportunities
• Be able to easily find information about Aliʻiōlani Hale

Introduction Problem Literature Methodology Results Discussion

3 2 1

Screencastify

Review recordings and notes taken during the sessions

DEEP

Consider the five dimensions of perceived usability

Errors

Classify errors as minor, major, and critical

4

Post-survey

Review post-test survey results and plan and execute changes

Qualitative & Quantitative Data

Introduction Problem Literature Methodology Results Discussion

Booking Field Trips

From the first to the third iteration there was an improvement in the ability to easily book a tour of the Center.

Introduction Problem Literature Methodology Results Discussion

Lesson Planning

From the first to the third iteration there was a slight improvement in the ability to find information to help with lesson planning.

Introduction Problem Literature Methodology Results Discussion

Professional Development Inquiry

Over the course of the study, the ability to find information on professional development remained relatively unchanged.

Introduction Problem Literature Methodology Results Discussion

Errors

Critical There were three critical that all involved a dead link or button. These errors were immediately corrected. Major There were two observed major errors:

• 1. The placement of the calendar of available dates for tour bookings - the

calendar of available dates and times is placed above the form used to request a tour. The calendar displays booked tours but cannot be clicked on to request a tour. It took a moment for teachers to realize that the calendar was merely for reference.

• 2. The placement of the information page for Aliʻiōlani Hale - the Aliʻōlani Hale

information page is placed under the VISIT menu header but many teachers first assumed they would find information on Aliʻiōlani Hale under the EDUCATION because the scenario for the task said they were looking for information for a school visit. Minor The minor errors with navigating the site resulted from a few teachers thinking of

• ptional ways to complete assigned tasks.

Introduction Problem Literature Methodology Results Discussion

Teacher Comments

It was easy to find the information on the website and the graphics were

• interesting. Also, there were videos available which I could show my classes.

It was user friendly, attractive, and had a lot of good information. Love the overall look of the website. Wish it included more visuals but it was very easy to navigate and user-friendly for educators. These were honest answers. I have actually spent hours on websites trying to find simple information or booking things. This website was really really user friendly and seemed like a professional (who kept others in mind) created this website. It was easy to find the information on the website and the graphics were

• interesting. Also, there were videos available which I could show my classes.

Nice images; includes Hawaiian language; easy to navigate overall. It is eye catching ...but if I want information I want it practically thrown at me. The site is too wordy. You should develop more content directly for students.

Introduction Problem Literature Methodology Results Discussion

DEEP

Using DEEP was extremely helpful in refining the design direction of the site

Wix was useful

Using Wix was useful for creating a draft design for the new site. Due to needed plugins, WordPress will continue to be used but the design will mimic Wix

Participant Reaction

All participants were interested in improving the usefulness of the Center’s site

Introduction Problem Literature Methodology Results Discussion

Mahalo nui loa…

Nā hoa kūpono – James, Lani, & Courtney

• Dr. Catherine Fulford, PhD and the faculty

at the University of Hawaiʻi at Mānoa