Experimental Design & Evaluation 1. Introduction to ED&E Sunyoung�Kim,�PhD
Today’s agenda Who are we? • What is this course about? • Syllabus • Evaluation • Class policy • Introduction to ED&E •
Instructor Sunyoung Kim Office: Room 329 in SC&I Email: sunyoung.kim@rutgers.edu Website: http://www.sunyoung.kim.org
Your turn
Experimental Design & Evaluation
Experimental Design & Evaluation “A scientific procedure undertaken to make a discovery, test a hypothesis, or demonstrate a known fact”
Experimental Design & Evaluation “The process of planning a study to meet specified objectives (of making a discovery, testing a hypothesis, or demonstrating a known fact)”
Experimental Design & Evaluation “The making of a judgment about the amount, number, or value of something; assessment”
Experimental Design & Evaluation Planning a scientific study to make a discovery, test a hypothesis, or demonstrate a known fact & assessing the results of the study
Interaction Design (IxD) The practice of designing interactive digital products, environments, systems, and services
What are factors to consider for its success?
UX Design (UxD) The process of enhancing user satisfaction with a product by improving the usability, accessibility, and pleasure provided in the interaction with the product
Experimental Design & Evaluation Planning a scientific study to make a discovery, test a hypothesis, or demonstrate a known fact & assessing the results of the study Planning a a hum human-sub subjec ect t stud study to ensure/improve user user sa sati tisfa sfacti tion on with th a intera nteracti tive e digita tal pr produ duct & assessing the usa usability ty, a , accessibi bility, a , and pl d pleasure when interacting with the product
Interaction design cycle
Syllabus 1. Learn (understanding the users, tasks and contexts) Data collection o - Interview, Survey - Contextual Inquiry Interpretation o - Work model - Affinity diagram - Grounded Theory Consolidation o 2. Design (building a system) Persona, Scenario, Storyboard o 3. Measure (evaluating the system) Heuristic evaluation o Think Aloud, usability testing o 4. Report
Grading 50% Project • 30% Quizzes • 20% Class participation • 10% Discussion and class activities - 10% Reading response -
Class Homepage http://www.sunyoungkim.org/class/ede_sp17/ • Access to slides- ID: rutgers PW: hci •
Project (50%) Schedule and grades • Due ( by midnight) Assignment Weight 9/5 Team building check 9/12 Select your project topic: a problem, user, task, and context 5% 9/19 Submit an IRB 10/3 Plan data collection 5% 10/17 Collect data 10% 10/24 Interpret data 10% 10/31 Consolidate findings - Modeling 10% 11/14 Prototype solutions 10% 12/5 Evaluate the solutions 10% 12/12 Final presentation 20% 12/19 Final report submission 20%
Project: Goals Acquire hands-on experiences of user-centered design process: data • collection, prototyping, analysis and report of the findings Write an academic paper and submit it to CHI (https://chi2018.acm.org/) • Student Design Competition (https://chi2018.acm.org/authors/student- o design-competition/) Late-Breaking Work (https://chi2018.acm.org/authors/late-breaking-work/) o Check out sample papers (Canvas > files) o
https://chi2018.acm.org/authors/student-design-competition/
CHI 2017 SDC Finalists (https://chi2017.acm.org/assets/chi2017printprogforweb.pdf)
CHI 2016 SDC Finalists (https://chi2016.acm.org/wp/wp-content/uploads/2016/05/chi2016-program-final-for-web-2.pdf)
Grading: Quiz (30%) 3 quizzes • Format: a number of short-answer/open-end questions • Schedule and points • Date Weight Topics 10/3 10% Understanding the context (data collection) 10/31 10% Interpreting and consolidating the collected data 12/5 10% Prototyping and Evaluating the system
Grading: Class participation (20%) 10% Discussion and class activities • 10% Reading response • o Due Tuesday midnight, before class - Reading is posted on the website - Submit a reading response (a summary of reading) to Canvas discussion board o Will be graded - 2 = good - 1 = partial - 0 = no submission
Class policies Be mindful during class time (aka no Facebook, Instagram, twitter, • snapchat, groupme, and so forth during class) Respect and collaborate with your team members • All assignments are due by midnight before the class • Assignments will not be accepted late - Notify two days in advance if you have to miss taking a quiz • Cheating (official) will get you an F in the course •
Questions?
Introduction to ED&E
Interaction Design (IxD) The practice of designing interactive digital products, environments, systems, and services
What is a successful digital product? What used to be a successful digital product?
What is a successful digital product?
Usability Usability refers to the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of user Usability measures the quality of a user's experience when interacting with a product or system Effectiveness: Can users successfully achieve their objectives? • Efficiency: How much effort is expended in achieving those • objectives? Satisfaction: Was the experience satisfactory? •
UI Hall of Fame or Shame?
UI Hall of Fame or Shame?
User-Centered Design (UCD)
User-Centered Design (UCD)
UI UI vs vs UX UX User Interface User Experience
A world without UCD
Why UCD?
Interface Bloat
Interface Bloat
User-Centered Design (UCD) Trying to satisfy the needs of all users may mean that you fail to satisfy the needs of any one user
User-Centered Design (UCD)
Why is User-centered approach important? We interact with computing systems • in every facet of our everyday lives When interacting with computing • systems, they do it through a user interface How a user interacts with a computer • is different for each system People do not accept products with • poor interfaces If a system doesn’t work as expected, • it is not a user’s fault but a designer’s fault!
Traditional s/w design process Waterfall model Waterfall model is bad for UI design because: Users are not involved in validation until acceptance testing à So we won’t • find out until the end UI flaws often cause changes in requirements and design à So we have to • throw away carefully-written and tested code
UCD Process Spiral model Iterative model Early focus on users and tasks • Plan to iterate with cheap, fast • prototypes Evaluate frequently •
Data-Driven Design An approach to defining a new system that collects multiple user- centered techniques into an integrated design process Not based on personal opinion, anecdotes, or unverifiable claims about what users would like but based on “DATA”!
Benefits of Data Requires design team to agree on common observations and their • representation Reveals hidden aspects of work, that may be implicit in the user’s • understanding Contextual Inquiry is a method for gathering and representing • data about the user and his/her work
Contextual Design Models Models Models Models Interview Sequence Notes Models Models Models Models Models Models Models Interview Models Sequence Idea Notes Models Models Models Models Idea Interview Sequence Sequence- Notes flow model Idea Models Models Models Models Interview Sequence Idea Affinity Notes Models Models Models Models Interview Sequence Notes
Contextual Design: Challenge How do we gather data from the users? How do we turn data into criteria? How do we design a system based on criteria?
Understanding the users
Interaction design cycle
Designer Users We (designers) are not the users!
Who are the users? Those who interact directly with the product Those who manage direct users Those who receive output from the product Those who make the purchasing decision Those who use competitors' product Stakeholders Primary users: people who use the product directly • Secondary users: people who are affected by the product, or who • influence its development
Who are the users? Physical • Age § Sex § Culture § Physical ability & disabilities § Educational background • Computer/IT experience • Psychological • Motivation § Attitude §
Identify potential participants Which participants? • Gender o Age o Education level o Computer users o How many participants? • Resource (physical, financial) o Validity o How to recruit participant? • Informed consent o IRB o
Recommend
More recommend
![Experimental Design for Simulation Experimental Design for Simulation [Law, Ch. 12][Sanchez et al.](https://c.sambuz.com/680818/experimental-design-for-simulation-s.jpg)
