TUMexam iOS Usage Analytics Final talk for the Bachelors Thesis by - - PowerPoint PPT Presentation

▶
tumexam ios usage analytics
SMART_READER_LITE
LIVE PREVIEW

TUMexam iOS Usage Analytics Final talk for the Bachelors Thesis by - - PowerPoint PPT Presentation

Nov 24, 2022 975 likes •1.25k views

Chair of Network Architectures and Services Department of Informatics Technical University of Munich TUMexam iOS Usage Analytics Final talk for the Bachelors Thesis by Henri Allgwer advised by Stephan Gnther, Maurice Leclaire, Benedikt

slide-1
SLIDE 1

Chair of Network Architectures and Services Department of Informatics Technical University of Munich

TUMexam iOS Usage Analytics

Final talk for the Bachelor’s Thesis by

Henri Allgöwer

advised by Stephan Günther, Maurice Leclaire, Benedikt Jaeger, Georg Wechslberger, Johannes Naab Monday 16th December, 2019 Chair of Network Architectures and Services Department of Informatics Technical University of Munich

slide-2
SLIDE 2

General Topic / Motivation

Background

(a) Home screen (b) Correction screen Figure 1: Screenshots of the TUMexam correction application

  • H. Allgöwer — TUMexam Usage Analytics

2

slide-3
SLIDE 3

General Topic / Motivation

Goal of the thesis

Improving the existing iOS application

  • 1. Analyze user behaviour
  • Does the user interact with the app as expected?
  • Is it intuitive, easy to use?
  • 2. Determine weaknesses in the app
  • UI / UX
  • Missing features
  • 3. Figuring out a way to optimize
  • Improve existing features
  • Add desired functionalities
  • H. Allgöwer — TUMexam Usage Analytics

3

slide-4
SLIDE 4

Measurements

  • 1. Time
  • Timestamps
  • Amount of time invested by correctors
  • Comparison to correction by hand
  • 2. Pencil gestures
  • Amount and length of lines drawn, erased,

undone, redone

  • UI elements clicked
  • Change in corrector behaviour
  • 3. User feedback
  • Questionnaire in the app
  • Rating system similar to App Store
  • Multiple flags to be set for each subproblem

Figure 2: Stopwatch Figure 3: Toolbar Figure 4: Flagging in the app

  • H. Allgöwer — TUMexam Usage Analytics

4

slide-5
SLIDE 5

Privacy Agreement

  • Privacy of users is of utmost importance
  • Inform the users on what is being tracked and why

Figure 5: Prototype privacy agreement screen in the app

  • H. Allgöwer — TUMexam Usage Analytics

5

slide-6
SLIDE 6

Privacy Agreement

  • Privacy of users is of utmost importance
  • Inform the users on what is being tracked and why

Figure 5: Prototype privacy agreement screen in the app

Necessity vs risk of jeopardizing progress

  • H. Allgöwer — TUMexam Usage Analytics

5

slide-7
SLIDE 7

Statistical Analyses

Overview

  • In-depth analysis of the correction of the GRNVS endterm and retake exams
  • Python script that evaluates user data gathered of any exam and provides the user with a

quick overview. This includes:

– Overall correction time – Correction time split into correction group (i.e. first and second correctors), problems, individual users – Performance overview for each corrector – Raw user data csv split into subgroups for easier analysis

  • H. Allgöwer — TUMexam Usage Analytics

6

slide-8
SLIDE 8

Statistical Analyses

Time stamp problematic Problem: Improper dismissal of the application resulting in inaccurate time stamps

(0–25) (25–50) (50–75) (75–100) (100–125) (125–150) (150–175) (175–200) (200–225) (225–250) (250–275) (275–300) (300–325) (325–350) 500 1,000 1,500 2,000 2,500 Time Intervalls in Seconds Amount of Database Entries

Figure 6: Time distribution of the correction of a single problem

  • H. Allgöwer — TUMexam Usage Analytics

7

slide-9
SLIDE 9

Statistical Analyses

Time stamp problematic Problem: Improper dismissal of the application resulting in inaccurate time stamps

(0–25) (25–50) (50–75) (75–100) (100–125) (125–150) (150–175) (175–200) (200–225) (225–250) (250–275) (275–300) (300–325) (325–350) 500 1,000 1,500 2,000 2,500 Time Intervalls in Seconds Amount of Database Entries

Figure 6: Time distribution of the correction of a single problem

⇒ Upper limit of 800s and need for more precise time stamps

  • H. Allgöwer — TUMexam Usage Analytics

7

slide-10
SLIDE 10

Statistical Analyses

GRNVS Endterm

> 1000 enrolled students 711 written exams 22 correctors/users 231 hours spent correcting

  • H. Allgöwer — TUMexam Usage Analytics

8

slide-11
SLIDE 11

Statistical Analyses

First vs. second correctors Problem First group Second group 1 2 min 07 s 1 min 35 s 2 1 min 16 s 1 min 03 s 3 1 min 48 s 1 min 42 s 4 1 min 10 s 0 min 36 s 5 1 min 09 s 0 min 49 s

Table 1: Average time per problem

  • H. Allgöwer — TUMexam Usage Analytics

9

slide-12
SLIDE 12

Statistical Analyses

First vs. second correctors Problem First group Second group 1 2 min 07 s 1 min 35 s 2 1 min 16 s 1 min 03 s 3 1 min 48 s 1 min 42 s 4 1 min 10 s 0 min 36 s 5 1 min 09 s 0 min 49 s

Table 1: Average time per problem

Overall time Group 30 h 49 min 2 22 h 31 min 2 18 h 48 min 2 18 h 00 min 1 12 h 47 min 1 12 h 23 min 1 12 h 15 min 1 . . .

Table 2: Overall correction time

  • H. Allgöwer — TUMexam Usage Analytics

9

slide-13
SLIDE 13

Statistical Analyses

Correction timeline 26 100 200 243 300 100 200 300 400 500 600 700 800

Index Time in Seconds Figure 7: Typical timeline of the correction process

  • H. Allgöwer — TUMexam Usage Analytics

10

slide-14
SLIDE 14

Statistical Analyses

Correction on subproblem basis

250 500 750 1,000 1,250 1,500 1,750 100 200 300 400 500 600 700 800 Index Time in Seconds Figure 8: Timeline of a subproblem by subproblem correction

  • H. Allgöwer — TUMexam Usage Analytics

11

slide-15
SLIDE 15

Statistical Analyses

Correction on subproblem basis

250 500 750 1,000 1,250 1,500 1,750 100 200 300 400 500 600 700 800 Index Time in Seconds Figure 8: Timeline of a subproblem by subproblem correction

⇒ Need for a selection of correction type

  • H. Allgöwer — TUMexam Usage Analytics

11

slide-16
SLIDE 16

Statistical Analyses

GRNVS Retake

343 registered students 243 written exams 16 correctors/users 84 hours spent correcting

  • H. Allgöwer — TUMexam Usage Analytics

12

slide-17
SLIDE 17

Statistical Analyses

First vs. second corrector Total Time tavg/prob Total Exams Correction group 8 h 02 min 3 min 24 s 142 1 7 h 31 min 1 min 51 s 243 2 7 h 21 min 1 min 49 s 243 2 6 h 38 min 2 min 37 s 152 1 6 h 32 min 1 min 26 s 176 1 5 h 49 min 1 min 26 s 243 2 5 h 48 min 1 min 25 s 243 2 5 h 35 min 3 min 20 s 101 1 5 h 19 min 2 min 06 s 151 1 4 h 54 min 1 min 32 s 191 1 4 h 39 min 1 min 09 s 243 2 . . .

Table 3: Overview of the corrector’s individual performance during the retake exam

  • H. Allgöwer — TUMexam Usage Analytics

13

slide-18
SLIDE 18

Statistical Analyses

Problem distribution

Figure 9: Distribution of time and pencil gestures for the five problems of the retake exam

  • H. Allgöwer — TUMexam Usage Analytics

14

slide-19
SLIDE 19

Statistical Analyses

Pencil pressure

25 50 75 100 0.5 1 1.5 2 2.5 Index Pencil Pressure First corrector Second corrector

Figure 10: Pencil pressure timeline for an exemplary first and second corrector

  • H. Allgöwer — TUMexam Usage Analytics

15

slide-20
SLIDE 20

Outlook

Future improvements

  • 1. Statistics view in the application
  • 2. Smart timestamps
  • 3. Selectable correction mode
  • 4. Shortcut to additional pages
  • 5. TUMexam for examinees
  • H. Allgöwer — TUMexam Usage Analytics

16

slide-21
SLIDE 21

References

Marie Davidian und Thomas A. Louis. “Why Statistics?” In: Science 336.6077 (2012),

  • S. 12–12.

Cem Kaner, Senior Member und Walter P . Bond. “Software Engineering Metrics: What Do They Measure and How Do We Know?” In: In METRICS 2004. IEEE CS. Press, 2004. Bruce D. Weinberg u. a. “Internet of Things: Convenience vs. privacy and secrecy”. In: Business Horizons 58.6 (2015). SPECIAL ISSUE: THE MAGIC OF SECRETS, S. 615 –624. Tobias Röhm. “The MALTASE Framework For Usage-Aware Software Evolution”. Diss. München: Technische Universität München, 2015. Stephan Günther. TUMexam Flyer. Available online at https://tumexam.de/static/tumexam-booklet-20191021.pdf; last accessed on 2019/10/31.

  • H. Allgöwer — TUMexam Usage Analytics

17

slide-22
SLIDE 22

References

Images

  • Figure 2 Icons made by https://www.flaticon.com/authors/freepik
  • Classroom icon on slide 8 made by https://www.flaticon.com/authors/freepik
  • Test icons on slides 8 and 13 made by https://www.flaticon.com/authors/surang
  • Hourglass icons on slides 8 and 13 made by https://www.flaticon.com/authors/smash

icons

  • Domain icon on slide 13 made by https://www.flaticon.com/authors/eucalyp
  • H. Allgöwer — TUMexam Usage Analytics

18

slide-23
SLIDE 23

Backup

Distribution endterm

Figure 11: Distribution of time and pencil gestures for the endterm’s five problems

  • H. Allgöwer — TUMexam Usage Analytics

19

slide-24
SLIDE 24

Backup

Timeline - First vs. second corrector

26 100 200 243 300 100 200 300 400 500 600 700 800

Index Time in Seconds

(a) First corrector

100 200 300 400 500 600 700 800 900 100 200 300 400 500 600 700 800 Index Time in Seconds

(b) Second corrector Figure 12: Timeline of correction time per problem for the two correction groups

  • H. Allgöwer — TUMexam Usage Analytics

20

slide-25
SLIDE 25

Backup

Timeline - First vs. second corrector

26 100 200 243 300 100 200 300 400 500 600 700 800

Index Time in Seconds

(a) First corrector

100 200 300 400 500 600 700 800 900 100 200 300 400 500 600 700 800 Index Time in Seconds

(b) Second corrector Figure 13: Timeline of correction time per problem for the two correction groups

  • H. Allgöwer — TUMexam Usage Analytics

21

slide-26
SLIDE 26

Backup

First vs. second correctors 83% 17%

Figure 14: Lines drawn

80% 20%

Figure 15: Lines erased

79% 21%

Figure 16: Lines undone

  • H. Allgöwer — TUMexam Usage Analytics

22