A Population Approach to Ubicomp System Design Matthew C. Higgs , - - PowerPoint PPT Presentation

▶
a population approach to ubicomp system design
SMART_READER_LITE
LIVE PREVIEW

A Population Approach to Ubicomp System Design Matthew C. Higgs , - - PowerPoint PPT Presentation

Nov 09, 2023 393 likes •847 views

A Population Approach to Ubicomp System Design Matthew C. Higgs , Mark Girolami , Matthew Chalmers, Muffy Calder, Alistair Morrison, Oana Andrei, Marek Bell, ScottSherwood, & John Rooksby Center for Computational Statistics and

slide-1
SLIDE 1

A Population Approach to Ubicomp System Design

Matthew C. Higgs⋆, Mark Girolami⋆, Matthew Chalmers, Muffy Calder, Alistair Morrison, Oana Andrei, Marek Bell, ScottSherwood, & John Rooksby

⋆Center for Computational Statistics and Machine Learning

Department of Statistical Science University College London

slide-2
SLIDE 2

Contents

Introduction Project Description Vision and Ambition Statistical Analysis Analysis of User Traces User life-time length Future Work Future Apps Socio-technical Predicted Legacy

slide-3
SLIDE 3

Mark Girolami

Matthew Higgs

Muffy Calder

Oana Andrei

Matthew Chalmers

Scott Sherwood Marek Bell John Rooksby Alistair Morrison

Computer Science

Human Computer Interaction Information Visuals Ubiquitous Computing Socio Technical Formal Methods Computer Statistics

Statistical Modelling Formal Modelling App Development Data Logging Industrial Collaboration

Project

Topics. People. Contributions.

slide-4
SLIDE 4

Mark Girolami

Matthew Higgs

Muffy Calder

Oana Andrei

Matthew Chalmers

Scott Sherwood Marek Bell John Rooksby Alistair Morrison

Project

Communication.

slide-5
SLIDE 5

Statistical Modelling Formal Modelling App Development Data Logging Industrial Collaboration

Scottish Premier League Edinburgh Festival Living PlanIT

Existing Infrastructure

Contributions.

slide-6
SLIDE 6

Statistical Modelling Formal Modelling App Development

Hungry Yoshi Post Factory Match FFIT App Tracker

Data Logging Industrial Collaboration

Existing Infrastructure

Contributions.

slide-7
SLIDE 7

Statistical Modelling Formal Modelling App Development Data Logging

SGLog

Industrial Collaboration

Existing Infrastructure

Contributions.

slide-8
SLIDE 8

Statistical Modelling Formal Modelling App Development Data Logging Industrial Collaboration

Existing Infrastructure

Work flow.

slide-9
SLIDE 9

Contents

Introduction Project Description Vision and Ambition Statistical Analysis Analysis of User Traces User life-time length Future Work Future Apps Socio-technical Predicted Legacy

slide-10
SLIDE 10

Adaptive Design

The ideal.

  • Observe software in the wild.
  • Initiate a positive change based on observation.

The ideal is ambiguous about what is observed, what change is made, and who makes the change.

slide-11
SLIDE 11

Software in the Wild

The emergence of the “app store” market has enabled researchers to run worldwide ubicomp trials with huge numbers of users.

slide-12
SLIDE 12

App Development

Hungry Yoshi Post Factory Match FFIT App Tracker

Trial methodology Modularity Modularity Ubiquity

Software in the Wild

Study type.

slide-13
SLIDE 13

Data Logging

SGLog

Trial methodology Modularity Modularity Ubiquity

d a t a d a t a d a t a d a t a d a t a d a t a d a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t

Software in the Wild

Study type.

slide-14
SLIDE 14

Statistical Modelling

User models Software models

Data Logging

SGLog d a t a d a t a d a t a d a t a d a t a d a t a d a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t a d a t

Data Analysis

slide-15
SLIDE 15

Vision Summary

Trials:

  • Create socio-technical environments and persuade

people to immerse themselves. Statistical:

  • Infer the structure and dynamics in the use and

evolution data of software populations. Ambition:

  • Develop visualisation tools and formal methods to

guide “developers” in their design decisions.

slide-16
SLIDE 16

Contents

Introduction Project Description Vision and Ambition Statistical Analysis Analysis of User Traces User life-time length Future Work Future Apps Socio-technical Predicted Legacy

slide-17
SLIDE 17

Analysis of User Traces

Consider a string of symbols a = a1a2 · · · aN. (1) Where each symbol ai takes values in a finite set A. Consider a set of strings A = {au, u = 1, . . . , M, |au| = N(u) ∈ N}. (2) Assume A to be a set of user traces, where A represents a set of possible actions.

slide-18
SLIDE 18

Analysis of User Traces

Given a set of user traces A, we want to:

  • Characterise the natural behaviours of the population.
  • Summarise the population based on these

behaviours. “Natural behaviours” are frequently occurring patterns of actions. A “summary” of the population is a representation of the population in a low dimensional space spanned by latent behavioural semantics. We use Probabilistic Latent Semantic Analysis.

slide-19
SLIDE 19

Analysis of User Traces

PLSA consists of:

  • A directed graph (digraph) G(A, E) representing the

set of possible action transitions in the app.

  • A set of K transition matrices {Tk, k = 1, . . . , K} over

the (actionable) nodes of the graph.

  • A mixture weighting θ = (θ1, . . . , θK) for each user.

We assume each string a is generated by:

  • Moving from ai to ai+1 using transition Tk with

probability θk. An admixture of (first-order) discrete-time Markov chains.

slide-20
SLIDE 20

Simplified Hungry-Yoshi

(a) Main-menu screen-shot. (b) Yoshi screen-shot. (c) Plantation screen-shot.

Symbol Yoshi Feed Plant Pick Description View a yoshi. Feed a yoshi. View a plant. Pick a fruit.

(d) Symbol table and descriptions of corresponding in-app events.

Yoshi Plant Plant Yoshi Plant Pick Pick Pick Yoshi Feed Feed Feed Plant Pick · · ·

(e) An example of a typical user-trace.

slide-21
SLIDE 21

Hungry-Yoshi Digraph

Yoshi Feed Plant Pick

p1|1 p2|1 p3|1 p4|1 p2|2 p1|2 p4|2 p3|2 p3|3 p1|3 p4|3 p2|3 p4|4 p3|4 p2|4 p1|4

Figure : Yoshi,Feed,Plant,Pick enumerated as {1, 2, 3, 4}.

slide-22
SLIDE 22

Hungry-Yoshi PLSA (K = 2)

Yoshi Feed Plant Pick

(a) Digraph 1.

Yoshi Feed Plant Pick

(b) Digraph 2.

20 40 60 80 100 120 −1 −0.5 0.5 1 User index − ordered by θ1|u (descending). Value indexed by u (see legend). θ1|u θ2|u

(c) Population weights.

slide-23
SLIDE 23

Successful vs. Unsuccessful Strategies

Yoshi Feed Plant Pick

(d) Successful?

Yoshi Feed Plant Pick

(e) Unsuccessful?

What is the criteria for success?

slide-24
SLIDE 24

Successful vs. Unsuccessful

  • User is successful if they complete a number of tasks.
  • Developer is successful if a number of users

complete a number of tasks.

20 40 60 80 100 120 0.2 0.4 0.6 0.8 1 User index − ordered by θ1|u (descending). Value indexed by u (see legend). θ1|u log(N(u)/2)/s

Figure : Population weights with trace-lengths.

slide-25
SLIDE 25

PLSA Summary

  • Maximum Likelihood PLSA is prone to over-fitting, but

do we care.

  • How will developers use the results. (Personalisation

vs fragmentation).

  • How will formal methods use these results.
  • How do we choose K.
  • Extend to MDPs, identify “states” and latent reward

mechanisms.

slide-26
SLIDE 26

Contents

Introduction Project Description Vision and Ambition Statistical Analysis Analysis of User Traces User life-time length Future Work Future Apps Socio-technical Predicted Legacy

slide-27
SLIDE 27

User life-time length

1 2 3 4 5 1 2 3 4 5 6 log t log N(t) Log−log PF population data plot.

Figure : Plot of Post-Factory population data on a log-log scale. N(t) denotes number of users who performed at least t actions.

slide-28
SLIDE 28

Coin-toss game model

  • Constant bias (CB). Every player in every round uses

the same coin with bias p ∈ [0, 1].

  • Empirical bias (EB). Every player in each round uses

the same coin, but in each round a new coin with bias pt ∈ [0, 1] is given to all players.

  • Functional bias (FB). The bias pt is assumed to have

a functional form pt = (1 − p0)(1 + e−t/α) + p0, (3) where p0 ∈ [0, 1] is an initial probability, and α > 0 a scale parameter.

slide-29
SLIDE 29

Coin-toss game model

50 100 150 0.75 0.8 0.85 0.9 0.95 1 t. MLE of p(t). Plot of model parameters as a function of time. CB−model EB−model FB−model

Figure : Bias parameters for each CTG model, estimated using the full Post-Factory population data set.

slide-30
SLIDE 30

Coin-toss game model

1 2 3 4 5 1 2 3 4 5 6 log t log N(t) CB−model MLE. Data Mean path 1 2 3 4 5 1 2 3 4 5 6 log t log N(t) EB−model MLE. Data Mean path 1 2 3 4 5 1 2 3 4 5 6 log t log N(t) FB−model MLE. Data Mean path

Figure : Mean path plots for each of the CTG models.

CB-model EB-model FB-model 2-fold 71.9 ± 17.4 17.5 ± 12.2 14.8 ± 11.6 5-fold 13.1 ± 1.74 4.44 ± 1.57 4.25 ± 1.52 10-fold 4.12 ± 0.48 1.97 ± 0.46 1.93 ± 0.45

Figure : Estimates of MSE using K-fold CV. Mean MSE ± one std, from 10,000 reps.

slide-31
SLIDE 31

CTG Summary

  • The best performer is the FB-model with bias

pt = (1 − p0)(1 + e−t/α) + p0. (4)

  • This is a psychometrically motivated choice model.
  • p0 can be thought of as a function of the properties of

the app at start up.

  • α can be thought of as a reward rate that determines

how “attached” the user becomes with time.

slide-32
SLIDE 32

CTG Future

  • The model can be used in A/B testing of user

experiences.

  • The model can be extended to the regression setting

where −t/α is replaced with a function of user specific context.

  • All latent decisions of the user can be modelled using

similar choice analysis methods with psychometric interpretations.

slide-33
SLIDE 33

Contents

Introduction Project Description Vision and Ambition Statistical Analysis Analysis of User Traces User life-time length Future Work Future Apps Socio-technical Predicted Legacy

slide-34
SLIDE 34

Post Factory

Modularity study.

Figure : Post Factory Screen Shots.

Exploration-exploitation model.

slide-35
SLIDE 35

App Tracker

Ubiquity study. Chrome#

Word#

Preferences#

EverNote#

Installer#

Finder#

Sleep#

TrueCrypt# Sequel#Pro#

VLC#

Preview#

# .ed.ac.uk# # ?# # .gla.ac.uk# # ?# .gla.ac.uk#

Figure : App tracker analytics.

PLSA.

slide-36
SLIDE 36

Match FFIT

Audience permeation.

Figure : Match FFIT logo.

Slowly introduce modularity.

slide-37
SLIDE 37

Contents

Introduction Project Description Vision and Ambition Statistical Analysis Analysis of User Traces User life-time length Future Work Future Apps Socio-technical Predicted Legacy

slide-38
SLIDE 38

Friend’s Phones Now

Figure : “Friends phones now” sketch.

slide-39
SLIDE 39

Stereotypes

Experiments that create specific socio-technical environments and ask humans to enter them. (Milner’s ubicomp “vision” 2006)

  • How do we attach labels to the latent behavioural

semantics.

  • How will people behave if these labels are made

public.

slide-40
SLIDE 40

Contents

Introduction Project Description Vision and Ambition Statistical Analysis Analysis of User Traces User life-time length Future Work Future Apps Socio-technical Predicted Legacy

slide-41
SLIDE 41

Statistical Modelling

User models Software models

Data Logging

SGLog SGTracker

SGTracker

Adaptive logging infrastructure.

slide-42
SLIDE 42

Statistical Modelling

User models Software models Visual tools

App Development Data Logging

SGViz-2.0

Sophisticated visualisation tools.

slide-43
SLIDE 43

Statistical Modelling

User models Software models

Formal Modelling

Population models

App Development Data Logging

Population Models

Treatment of class as stochastic.

slide-44
SLIDE 44

Last slide

Thank you.