CloakingNote: A Novel Desktop Interface for Subtle Writing Using - - PowerPoint PPT Presentation

▶
cloakingnote
SMART_READER_LITE
LIVE PREVIEW

CloakingNote: A Novel Desktop Interface for Subtle Writing Using - - PowerPoint PPT Presentation

Feb 18, 2023 292 likes •731 views

CloakingNote: A Novel Desktop Interface for Subtle Writing Using Decoy Texts Sehi L'Yi 1 Kyle Koh 1 Jaemin Jo 1 Bohyoung Kim 2 Jinwook Seo 1 Human-Computer Interaction Laboratory, Seoul National University , Seoul, South Korea 2 Division of

slide-1
SLIDE 1

CloakingNote:

A Novel Desktop Interface for Subtle Writing Using Decoy Texts

Sehi L'Yi1 Kyle Koh1 Jaemin Jo1 Bohyoung Kim2 Jinwook Seo1

¹ Human-Computer Interaction Laboratory, Seoul National University, Seoul, South Korea

2 Division of Biomedical Engineering, Hankuk University of Foreign Studies, Yongin-si, South Korea

slide-2
SLIDE 2

Writing Personal Texts in Offices

2

Social Issues

X

1 new message

shlyi@hcil.snu.ac.kr

X

new messages arrived!

Colleague Boss

slide-3
SLIDE 3

Writing Personal Texts in Offices

3

Privacy Issues

slide-4
SLIDE 4

Subtle Input Interaction Methods Shoulder-Surfing Resistant Input Interfaces

Supporting Subtlety with Deceptive Devices and Illusory Interactions

  • Anderson et al.

Nenya: Subtle and Eyes-free Mobile Input with a Magnetically- tracked Finger Ring

  • Ashbrook et al.

Intimate Interfaces in Action: Assessing the Usability and Subtlety

  • f EMG-based Motionless Gestures
  • Costanza et al.

Using Fake Cursors to Secure On-Screen Password Entry

  • De Luca et al.

ColorPIN: Securing PIN Entry through Indirect Input

  • De Luca et al.

Vibrapass: Secure Authentication Based on Shared Lies

  • De Luca et al.

Writing performance is largely limited 
 by the use of gestural inputs Suitable only for acquiring short inputs from users
 such as passwords and PIN numbers

slide-5
SLIDE 5

CloakingNote

Is my presentation going well?

A novel desktop interface for subtle writing using decoy texts

slide-6
SLIDE 6

Main Idea from Magic

  • Misdirection*

6

* Kuhn et al. Misdirection in Magic: Implications for the Relationship Between Eye Gaze and Attention ➡ Misdirection from the previous slide

IMAGE - The Art of Misdirection

  • Drawing audiences’ attention away from magicians’ tricks
slide-7
SLIDE 7

CloakingNote

Is my presentation going well?

A novel desktop interface for subtle writing

slide-8
SLIDE 8

CloakingNote

Is my presentation going well?

A novel desktop interface for subtle writing using decoy texts Decoy Text Real Text

slide-9
SLIDE 9
slide-10
SLIDE 10

CloakingNote Design

10

Main Interface of CloakingNote

  • Based on Design Guidelines

for Subtle Interfaces*

* Anderson et al. Supporting Subtlety with Deceptive Devices and Illusory Interactions

  • Simulation and Dissimulation
  • Separating Cause and Effect
  • Modularity
  • User Customization
slide-11
SLIDE 11

11

* Anderson et al. Supporting Subtlety with Deceptive Devices and Illusory Interactions

Modularity

➡ Alter the appearance of

secret interaction to maintain its subtlety User Customization

➡ Let users customize it

according to their contexts

CloakingNote Supports Various Configurations
 for Real & Decoy Texts

Details are in our paper

slide-12
SLIDE 12

Supporting Modularity & User Customization*

12

* Anderson et al. Supporting Subtlety with Deceptive Devices and Illusory Interactions

Configuration window 
 for decoy text Configuration window 
 for real text

slide-13
SLIDE 13

13

Supporting Modularity & User Customization*

* Anderson et al. Supporting Subtlety with Deceptive Devices and Illusory Interactions

slide-14
SLIDE 14

Supporting Modularity & User Customization*

14

* Anderson et al. Supporting Subtlety with Deceptive Devices and Illusory Interactions

Configuring a decoy text 1. Choose a writing interface 2. Customize texts to make it suitable for the interface 3. When start writing, the configuration window becomes invisible

slide-15
SLIDE 15

CONFIGURATION EXAMPLES BY USERS

slide-16
SLIDE 16
slide-17
SLIDE 17
slide-18
SLIDE 18

SUBTLETY TEST

slide-19
SLIDE 19

Study Design

  • Between-subject factor: Contrast
  • 36 participants (12 female)
  • Normal or corrected-to-


normal vision (20/25)

  • no color-blindness

19

Participant Actor

slide-20
SLIDE 20

Contrast between the real text and its background

20

Real Text 
 Using
 Low-contrast


(1.5:1)

Real Text 
 Using
 Mid-contrast


(3:1)

Real Text 
 Using
 High-contrast


(11:1)

Optimal Ratio for Visual Acuity* Minimum Ratio for Proper Display Quality* Lower Than The 
 Minimum Ratio

* Lin et al. Effects of Viewing Angle and Contrast Ratio on Visual Performance using TFT-LCD * Snyder et al. Image Quality. In: M. Helander (Ed.), Handbook of Human–Computer Interaction.

slide-21
SLIDE 21

Screen Layout In Subtlety Test

21

slide-22
SLIDE 22

Procedure

  • In each session (30 seconds)
  • A participant entered a room and stand behind the actor’s right shoulder
  • Carried out two tasks
  • 1. Ask a question which was related to the decoy text
  • e.g., “What are you writing about?”
  • 2. Observe what the actor was doing
  • Report to experimenter
  • Repeated the session until the participant either 1) noticed what the actor

was really doing or 2) completed nine sessions

22

slide-23
SLIDE 23

Results - Number of Participants Who Had Not Noticed

23

12 4 2

2 4 6 8 10 12

low- contrast mid- contrast high- contrast

# of Participants Who Had Not Found

➡ The results suggested that CloakingNote successfully

hides the content of secret writing as well as the use of the interface itself

slide-24
SLIDE 24

apple|

Difference between key pressed and input decoy letter

real_| Huma|

Decoy text can unnaturally stop in the middle of the word I tried to compare between key pressed and the letter appear on the screen, but it was impossible .…

P25

How Finders Noticed The Use of CloakingNote

24

slide-25
SLIDE 25

➡

Visual subtlety of the real text is much more important!

How Finders Noticed The Use of CloakingNote

25

Just recognized the real text being rendered in an unexpected place

16 out of 18

slide-26
SLIDE 26

PERFORMANCE TEST

slide-27
SLIDE 27

Study Design

  • 1. Quantitative evaluation
  • 1. Writing phrases using different contrast ratios
  • 2. Qualitative evaluation
  • 1. Using CloakingNote while in 


shoulder-surfing risks

  • 18 participants (six female)

27

Actor

(Only in Qualitative Evaluation)

Participant

slide-28
SLIDE 28

Participant

Study Design - Quantitative Evaluation (First Session)

  • Dependent variable
  • Typing Speed: Character per second (CPS)
  • Accuracy: Character error rate (CER) using Damerau-Levenshtein

distance

  • Within-subjects factor
  • Configure (low-, mid-, high-


contrast & baseline)

28

slide-29
SLIDE 29

Screen Layout In Performance Test

29

video camera with a zoom lens video camera with a zoom lens

Low-, Mid-, High-contrast

video camera with a zoom lens

video camera with a zoom lens

Baseline

slide-30
SLIDE 30
  • Writing a short essay (i.e., TOEFL writing topics) for five minutes

while an actor peeked over their shoulder

  • For 10, 20, and 30 secs at the 0.5-, 1-, and 2-minute marks
  • Before writing, participants freely 


changed layout of on-screen 
 components Procedure - Qualitative Evaluation (Second Session)

30

Actor

(Only in Qualitative Evaluation)

Participant

slide-31
SLIDE 31

Results - Speed & Accuracy

➡ CloakingNote supports reasonable writing performance

compare to the use of gestural inputs

31

5.1 5.3 5.4 5.5

1 2 3 4 5 6 7

low- contrast mid- contrast high- contrast baseline

Character Per Second (CPS)

* *

<Typing Speed> <Accuracy>

slide-32
SLIDE 32

USERS’ LAYOUT STRATEGY

slide-33
SLIDE 33

Foreground-match Strategy

33

slide-34
SLIDE 34

Background-match Strategy

34

slide-35
SLIDE 35

35

Distractive Strategy

“I supposed that the

  • bserver wouldn’t ever

noticed my [real text].”

P49

slide-36
SLIDE 36

EXTENDING CLOAKINGNOTE

slide-37
SLIDE 37

37

Extending CloakingNote - Subtle Video Watching

slide-38
SLIDE 38

38

* De Luca et al. I Feel Like I'm Taking Selfies All Day!: Towards Understanding Biometric Authentication on Smartphones

Extending CloakingNote - FaceUnlock

slide-39
SLIDE 39

SUMMARY

1. We designed CloakingNote, a novel desktop interface for subtle writing using decoy texts 2. User studies showed that CloakingNote was effective in hiding the real text as well as the secret action and could be used with reasonable writing performance 3. We categorized and discussed the users’ layout strategies and introduced possible extensions of CloakingNote

39

Contact

shlyi@hcil.snu.ac.kr

slide-40
SLIDE 40

40

APPENDIX

slide-41
SLIDE 41

APPENDIX: Knowledge Levels

41

  • Diverse observation purposes in the real-world

1st - 3rd
 sessions 4th - 6th
 sessions 7th - 9th
 sessions

Suspicion-knowledge level

  • Look out for any suspicious activity

No-knowledge level

  • Observe the actor without any knowledge

Awareness-knowledge level

  • Look out for other activity than writing a text on a

word processor

slide-42
SLIDE 42

APPENDIX: Accumulated Number of Finders

  • Few participants (11%) identified the real text during the first three sessions
  • 21 participants (58%) did not identify the real text in suspicion-knowledge level
  • Five finders noticed the content of the real text

42

slide-43
SLIDE 43

APPENDIX: Foreground-Match Strategy in Mobile

43