What is a good pen based application? The windows desktop and - PDF document

What is a good pen based application? � The windows desktop and browser are HCI For Pen Based Computing NOT good pen based apps! Richard Anderson CSE 481 B Winter 2007 What is a good UI? How do you measure it? Mechanical Properties Keystroke level model Targeting � Analyze task by summing individual � Fundamental operation operation times � Moving a cursor to a specific location � Pointing and selection Moving hand to mouse 360 ms Pointing to a new line with mouse 1500 ms Clicking the mouse 230 ms Moving hand to keyboard 360 ms Total 2450 ms 1

W Experiment: measure time to A move cursor to target Index of difficulty W � How does T behave as a function of A T(A, W) Targeting time for (W fixed)? amplitude A and width W T A A � How does T behave as a function of W Cursor (A fixed)? T Target W Fitts’ Law Interpretation of Fitt’s Law � A task’s movement difficulty is given by � Scale invariance ID = log 2 (A / W) � Dependence on A/W � ID – index of difficulty � Exponential targeting � A – amplitude of the move � Log factor – as in binary search � W – width of the target region � T = a + b ID Menu design Low level mechanisms � What can you say about the cost of � State machine model accessing items in the following menu X � Registers � Cursor is at the top of the menu Y � X, Y, Pressure � Pen state Pres � Down, Hover, Out-of- range � Pen button � Up, Down � Polling model 2

Control primitives Mode Problem � Cognitive difficulties in remembering / keeping track � Hover of modes � Tap � Which mode? � Double Tap � Remapping operations � Retaining mode across context switch � Press-and-hold � But modes are very useful � Hold-through � Efficient use of limited input controls � Not all modes are the same � Drag � Shift key vs. Caps Lock � Hold-drag � Mouse move vs. mouse drag � Pen color Do cars have modes? Pen mode solutions � A system has modes if it has states where the � Problem: How do you allow different controls have different functions. operations with a pen � Do cars have modes? If so, give an example � Ink vs. erasing � Explicit modes � Ink vs. gesture � Recognition of gesture overrides ink � Ink vs. recognition vs. control � Area based modes Student Submission Pen mode study Yang Li et al., CHI 2005 � Barrel Button � Hold � Non-preferential hand button � Pressure � Eraser 3

How many distinct mechanisms can you construct to select between four choices Discrete selection with a pen? � Choose from a finite set � Command from a menu � Character from an alphabet � Repeated selection from finite sets � Hierarchical menus � Commands with arguments � Sequences of characters � Words � Multi-digit numbers Assume a pen without a button Selection mechanisms Basic pen operation � Crossing � Crossing � Pointing � Operation triggered by a stroke crossing a line segment � Writing � Tapping � Pressure CrossY: Crossing based UI Hierarchical crossing � Specify operations � Principle – multiple commands without by drawing through lifting the pen 4

Flow Menu Abstract writing � Use movement through octants for � Enter text with specialized, stroke based control information recognition � Optimized for automatic recognition � Not human readable Move Item 100% 100% � Character based or word based Text Item Highlight Zoom 66.6% 200% 66.6% 200% Shape 50% 75.0 400% 50% 100.0 400% 25% 800% 25% 800% Custom Custom Quikwrite [Perlin, NYU] Cirrus (Georgia Tech) c a r p interaction education Write helloworld Graffiti (Palm) 5

More Graffiti � Mostly single stroke � Close to standard alphabet (learnability) � Write only � Location written for additional meaning If the red circle is a selection Selection problem tool, what is selected? � Identify one or more graphical elements from a domain � Mechanisms � Bounding Region � Geometric defined by stroke � Distance from cursor Bubble cursor Recognition UIs � Selection radius depends on object � UIs based on attaching meaning to ink proximity � Gestures � Diagram recognition � Handwriting recognition � Free form � Constrained recognition 6

Gestures Gestures � Commands issued with a single stroke � Ambiguity � May be drawn or invisible � Distinction between gestures � Distinction between gesture and other ink � Support from SDK � Robustness � Register gestures to be recognized � Handling misrecognized gestures � UI Issues � False positive � Similar to keyboard short cuts � False negative � Speed up for experts � Gesture initiated actions should be undoable � Hard to learn / remember Diagram recognition Text recognition � Challenges to recognition � We will have a great lecture later in the course! � Even simple shapes are hard! � Basic approach � Variation in drawing � Ink artifacts � Collect a huge amount of data � Use data to train neural net Handwriting Recognition: Identify the following words Recognition results 7

Recognition scenarios Other details � What level of error is tolerable � Hands, obstructions, orientation � How is feedback provided to the user on recognition � How does the user specify corrections? Obstructions and handedness Screen orientation � Hand blocks the � Landscape vs. Portrait screen mode � Accommodate left � Surprisingly big and right difference in feel of applications handedness � Tablet PC requires rapid � Menu direction orientations switch � Context menus � Many standard desktop � Difficulties at the apps not designed for edge of the screen portrait mode 8