Intelligently Recommending Key Bindings on Physical Keyboards with - PowerPoint PPT Presentation

Intelligently Recommending Key Bindings on Physical Keyboards with Demonstrations in Emacs Shudan Zhong 1 Hong Xu 2 sdz@berkeley.edu, hongx@usc.edu March 18, 2019 1 University of California, Berkeley 2 University of Southern California the 24th International Conference on Intelligent User Interfaces (IUI) Marina del Ray, CA, USA

Executive Summary • Key bindings are the main means to interact with a computer with a physical keyboard. • Different key bindings can lead to drastically different user experience and productivity. • Surprisingly, research on recommender systems for key bindings are lacking. • We propose the fjrst (to the best of our knowledge) such recommender system and demonstrate it in Emacs. Zhong and Xu Recommending Key Bindings on Physical Keyboards 1 / 14

Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion Zhong and Xu Recommending Key Bindings on Physical Keyboards 2 / 14

Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion

What Are Key Bindings? • Key bindings defjne how users interact with computers via physical keyboards. • (Classic) examples: c v s • Applications defjne their own key bindings. Zhong and Xu Recommending Key Bindings on Physical Keyboards 3 / 14 • Copy: Ctrl + • Paste: Ctrl + • Save: Ctrl +

Key Bindings Are Important Key bindings play essential roles in many professional applications. (a) Programming (b) Computer-Aided Design (c) Professional Gaming Image sources: (a) http://blog.markpearl.co.za/Ultimate-Developer-Keyboard (b) https://allaboutcad.com/tutorial-create-a-command-alias-keyboard-shortcut/ (c) https://www.logitechg.com/en-us/products/gaming-mice/g600-mmo-gaming-mouse.html Zhong and Xu Recommending Key Bindings on Physical Keyboards 4 / 14

Key Bindings Are Important Different key bindings can lead to drastically different productivity and user experience. • An absurd example: for moving down and for moving up. Image source: https://i.redd.it/al15957ctaky.jpg Zhong and Xu Recommending Key Bindings on Physical Keyboards 5 / 14

Key Bindings Are Important Different key bindings can lead to drastically different user experience. • In browsers, Space is commonly used for scrolling one page down. Users can comfortably sit back and tap the spacious Space . Image source: https: //softwarerecs.stackexchange.com/questions/19804/browser-plugin-to-change-the-amount-of-scroll-on-space-key Zhong and Xu Recommending Key Bindings on Physical Keyboards 6 / 14

Key Bindings Are Important Different key bindings can lead to drastically different productivity and Recommending Key Bindings on Physical Keyboards Zhong and Xu (c) https://commons.wikimedia.org/wiki/File:KB_US-Colemak.svg (b) https://commons.wikimedia.org/wiki/File:KB_United_States_Dvorak.svg (a) https://commons.wikimedia.org/wiki/File:KB_United_States.svg Image source: v (c) Colemak (a) qwerty (b) Dvorak user experience. 7 / 14 Caps Lock Caps Lock Shift Shift Shift Shift Shift Shift Ctrl + z , Ctrl + x , Ctrl + c , Ctrl +

Intelligently Recommending Key Bindings • Although key bindings are important, surprisingly, research on applying recommender systems to key bindings on physical keyboards is lacking. • We propose the fjrst (to the best of our knowledge) recommender system that recommends key bindings. • We demonstrate such the recommender system in Emacs. Zhong and Xu Recommending Key Bindings on Physical Keyboards 8 / 14

Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion

Why Demonstrate in Emacs? • Key bindings matter in Emacs: It heavily exploits key bindings in its user interface. • Extensive customizability: • Emacs is a Lisp machine (Lisp interpreter + Lisp library + user interface). • Sequences of key strokes can be effectively bound to invoke virtually any existing/user-defjned Lisp function. • It allows sophisticated sequence of key strokes in key bindings, e.g., x Zhong and Xu Recommending Key Bindings on Physical Keyboards 9 / 14 Ctrl + u Ctrl + s .

Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion

Emacs Key Binding Recommender System (EKBRS) v Recommending Key Bindings on Physical Keyboards Zhong and Xu recommends recommends recommends combine 10 / 14 Lisp function describe-variable Suffjx Prefjx prior-word-suffjx posterior-word-suffjx word-suffjx module function-function module Ctrl + h

Emacs Key Binding Recommender System (EKBRS) Most help-related Lisp Lisp functions with “buffer” in their names are often bound by a key sequence with functions are bound by Example a key sequence with Zhong and Xu Recommending Key Bindings on Physical Keyboards prior-word-suffjx next-line / previous-line Key binding database Key binding database posterior-word-suffjx function-function Exploit relation- ship between suffjces, English words suffjces, English words Lisp functions, Lisp functions Relationship derived from Prior knowledge 11 / 14 is bound to Ctrl + n / Ctrl + p . prefjx Ctrl + h . b .

Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion

Setup • We used the default global key binding database in vanilla Emacs 26.1. • 391 key bindings after removing key bindings involving keys that are outside the main area of a qwerty keyboard. • Evaluation: • Leave-one-out strategy: In each round, we temporarily remove one key is recovered if the removed key binding is among the recommended key bindings. • We use the percentage of recovery as an evaluation metric. Zhong and Xu Recommending Key Bindings on Physical Keyboards 12 / 14 binding (for a Lisp function ℓ ) from the key binding database. • We let EKBRS recommend n = 1 . . . 5 key bindings for ℓ . The key binding

Results (a) Prefjxes Recommending Key Bindings on Physical Keyboards Zhong and Xu (c) Key bindings (b) Suffjces 13 / 14 0.5 Percentage of Recovery Percentage of Recovery dice prior-word-suffix Percentage of Recovery dice embedding-n posterior-word-suffix 1.0 0.8 embedding-n 0.43 0.4 0.41 embedding-s combined embedding-s 0.95 0.95 0.37 0.37 embedding-sd 0.93 0.67 embedding-sd 0.92 0.91 0.92 0.36 0.9 0.90 0.65 0.35 0.89 0.35 0.89 0.88 0.62 0.33 0.88 0.6 0.33 0.33 0.32 0.3 0.31 0.29 0.83 0.28 0.82 0.8 0.80 0.49 0.49 0.27 0.48 0.48 0.79 0.25 0.76 0.43 0.44 0.4 0.22 0.38 0.39 0.2 0.20 0.19 0.34 0.18 0.7 0.29 0.64 0.22 0.62 0.63 0.63 0.2 0.21 0.1 0.6 0.5 0.0 0.0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 n n n

Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion

Conclusion • Key bindings for physical keyboards often play critical roles in key bindings. • We proposed the fjrst (to the best of our knowledge) recommender system for key bindings, with a demonstration in Emacs. • Future work: Develop recommender systems for key bindings in other contexts. Zhong and Xu Recommending Key Bindings on Physical Keyboards 14 / 14 productivity and user experience. We need recommender systems for