SLIDE 1 Supporting Opportunistic Programmers with Better Visualizations
VISSOFT 2016 Joel Brandt Adobe Research
SLIDE 2 Creative Technologies Lab
§ Open or shared IP, ongoing collaborations with universities § Graphics, HCI, Vis, ML/AI, Programming Languages § 22 Researchers, 9 Research Engineers § ~70 academic publications per year at CHI, UIST,
SIGGRAPH, CVPR, ICCV, ICML, etc.
§ ~15 major contributions to Adobe products per year § 50-60 Ph.D.-student-level interns per year § Expect to grow by about 10 technical staff in 2017
SLIDE 3
research.adobe.com
SLIDE 4
“good grief, I don’t even remember the syntax for forms!”
SLIDE 5
SLIDE 6
SLIDE 7
SLIDE 8
SLIDE 9
SLIDE 10
SLIDE 11
SLIDE 12
SLIDE 13
SLIDE 14
SLIDE 15
SLIDE 16
SLIDE 17 subjects time (minutes)
[Brandt CHI 09]
Frequent Information Access
SLIDE 18 subjects time (minutes)
[Brandt CHI 09]
Frequent Information Access
SLIDE 19 Frequent Information Access
19% of time spent using Web (σ= 15.1 minutes) Used for 18 distinct tasks (σ= 9.1)
[Brandt CHI 2009]
SLIDE 20
Opportunistic Programmers
SLIDE 21 http://flickr.com/photos/royalsapien/2387707860/
Designers…
SLIDE 22 http://flickr.com/photos/niosh/2492023651/
Scientists…
SLIDE 23 http://www.flickr.com/photos/samthor/3198975900/
Makers…
SLIDE 24 By 2012:
13 million who program as part of their job… …but only 3 million who are professional programmers
[Scaffidi 2005]
SLIDE 25
Vast increase in information resources Democratization of access and distribution
SLIDE 26
How do we redesign the programming experience?
SLIDE 27 [Brandt CHI 2009]
just-in-time learning is common examples are important
SLIDE 28
Empirical Work Systems Building
SLIDE 29 Blueprint
Code Search in the Editor
[Brandt CHI 2010]
SLIDE 30
SLIDE 31
SLIDE 32
SLIDE 33
SLIDE 34
SLIDE 35
SLIDE 36
SLIDE 37
SLIDE 38
SLIDE 39
SLIDE 40
SLIDE 41
SLIDE 42
SLIDE 43
SLIDE 44
SLIDE 45
SLIDE 46 Write code faster in directed tasks…
Wilcoxon-Man-Whitney statistic = 2.38, p < .01,
57 vs. 121
seconds to first code paste
Spearman correlation coeff. = 0.52, p < .01
Task completion time strongly correlated with time of first paste
Results
SLIDE 47 …and the code is better!
worst best
blueprint control
Wilcoxon-Mann-Whitney statistic = 2.15, p < .02 Results
SLIDE 48 [Brutlag 2009, O’Hara 1998]
milliseconds matter
SLIDE 49 Codelets
Interactive Examples
[Oney CHI 2012]
In collaboration with Stephen Oney and Brad Myers at Carnegie Mellon University
SLIDE 50
SLIDE 51 User Study: Method
§ 20 participants § conducted remotely § between subjects
§ control group: pointed to jQM docs § treatment group: used exemplar examples
§ two parts (will only present part one)
51
SLIDE 52 User Study: Results
5 10 15 20 25 30
Time (m) # refreshes
Control Codelets
SLIDE 53 2 4 6 8 10 12
page list button page & button Time (m), lower is better Control Codelets
SLIDE 54 2 4 6 8 10 12
page list button page & button Time (m), lower is better Control Codelets
SLIDE 56 2 4 6 8 10 12
page list button page & button Time (m), lower is better Control Codelets
SLIDE 57
SLIDE 58 Codelets users
§ did not read accompanying text § had two usage patterns
§ leaving codelet open § closing codelet and then editing code
58
SLIDE 59 Codelets API
§ Editor exposes API § 3rd parties implement examples
§ Not implemented by editor
§ Uses HTML & JS
§ What most examples are already in
59
SLIDE 60
let’s implement one:
SLIDE 61 <codelet> <head> <title> Creating an int </title> <keywords> int </keywords> <type> block </type> <lang> c </lang> <script>$(function() { attach_input_to_mark($(“input#name_inp]”), “name” ); attach_input_to_mark($(“input#val_inp ]”), “value” ); });</script> </head> <body> <example> int <mark id=“name” /> = <mark id=“value” />; </example> <page> Name : <input type=“text” id=“name_inp” /> <br /> Value: <input type=“text” id=“val_inp” /> </page> </body> </codelet>
meta-information & interactivity (Javascript) example code & documentation (HTML)
SLIDE 62 <codelet> <head> <title> Creating an int </title> <keywords> int </keywords> <type> block </type> <lang> c </lang> <script>$(function() { attach_input_to_mark($(“input#name_inp]”), “name” ); attach_input_to_mark($(“input#val_inp ]”), “value” ); });</script> </head> <body> <example> int <mark id=“name” /> = <mark id=“value” />; </example> <page> Name : <input type=“text” id=“name_inp” /> <br /> Value: <input type=“text” id=“val_inp” /> </page> </body> </codelet>
example code & documentation (HTML) interactivity (Javascript) (meta-information)
SLIDE 63 <codelet> <head> <title> Creating an int </title> <keywords> int </keywords> <type> block </type> <lang> c </lang> <script>$(function() { attach_input_to_mark($(“input#name_inp]”), “name” ); attach_input_to_mark($(“input#val_inp ]”), “value” ); });</script> </head> <body> <example> int <mark id=“name” /> = <mark id=“value” />; </example> <page> Name : <input type=“text” id=“name_inp” /> <br /> Value: <input type=“text” id=“val_inp” /> </page> </body> </codelet>
interactivity (Javascript) documentation (HTML) (meta-information) (example code)
SLIDE 64 <codelet> <head> <title> Creating an int </title> <keywords> int </keywords> <type> block </type> <lang> c </lang> <script>$(function() { attach_input_to_mark($(“input#name_inp]”), “name” ); attach_input_to_mark($(“input#val_inp ]”), “value” ); });</script> </head> <body> <example> int <mark id=“name” /> = <mark id=“value” />; </example> <page> Name : <input type=“text” id=“name_inp” /> <br /> Value: <input type=“text” id=“val_inp” /> </page> </body> </codelet>
interactivity (Javascript) (meta-information) (example code) (documentation)
SLIDE 65 <codelet> <head> <title> Creating an int </title> <keywords> int </keywords> <type> block </type> <lang> c </lang> <script>$(function() { attach_input_to_mark($(“input#name_inp”), “name” ); attach_input_to_mark($(“input#val_inp ”), “value” ); });</script> </head> <body> <example> int <mark id=“name” /> = <mark id=“value” />; </example> <page> Name : <input type=“text” id=“name_inp” /> <br /> Value: <input type=“text” id=“val_inp” /> </page> </body> </codelet>
(meta-information) (example code) (documentation) (interactivity)
SLIDE 66 <codelet> <head> <title> Creating an int </title> <keywords> int </keywords> <type> block </type> <lang> c </lang> <script>$(function() { attach_input_to_mark($(“input#name_inp”), “name” ); attach_input_to_mark($(“input#val_inp ”), “value” ); });</script> </head> <body> <example> int <mark id=“name” /> = <mark id=“value” />; </example> <page> Name : <input type=“text” id=“name_inp” /> <br /> Value: <input type=“text” id=“val_inp” /> </page> </body> </codelet>
SLIDE 67 In the lab, participants complete tasks involving example code 43% faster Examples have a lifetime: they’re useful throughout the entirety of a project
Results
SLIDE 68 Theseus
Always-On JavaScript Debugging
[Lieber CHI 2014]
In collaboration with Thomas Lieber and Rob Miller at MIT
SLIDE 69 Programmers iterate rapidly
[Brandt IEEE Software 09]
SLIDE 70
SLIDE 71
SLIDE 72 Debugging is hard because user’s mental model is different than program logic Always-on visualizations help programmers build & maintain an accurate mental model
Results
SLIDE 73 Vesta
Using Runtime Traces to Improve Documentation and Testing
[Krämer CHI 2016]
In collaboration with Jan-Peter Krämer and Jan Borchers at RWTH Aachen University
SLIDE 74
SLIDE 75 All documentation created with Vesta was correct, compared to only 60% in control condition Vesta could generate significant portions of all unit tests, including tests that were written in control condition
Results
SLIDE 76
Blueprint Codelets Theseus Vesta
SLIDE 77
Growing importance on fast information access and just-in-time skill acquisition
SLIDE 78
In the future, knowledge work may become less about knowing how to do a task, and more about knowing how to find the right answer.
SLIDE 79
In the future, knowledge work may become less about knowing how to do a task, and more about knowing how to find the right answer.
SLIDE 80
How can the software visualization community help opportunistic programmers?
SLIDE 81
Build interfaces that assume the programmer knows nothing about the codebase
SLIDE 82
Help professionals build software that is easy for non-professions to augment
SLIDE 83 Make opportunistic programmers’ building blocks first-class citizens
- f their development tools
SLIDE 84
Focus collaboration tools on learning rather than building and maintaining software
SLIDE 85
Remember that you’re not designing for yourself
SLIDE 86
