SLIDE 1
Jeffrey P. Bigham @jeffbigham
Jeffrey P. Bigham Carnegie Mellon University
Deep Integration of Human and Machine Intelligence for Accessibility
Jeffrey P. Bigham
Carnegie Mellon University @jeffbigham
Jeffrey P. Bigham @jeffbigham
Deep Integration of Human and Machine Intelligence for Accessibility - - PowerPoint PPT Presentation
Deep Integration of Human and Machine Intelligence for Accessibility - - PowerPoint PPT Presentation
Deep Integration of Human and Machine Intelligence for Accessibility Jeffrey P. Bigham Carnegie Mellon University Jeffrey P. Bigham Carnegie Mellon University Jeffrey P. Bigham @jeffbigham @jeffbigham What do blind people want to know?
SLIDE 2
SLIDE 3
Jeffrey P. Bigham @jeffbigham
SLIDE 4
Jeffrey P. Bigham @jeffbigham
SLIDE 5
Jeffrey P. Bigham @jeffbigham
SLIDE 6
Jeffrey P. Bigham @jeffbigham
SLIDE 7
Jeffrey P. Bigham @jeffbigham
VizWiz Dataset
vizwiz.org/data
Check out Microsoft’s Seeing AI!
SLIDE 8
Jeffrey P. Bigham @jeffbigham
Using Devices
User + AI + Crowds
X
Bigham, Jeffrey, et al. "VizWiz: nearly real-time answers to visual questions." UIST 2010. Brady, Erin, et al. “Visual challenges in the everyday lives of blind people.” CHI 2013.
SLIDE 9
Jeffrey P. Bigham @jeffbigham
SLIDE 10
Jeffrey P. Bigham @jeffbigham
SLIDE 11
Jeffrey P. Bigham @jeffbigham
SLIDE 12
Jeffrey P. Bigham @jeffbigham
SLIDE 13
Jeffrey P. Bigham @jeffbigham
SLIDE 14
Jeffrey P. Bigham @jeffbigham
SLIDE 15
Jeffrey P. Bigham @jeffbigham
What do blind people want to know?
vizwiz.org
SLIDE 16
Jeffrey P. Bigham @jeffbigham
Working with AI
SLIDE 17
Jeffrey P. Bigham @jeffbigham
Supporting Blind Photography
Supporting Blind Photography. Jayant, Chandrika, Ji, Hanjie, White, Samuel, Bigham, Jeffrey P. ASSETS 2011.
Working with AI
SLIDE 18
Jeffrey P. Bigham @jeffbigham
Wu, et al. Automatic Alt-text: Computer-generated Image Descriptions for Blind Users on a Social Network Service. CSCW 2017.
Target What People Want to Know
Working with AI
SLIDE 19
Jeffrey P. Bigham @jeffbigham
Errors are Important and Not Created Equa
MacLeod, Bennett, Morris, Cuttrell, et al. Understanding Blind People’s Experiences with Computer-Generated Captions of Social Media Images. CHI 2017.
Measure and Express Confidence Well
Working with AI
SLIDE 20
Jeffrey P. Bigham @jeffbigham
* Amplify Users with AI Support (blind photograph * Solve Real Problems (understand user needs) * Embrace and Understand Errors (all errors are * Measure Confidence and Express it Well
Working with AI
SLIDE 21
Jeffrey P. Bigham @jeffbigham
The End
VizWiz Dataset: http://www.vizwiz.org/data VizWiz Publications: http://www.jeffreybigham.com
* Amplify Users with AI Support (blind photography) * Solve Real Problems (understand user needs) * Embrace and Understand Errors (all errors are not equal) * Measure Confidence and Express it Well
Funded by: Google, Microsoft, NSF, NIDILRR