Pervasive Displays Reto Achermann acreto@student.ethz.ch March 12, - - PowerPoint PPT Presentation

▶
pervasive displays
SMART_READER_LITE
LIVE PREVIEW

Pervasive Displays Reto Achermann acreto@student.ethz.ch March 12, - - PowerPoint PPT Presentation

Aug 10, 2023 10 likes •484 views

Pervasive Displays Reto Achermann acreto@student.ethz.ch March 12, 2013 Distributed Systems Seminar 1 March 12, 2013 PERVASIVE DISPLAYS PART I: Distributed Systems Seminar http://www.1815.ch/images/4419_1.jpg 2

slide-1
SLIDE 1

Pervasive Displays

Reto Achermann

acreto@student.ethz.ch

March 12, 2013 1 Distributed Systems Seminar

slide-2
SLIDE 2

PART I: PERVASIVE DISPLAYS

March 12, 2013 2 Distributed Systems Seminar http://www.1815.ch/images/4419_1.jpg http://www.google.com/sync/images/sync-contacts.png http://www.google.com/glass/start/what-it-does/ http://g-ecx.images-amazon.com/images/G/01/electronics/detail-page/B004TT01JS-cloudl.jpg

slide-3
SLIDE 3

Pervasive Displays

  • Just “screens everywhere” ?

March 12, 2013 3 Distributed Systems Seminar http://www.timessquarenyc.org/image.aspx?id=1838&width=1370&height=870

Not really…

slide-4
SLIDE 4

Pervasive Displays

  • Pervasive
  • “spreading widely throughout an area or a group of people”
  • Display
  • “electronic device for visual presentation”
  • “printing the arrangement and choice of type in a style intended to

attract attention”

  • “a collection of objects for public viewing”
  • Oxford English Dictionary

March 12, 2013 4 Distributed Systems Seminar

slide-5
SLIDE 5

Pervasive / Ubiquitous Computing

  • Pervasive displays as part of pervasive / ubiquitous

computing

  • Principles of Ubiquitous Computing
  • “The purpose of a computer is to help you do something else.”
  • “The best computer is a quiet, invisible servant.”
  • “The more you can do by intuition

the smarter you are; the computer should extend your unconscious.”

  • “Technology should create calm.”

March 12, 2013 5 Distributed Systems Seminar http://en.wikipedia.org/wiki/File:Mark_weiser.jpg

  • Mark Weiser

(1952-1999)

slide-6
SLIDE 6

“Today’s public display systems are largely closed and isolated, with tightly controlled screen access.” Analogy: Old mobile phones – current public display systems

  • Nigel Davies et al.

March 12, 2013 6 Distributed Systems Seminar

slide-7
SLIDE 7

In the early days…

March 12, 2013 7 Distributed Systems Seminar http://4.bp.blogspot.com/_UtUFvF_vRlo/SFAlhV1cwXI/AAAAAAAAAUU/G5GpdArwqtg/s1600/1904+coke+ad.jpg http://store.ushistory.org/images/medium/want-dalton.jpg

Public Information Advertisement

slide-8
SLIDE 8

… and today

March 12, 2013 8 Distributed Systems Seminar http://www.1815.ch/images/4419_1.jpg http://blog.hslu.ch/outofhomedisplays/wp-content/blogs.dir/183/files/point-of-transportation/apg_epanel_zuerich_2.jpg

Information / News Display Electronic Advertising

  • Information presented to the

user interleaved with ads

  • Content is very repetitive:

User gets bored

  • No interaction or possibility to

change content

  • Showed content may be

moving / flickering to attract the attendance

slide-9
SLIDE 9

… and today

March 12, 2013 9 Distributed Systems Seminar http://upload.wikimedia.org/wikipedia/commons/thumb/7/7b http://www.hersche.at/uploads/plakat_appenzeller_kalte_fusse_gr10_zugeschn_korr.jpg

Poster Advertising Augmentation

  • Interaction needs application

(tag reader) or sending SMS

  • Augmentation is sometimes

inaccurate / impossible without location service

  • Maps just show “Here are you”

indicator (hard to find)

slide-10
SLIDE 10

Display Types

  • Static Displays:
  • Cannot change the content

shown

  • Provide no direct feedback
  • Cheap (big surface)
  • Require no energy
  • Posters, Maps, Objects
  • Dynamic Displays:
  • Can change content shown

dynamically

  • Can provide direct feedback
  • Relatively expensive
  • Require energy and actively

driven input

  • LCD Screens, Projectors

March 12, 2013 10 Distributed Systems Seminar http://www.1815.ch/images/4419_1.jpg http://www.hersche.at/uploads/plakat_appenzeller_kalte_fusse_gr10_zugeschn_korr.jpg

Both may be location / environment aware

slide-11
SLIDE 11

Are today’s displays pervasive?

  • “Yes” in the sense of Oxford dictionary
  • No in the sense of ubiquitous computing
  • People have adapted to ignore the displays
  • Fast moving pictures distract calmness
  • Almost no interaction / non-intuitive interaction
  • Interaction / augmented reality is “plug-and-play”

instead of “arrive-and-operate”

  • Displays provide no / too less help in

doing something

March 12, 2013 11 Distributed Systems Seminar http://www.beobachter.ch/typo3temp/pics/SBB-Automaten_d5c70a1bcd.jpg http://1.bp.blogspot.com/ http://www.timessquarenyc.org/image.aspx?id=1838&width=1370&height=870

slide-12
SLIDE 12

Observation: Smartphones are everywhere

“The first truly pervasively available interaction devices”

  • Robert Hardy

12 http://www.springwise.com/img/uploads/2012/09/dandeacon.jpg

slide-13
SLIDE 13

Observation: Smartphones are everywhere

  • Equipped with various sensors, radio interfaces, acoustic &

haptic feedback and touch displays

  • Problem: Interaction with displays using smart phones

today is inconvenient, not always working and clumsy

  • Idea: use of technology provided by the smartphone in a

smart way for interaction

March 12, 2013 13 Distributed Systems Seminar

slide-14
SLIDE 14

The vision in the movies

March 12, 2013 14 Distributed Systems Seminar http://jordster4000.blogspot.ch/2011/09/layout-design-research.html http://www.bitrebels.com/wp-content/uploads/2010/12/Kinect-Minority-Report-UI-2.jpg

Minority Report, DreamWorks Pictures Avatar 20th Century Fox

Visions

slide-15
SLIDE 15

Interactive Street Map

  • Intuitively select an element on the map
  • Get related information of selected object
  • Query / Filter:

Get list of all restaurants in a specific area

  • Show directions to object

March 12, 2013 15 Distributed Systems Seminar http://www.pre.ethz.ch/address/

Visions

slide-16
SLIDE 16

Interactive Mensa Menu

  • Show additional information:
  • Menu ratings
  • Alternative Menus at other location
  • Filter according to preferences
  • Example: Diet help
  • Caloric values directly added to diary
  • Do not show menu which do not fit
  • Example: Allergic / Health
  • Hide menus containing certain ingredients
  • Show warnings

March 12, 2013 16 Distributed Systems Seminar http://a5.mzstatic.com/us/r1000/103/Purple/ef/b1/26/mzl.jlmdxofp.320x480-75.jpg

Visions

slide-17
SLIDE 17

Passive Interaction with Displays

  • No need for taking the phone out of the pocket
  • Walk-by / be present as interaction event
  • Display spontaneously react to your presence and shows

content you like (pictures, videos) Privacy?

  • Example: Screen at CAB Foyer
  • You like to see images of D-INFK

events

  • Your phone tells the screen wirelessly
  • Screen displays selection of photos

March 12, 2013 17 Distributed Systems Seminar

Visions

slide-18
SLIDE 18

Personalized Content

  • Content is chosen according to personal preferences
  • Switch paintings of e-gallery
  • Show news you’re interested in / weather of your location
  • Show tailored advertisement
  • Showing related information to query
  • Several social issues
  • Example: Team Support
  • A group of soccer fans meet
  • The displays show the colors of

the favorite team

March 12, 2013 18 Distributed Systems Seminar www.google.com http://i.telegraph.co.uk/multimedia/archive/01655/flags-street1_1655777i.jpgc

Visions

slide-19
SLIDE 19

Augmented Displays

  • A tablet’s display augment the view of the current display
  • Different angle / layer
  • Annotations
  • Show additional information of the elements shown
  • Example: Doctors
  • Different experts analyze a patient
  • Cardiologist
  • Sinologist
  • Surgeon
  • All have different interests concerning

the patient’s health state

March 12, 2013 19 Distributed Systems Seminar http://jordster4000.blogspot.ch/2011/09/layout-design-research.html

Visions

slide-20
SLIDE 20

Houses as Displays

March 12, 2013 20 Distributed Systems Seminar

  • Using the façade of a house as a game display

Boring et al 2011 http://troyh.us/photos/City%20Skylines%20&%20Downtown/13704%20Dallas%20skyline%20at%20night.jpg

Visions

slide-21
SLIDE 21

Projected Displays

March 12, 2013 21 Distributed Systems Seminar

Visions

  • Video:
  • http://www.youtube.com/watch?feature=player_embedded

&v=df1NO7MoAUY

slide-22
SLIDE 22

PART II INTERACTION WITH DISPLAYS

March 12, 2013 22 Distributed Systems Seminar http://farm4.staticflickr.com/3259/3221222512_723ed0e083_z.jpg?zz=1 http://b.vimeocdn.com/ts/356/389/356389029_640.jpg

slide-23
SLIDE 23
  • 1. Physical Buttons
  • Select elements by pressing buttons
  • Buttons can be marked to distinguish them blindly
  • Drawbacks:
  • Not always clear what different buttons do
  • Unclear element highlighting
  • No button adaption to different content
  • Button interface is not extensible

March 12, 2013 23 Distributed Systems Seminar http://preview.turbosquid.com/Preview/2011/09/

Physical Buttons

slide-24
SLIDE 24
  • 2. Touch Screens
  • Interact by touching the element directly on the screen
  • More intuitive than physical buttons, adaptable interface
  • Drawbacks:
  • Indistinguishable buttons (no blind navigation)
  • Dirty displays
  • Not well suited for very large screens
  • Not working with
  • Displays behind security glass
  • Displays far away
  • Wet fingers / gloves

March 12, 2013 24 Distributed Systems Seminar https://activecaptain.com/articles/mobilePhones/iPhone/iPhone_Keyboard.jpg

Touch Screens

slide-25
SLIDE 25
  • 3. Bluetooth Device Names for Interaction
  • Idea: send service requests to displays wirelessly
  • Issues tackled:
  • SMS: untrusted number (premium service), need to know display ID
  • App: download necessary (inconvenient)
  • Touching may not be possible
  • Observation: many users have device supporting

Bluetooth and have set a custom USB device names

  • Approach: Use of Bluetooth device discovery and

Bluetooth device names to send requests

March 12, 2013 25 Distributed Systems Seminar

Bluetooth

[Nigel Davis et al]

slide-26
SLIDE 26

Bluetooth Device Names as Commands

  • No need for additional software
  • Users set the device name to a special command string, to

send a request command to the system

  • Identifier followed by service name
  • Examples:

March 12, 2013 26 Distributed Systems Seminar

Bluetooth

ec <service_name> <params> ec youtube: eth ec map: CAB H52

slide-27
SLIDE 27

Scanning Area

Principle

March 12, 2013 27 Distributed Systems Seminar http://en.wikipedia.org/wiki/File:BluetoothLogo.svg

Display Smart Phone Smart Phone Smart Phone Smart Phone Display Machine

Problem: reading the device ID takes time discovery happens too early / late

Bluetooth

slide-28
SLIDE 28

Serving the Requests

  • Each request (recognized command) is put into a queue
  • Each request is served for a maximum specified time
  • If a user leaves the display area, the requests is marked as

served

  • Social issues arise

ec youtube: puke

March 12, 2013 28 Distributed Systems Seminar

Bluetooth

slide-29
SLIDE 29
  • 4. Visual Markers
  • Using special patterns for interaction
  • Identify the display / element by

special marker (QR Code / edges)

  • Touch like interaction at distance

March 12, 2013 29 Distributed Systems Seminar

[Ballagas et al. - Point & Shoot (2005)]

http://www.idea.org/blog/wp-content/uploads/2011/09/qr-code-01.jpg

Visual Markers

slide-30
SLIDE 30
  • 5. Mobile Interaction with NFC enabled displays
  • Near field Communication
  • Wireless communication technology, Point-to-Point
  • Small Range: < 0.2m
  • Frequency: 13.56 MHz
  • Bandwidth: 424 MHz
  • Set-up time: <0.1s
  • Low-Power, tag is unpowered
  • Applications
  • Payments (Credit Cards, …)
  • Keys
  • Data Exchange (Business Cards)

March 12, 2013 30 Distributed Systems Seminar http://askbobrankin.com/google-wallet-nfc.jpg http://en.wikipedia.org/wiki/File:NFC-N-Mark-Logo.png

NFC

slide-31
SLIDE 31

NFC enabled displays

  • A mesh of NFC tags attached to the

rear, no need for visual markers

  • Interaction by “touching” i.e. holding

the device close

  • Mobile device reads content of the

NFC tag:

  • Object ID to look up on internet
  • Self contained information (no lookup)
  • Mobile device may augment the

display by showing information

March 12, 2013 31 Distributed Systems Seminar

Robert Hardy, Enrico Rukzio, Paul Holleis, Matthias Wagner Mobile interaction with static and dynamic NFC-based displays

NFC

slide-32
SLIDE 32

Resolution Problem of NFC Enabled Displays

  • Resolution Problem: Mesh grid of NFC tags is coarse.

One tag covers many elements

  • Dynamic Solution:
  • “Zoom in”: Show pop up
  • Static Solution:
  • Show list on mobile phone
  • Generic Approach: Filtering
  • Enable filter to reduce the

result set

March 12, 2013 32 Distributed Systems Seminar http://www.pre.ethz.ch/address/

NFC Tag Area Filters Zoom In NFC

slide-33
SLIDE 33
  • 6. Accelerometers: PhoneTouch
  • Goal: Distinguish multiple users interacting with a touch

screen at the same time

  • Touchscreen registers location of touch event
  • Phone registers movement

using accelerometer

  • Tabletop server matches touch

event with phone movement to identify user

March 12, 2013 33 Distributed Systems Seminar

Accelerometers

PhoneTouch: A Technique for Direct Phone Interaction on Surfaces

Dominik Schmidt, Fadi Chehimi, Enrico Rukzio, Hans Gellersen Computing Department, Lancaster University, Lancaster, UK

slide-34
SLIDE 34
  • 7. Touch Projector: Touch Screen from Distance
  • Observation:
  • Not all display support touch input
  • Displays may be out of arm’s reach
  • Elements cannot be moved between displays easily
  • Idea: interaction through live video

March 12, 2013 34 Distributed Systems Seminar

Sebastian Boring, Dominikus Baur, Andreas Butz, Sean Gustafson, Patrick Baudisch TouchProjector: Mobile interaction through video

Touch Projector

slide-35
SLIDE 35

Touch Projector Usage

  • All touch events are routed through

a server (the environment manager)

  • Basic Usage:
  • User points at display and touches element
  • User moves element within / between screens
  • User releases touch and element is placed on new location

March 12, 2013 35 Distributed Systems Seminar

Touch Projector

slide-36
SLIDE 36

Touch Projector Implementation Issues

  • Screen too small on mobile device: Automatic zoom in

when display is recognized

  • Device needs to be pointed on screen: Freeze image on

mobile device for stability and fine tuning

  • Display identification based on computer vision may result

in incorrect identification

March 12, 2013 36 Distributed Systems Seminar

Touch Projector

slide-37
SLIDE 37
  • 8. Augmented Displays
  • Problem:

One screen – different people and different interests

  • Idea: Augment main display with different views / layers

March 12, 2013 37 Distributed Systems Seminar http://qvectors.net/downloads/images/fullpreview/vector-Business-Presentation_full.jpg

Augmentation

slide-38
SLIDE 38

The Magic Lens Metaphor

  • Show alternate view of the data
  • Show annotations / information on the object

March 12, 2013 38 Distributed Systems Seminar

Augmentation

slide-39
SLIDE 39
  • 7. Gesture Recognition using Kinect
  • Improved hygiene: no need to touch

March 12, 2013 39 Distributed Systems Seminar http://www.digitaltrends.com/computing/kinect/

Gesture

slide-40
SLIDE 40

SOCIAL ACCEPTANCE ISSUES

Security Content Privacy

March 12, 2013 40 Distributed Systems Seminar http://gigaom2.files.wordpress.com/2012/06/comscore-facebook.png

slide-41
SLIDE 41

Security: SMS / QR Codes

  • Sending an SMS can end in a premium service
  • QR codes may refer to phishing sites
  • QR codes / SMS numbers may be forged

March 12, 2013 41 Distributed Systems Seminar http://mobhappy.com/blog1/wp-content/uploads/2006/03//sweetrelief.jpg http://1.bp.blogspot.com/http://www.t-online.de/handy/smartphone/id_61301376/falsche-qr-codes- experten-warnen-vor-smartphone-phishing.html

Social

slide-42
SLIDE 42

Privacy Concerns: Unveiling Personal Data

  • Personalized display content unveil personal data
  • e.g. advertisement of products recently bought
  • e.g. support for political party / sports team
  • User may need to fill in his preferences
  • Scanning QR code / sending SMS

unveils phone ID

  • Technology may be used to

track the user where he goes

March 12, 2013 42 Distributed Systems Seminar http://medaidpharmacy.com/wp-content/uploads/2011/05/top-logo_small3.png http://www.masterfile.com/stock-photography/image/400-05059664/speaker-or-film-director-and- screen---cartoon-style

Social

slide-43
SLIDE 43

Effort Needed to Use the Displays

  • Must be kept as low as possible
  • No plug-and-play but arrive-and-operate:

the use has to be intuitive

  • Effort needed is percept as cost

March 12, 2013 43 Distributed Systems Seminar http://helpdesk.nex-tech.com/print.php?id=657 http://tutznet.com/2966-fix-google-play-store-error-403-android/

Social

slide-44
SLIDE 44

Content

  • User decides what to display
  • May be inappropriate

(e.g. offensive views)

  • May distract other people

(e.g. music styles)

  • The content providers must be trusted
  • Not all display owners want all contents on their display
  • Concept of “Trusted Store” like an app store
  • Content war between users

March 12, 2013 44 Distributed Systems Seminar http://www.20min.ch/schweiz/news/story/18185935

Sunrise Demonstration Incident

Social

slide-45
SLIDE 45

Summary of Today

  • Today’s displays are not really pervasive (not helpful)
  • Intuitive interaction with displays needed to turn them into

helping assistants

  • There are many technical & social issues to solve
  • Most of the basic

technology is already available

45 http://www.google.com/glass/start/

slide-46
SLIDE 46

THANKS Questions ?

March 12, 2013 46 Distributed Systems Seminar http://podcasts.insitewebsitedesign.com/intersect-video-wall-sequence-from-chuck-HD-stills-04.jpg

slide-47
SLIDE 47

References

  • Nigel Davies, Marc Langheinrich, Rui Jose, Albrecht Schmidt

Open Display Networks: A Communications Medium for the 21st Century IEEE Computer, Volume 45, Issue 5, Pages 58-64, 2012.

  • Florian Alt et al.

Designing Shared Public Display Networks: Implications from Today's Paper-Based Notice Areas Proceedings of the 9th international conference on Pervasive computing (Pervasive'11), San Francisco, CA, USA, 2011.

  • Sebastian Boring, Dominikus Baur, Andreas Butz, Sean Gustafson, Patrick Baudisch

TouchProjector: Mobile interaction through video Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'10), Atlanta, USA, 2010.

  • Robert Hardy, Enrico Rukzio, Paul Holleis, Matthias Wagner

Mobile interaction with static and dynamic NFC-based displays Proceedings of the 12th international conference on Human computer interaction with mobile devices and services (MobileHCI'10), Pages 123-132, Lisboa, Portugal, 2010.

  • Claudio S. Pinhanez

The Everywhere Displays Projector Proceedings of the International Conference on Ubiquitous Computing (UbiComp'01), Pages 215-331, Atlanta, USA, 2001.

  • Gabor Soros, Peter Rautek, Hartmut Seichter, Eduard Groller

Augmented visualization with natural feature tracking Proceedings of the 10th ACM Conference on Mobile and Ubiquitous Multimedia (MUM'11), Beijing, China, 2011.

  • John Hardy, Jason Alexander

Toolkit Support for Interacitve Projected Displays, MUM’12

  • Dominik Schmidt, Fadi Chehimi, Enrico Rukzio, Hans Gellersen

PhoneTouch: A technique for Direct Phone Interaction on Surfaces

  • Sebastian Boring, Sven Gehring, Alexander Wiethoff, Magdalena Blöckner, Johannes Schöning, Andreas Butz

Multi-User Interaction on Media Facades Through Live Video on Mobile Devices CHI’11

47