Interacting w ith Smart Objects: Application Scenarios w ith the - - PowerPoint PPT Presentation

▶
interacting w ith smart objects application scenarios w
SMART_READER_LITE
LIVE PREVIEW

Interacting w ith Smart Objects: Application Scenarios w ith the - - PowerPoint PPT Presentation

Apr 05, 2023 255 likes •911 views

Interacting w ith Smart Objects: Application Scenarios w ith the BTnode Platform Friedemann Mattern ETH Zurich Institute for Pervasive Computing ET ETH Eidgenssische Dagstuhl March 2004 Technische Hochschule Zrich Friedemann

slide-1
SLIDE 1

Dagstuhl March 2004

Friedemann Mattern

ETH Zurich Institute for Pervasive Computing

ET ETH

Eidgenössische Technische Hochschule Zürich

Interacting w ith Smart Objects: Application Scenarios w ith the BTnode Platform

slide-2
SLIDE 2
  • F. Ma. 3

Friedemann Mattern

  • Smart-Its project (2001-2003)
  • part of the European

„Disappearing Computer“ initiative

Goal: enable everyday objects as smart

interconnected information artifacts

by attaching wireless sensors („Smart-Its“) to them

  • bjects become self aware, context sensitive, cooperative

Integration into background computing environment

Making Things Smart w ith Wireless Sensors

slide-3
SLIDE 3
  • F. Ma. 4

Friedemann Mattern

The NCCR „Mobile Information and Communication Systems“

  • National Center of Competence in Research

First phase until 2005, planned total time: 10 years Focus: decentralized, self-organizing mobile networks Sub-project IP9 “Communicating

Embedded Systems”

  • ne out of 11 projects

Jan Beutel, Kay Römer,

Roger Wattenhofer,…

BTnodes sensor node platform

slide-4
SLIDE 4
  • F. Ma. 6

Friedemann Mattern

BTnode – An Experimental Hardw are Platform

CPU: 8 bit RISC, 8 MIPS Generic sensor interfaces Communication: Bluetooth Lightweight event-driven OS Unit cost @ 200 units: $110

slide-5
SLIDE 5

8

Lightweight wireless communica Lightweight wireless communication and computing platform tion and computing platform based on a Bluetooth radio module and a mi based on a Bluetooth radio module and a microcontroller. crocontroller. Bluetooth has the advantage of Bluetooth has the advantage of

– availability today for experimentation – compatibility to interface to consumer appliances – an abstract, standardized high level digital interface

BTnode architecture

slide-6
SLIDE 6

9

61 m 61 mm 40 mm 40 mm

BTnode – Some Hardware Details

Integrated PIFA antenna LEDs, Reset, Clocks Communication Communication

Bluetooth module

Generic sensor interfaces UART and I2C data interfaces Power management Memory Memory

128 kB Flash, 244 kB SRAM, 4 kB EEPROM

CPU CPU

Atmel ATmega 128L MCU 8-Bit RISC (max. 8 MHz ~8MIPS)

12 cm3, 2 mW (low power) – 150 mW

slide-7
SLIDE 7

10

Lightweight OS Lightweight OS

–event-driven application model –cooperative multithreading –device drivers (UART, RTC, ADC, ...) –static memory allocation –minimum memory footprint

Programming Programming

–standard C language –high-level Bluetooth interface –system software available as library –emulation environment on Linux

BTnode System Software

slide-8
SLIDE 8

11

Event driven OS/application integration

Approach common to embedded systems Approach common to embedded systems Geared towards processing of external events Geared towards processing of external events

– sensor values, data packets, state changes – only one handler active at a time

One application per system at a time One application per system at a time

– application resident in device – no dynamic process model – events triggered by OS/driver functions and applications

No real-time OS know No real-time OS knowled ledge necessary e necessary for application design for application design

slide-9
SLIDE 9

13

Rapid prototyping – Linux emulation

Native compilation and execution on Linux Native compilation and execution on Linux

– using adapted drivers to match the host system – with a serial Bluetooth device on a PC or iPAQ

≈

Bluetooth PC + use unlimited resources of host + bridging networks + comfortable application debugging BTnode + deployment platform − slow upload necessary

slide-10
SLIDE 10

14

BTnode rev3 architecture

Multiple radio frontend BTnode Multiple radio frontend BTnode

– wake-up radio and low duty cycle power schemes

Integrated modular extension port Integrated modular extension port Integrated power management Integrated power management

– power supply (2x AA) or DC input – main power switch and reset – switchable supplies for subsystems – power consumption 0.5-50 mA @ 3.3V (projected)

slide-11
SLIDE 11
  • F. Ma. 17

Friedemann Mattern

Attachable Sensor Module

TeCO, University of Karlsruhe

slide-12
SLIDE 12
  • F. Ma. 18

Friedemann Mattern

BTnode Backend Connectivity

Easy access to background

infrastructure via BT-gateway

GSM SMS short text services WLAN Internet ...

Bluetooth gateway

Internet

Easy to integrate with

commodity devices

Simplifies development,

monitoring and debugging

Long- range link Short-range links within WSN

slide-13
SLIDE 13
  • F. Ma. 19

Friedemann Mattern

BTnode – Easy Integration w ith User Interface Devices

PDA BTnode PC peripherals BTnode Mobile phone Camera

slide-14
SLIDE 14
  • F. Ma. 20

Friedemann Mattern

Outsourcing of Computations as a System service

BTnodes have limited resources Outsource computations on nearby more powerful devices Mobile code is executed on remote device

PDA, mobile phone, laptop computer…

slide-15
SLIDE 15
  • F. Ma. 21

Friedemann Mattern

An Early Application of BTnodes: Smart Products

Acceleration

sensor

Temperature

sensor

Standard

Bluetooth profiles for SMS, object push and RFCOMM

slide-16
SLIDE 16
  • F. Ma. 22

Friedemann Mattern

Smart Products

sensor event user control

The BTnodes act as “smart

tags“ for valuable products

Alert via SMS of sensor events

such as shock or heat

Interactive dialogue with the

smart product

Bluetooth SMS via GSM

slide-17
SLIDE 17

I nteracting with Smart Everyday Objects…

slide-18
SLIDE 18

(1) …Using Sensors

slide-19
SLIDE 19
  • F. Ma. 25

Friedemann Mattern

Interacting w ith Disappearing Computers?

Hold two artifacts (with attached Smart-Its)

together – and shake!

slide-20
SLIDE 20
  • F. Ma. 26

Friedemann Mattern

Interacting w ith Disappearing Computers?

Hold two artifacts (with attached Smart-Its)

together – and shake!

slide-21
SLIDE 21
  • F. Ma. 27

Friedemann Mattern

The shaking motion establishes a shared context (i.e.,

acceleration pattern) that no other devices will have

Shaking Tw o Objects Together Establishes a ”Friendship”

image: TecO

After the shared context has been established, the

two devices can open a direct communication link

slide-22
SLIDE 22
  • F. Ma. 28

Friedemann Mattern

If the two objects are too far apart (e.g., radio communication

breaks down), the user is notified with an audible ”beep”

Beep! Beep!

Crying When Lonely

image: TecO

slide-23
SLIDE 23
  • F. Ma. 29

Friedemann Mattern

Application: Credit Card and Wrist Watch

Shake together before first use (establish friendship) Credit card will only work when in proximity of watch

slide-24
SLIDE 24
  • F. Ma. 30

Friedemann Mattern

Another Application: Poor Man’s Theft Alarm

Many mobile phones will be

equipped with motions sensors

interaction feature for games

Shake sensor cube together

with phone to activate the cube

no bottons etc. on the cube ”location limited channel” feedback via phone configuration via phone

Sensor Cube

slide-25
SLIDE 25
  • F. Ma. 31

Friedemann Mattern

Poor Man’s Theft Alarm

Place cube on object

to be protected

Receive alarm when

cube is moved

slide-26
SLIDE 26
  • F. Ma. 32

Friedemann Mattern

Poor Man’s Theft Alarm

Spontaneous networking

with gateway station?

Raise alarm only when

two or more sensor cubes report motion?

Deactivate when owner

is nearby?

slide-27
SLIDE 27

I nteracting with Smart Everyday Objects…

slide-28
SLIDE 28

(2) …Using RFI D

slide-29
SLIDE 29
  • F. Ma. 35

Friedemann Mattern

Making Things Smart w ith Electronic Labels (RFID)

… and a chip attached to it … on a substrate e.g. a plastic foil An antenna, printed, etched

  • r stamped ...

A paper label with RFID inside

Identify objects from distance

small IC with

RF-transponder

Wireless energy supply

magnetic field

(induction)

Read and write a few 100 bits „over the air“

~ 1 m

slide-30
SLIDE 30
  • F. Ma. 36

Friedemann Mattern

Smart Playing Cards

Playing cards have RFID labels

reader antenna is placed under the table wireless transmission to player‘s PDA

Features:

count score hints for beginners determine winner cheat alarm

Friedemann Mattern

slide-31
SLIDE 31
  • F. Ma. 37

Friedemann Mattern

Simulating Smart Objects w ith “Virtual Counterparts”

Identify objects here Implement all the smart behavior there

slide-32
SLIDE 32
  • F. Ma. 38

Friedemann Mattern

Virtual Counterparts of Playing Cards

:...;;.;...,.:,.. ;;.;.. :....,.:,.. ;.. :.. ;;...,.:,.. ;.. ...,:.. ;;.:,.. :...;;.;...,.:,.. ;;.;.. :....,.:,.. ;.. :.. ;;...,.:,.. ;.. ...,:.. ;;.:,.. :...;;.;...,.:,.. ;;.;.. :....,.:,.. ;.. :.. ;;...,.:,.. ;.. ...,:.. ;;.:,..

Hi, I‘m new here

Friedemann Mattern

slide-33
SLIDE 33
  • F. Ma. 39

Friedemann Mattern

Cards as Personalities

Do playing cards have a

memory?

What do they communicate? How do they react to msgs? How do playing cards interact

with a backend system?

General infrastructure

Alice in Wonderland

Friedemann Mattern

slide-34
SLIDE 34
  • F. Ma. 40

Friedemann Mattern

Softw are Infrastructure for Smart Objects?

:...;;.;.... ;;.;.. :..... ;.. :.. ;;...,. ;.. ...,:..

Friedemann Mattern

:...;;.;.... ;;.;.. :..... ;.. :.. ;;...,. ;.. ...,:..

Hi, I ‘m new here

:...;;.;.... ;;.;.. :..... ;.. :.. ;;...,. ;.. ...,:.. :...;;.;.... ;;.;.. :..... ;.. :.. ;;...,. ;.. ...,:..

alert event virtual counterpart

slide-35
SLIDE 35

I nteracting with Smart Everyday Objects…

slide-36
SLIDE 36

(3) …Using 2D-Labels

slide-37
SLIDE 37
  • F. Ma. 43

Friedemann Mattern

Responsive Objects

An objects tells

something about itself

e.g., by displaying a

dynamically generated homepage

Content

depends on circumstances

such as context and privileges

Image source: Nokia

  • Cf. Cooltown project (HP)

Friedemann Mattern

slide-38
SLIDE 38
  • F. Ma. 44

Friedemann Mattern

Responsive Objects

WWW server

I nternet

Direc- tory

Label

Location Context

Friedemann Mattern

Copy by

reference instead of copy by value

slide-39
SLIDE 39
  • F. Ma. 46

Friedemann Mattern

Object-Interaction w ith Camera-Equipped Mobile Phones

„Picture phones“ are

becoming quite popular

low to medium picture quality typically 640 x 480 pixels programmable Patents pending

slide-40
SLIDE 40
  • F. Ma. 48

Friedemann Mattern

Visual Codes

Many different visual codes exists Differences in

application domain number of encoded bits robustness

Low resolution CCD

camera requires coarse grained code

Data Matrix Philips Dot Code UPS MaxiCode TRIPcode QR Code PDF417

slide-41
SLIDE 41
  • F. Ma. 49

Friedemann Mattern

  • rigin of code

coordinate system (0,0) (10,0) (0,10)

  • rientation

bars code bits

(capacity: 83 bit)

Our Code (Suited for Low - Resolution CCD Cameras)

x y (0,0)

slide-42
SLIDE 42
  • F. Ma. 52

Friedemann Mattern

Visual Code Recognition

Recognition

slide-43
SLIDE 43
  • F. Ma. 54

Friedemann Mattern

Application: Buy Now

ID = 28AF36B2

slide-44
SLIDE 44
  • F. Ma. 55

Friedemann Mattern

Music Charts: Vote Now

ID = 28AF36B2

slide-45
SLIDE 45
  • F. Ma. 56

Friedemann Mattern

I nternet

I nfor- mation server Transmit I D via GPRS, GSM or WAP

I D- Resolver

„Talking Objects“ Simulated by a Backend Infrastructure

Reply (e.g., WAP/ WML) Display of I nfor- mation belonging to the tagged object

slide-46
SLIDE 46
  • F. Ma. 57

Friedemann Mattern

Augmenting Paper w ith Hyperlinks

x y (0,0)

Determine the coordinates of

the camera focus

w.r.t the visual code origin

Associate physical hyperlinks

with image regions

Augmented reality: the

picture phone can follow the hyperlink, fetch the additional information via the wireless link and display it

slide-47
SLIDE 47
  • F. Ma. 58

Friedemann Mattern

Item Selection w ith Relative Focus Position Determination

slide-48
SLIDE 48
  • F. Ma. 59

Friedemann Mattern

Item Selection w ith Relative Focus Position Determination

slide-49
SLIDE 49
  • F. Ma. 60

Friedemann Mattern

Item Selection w ith Relative Focus Position Determination

slide-50
SLIDE 50
  • F. Ma. 61

Friedemann Mattern

Item Selection w ith Relative Focus Position Determination

slide-51
SLIDE 51
  • F. Ma. 62

Friedemann Mattern

Item Selection w ith Relative Focus Position Determination

slide-52
SLIDE 52
  • F. Ma. 63

Friedemann Mattern

Item Selection: Current Snow Conditions in Andermatt

Original camera image Augmented image

slide-53
SLIDE 53
  • F. Ma. 64

Friedemann Mattern

Orientation Feature: Current Temperature in Andermatt

Camera orientation as an additional input parameter

α > 30°

slide-54
SLIDE 54

Mobile Sensing of ‘Superdistributed’ RFI D Tags

slide-55
SLIDE 55
  • F. Ma. 66

Friedemann Mattern

RFID antenna vacuum cleaner intake tube dust bag RCX

RFID-based Smart Vacuum Cleaner

slide-56
SLIDE 56
  • F. Ma. 67

Friedemann Mattern

µ-Chip (Hitachi)

Size:

0.4 mm2

Carrier frequency:

2.45 GHz

Operating distance: 0-25 cm Memory capacity:

128bit ROM

Anti-collision:

no

Response time:

20 ms

slide-57
SLIDE 57
  • F. Ma. 68

Friedemann Mattern

RFID-based Smart Vacuum Cleaner

  • Huge number of RFID tags fixed
  • n the floor
  • Robot carries a mobile RFID

reader

  • Robot learns its position (tag ID)
slide-58
SLIDE 58
  • F. Ma. 69

Friedemann Mattern

Fixed RFID Tag and Mobile Reader Antenna

slide-59
SLIDE 59
  • F. Ma. 70

Friedemann Mattern

Mobile Robot Positioning

Robot learns its position (tag ID) Knows the border, remembers its track,… Performs appropriate action (move, turn, clean,…)

depending on the position

slide-60
SLIDE 60
  • F. Ma. 71

Friedemann Mattern

Position Dependent Behavior

Border case Robot turns left

  • r right when it

senses a tag that is known to be a border tag Cleaning mode Robot performs cleaning action

  • n a small area

when it detects a “new” tag Déjà-vu Robot makes a turn when it detects the same two tags in a sequence

A B A B

slide-61
SLIDE 61
  • F. Ma. 73

Friedemann Mattern

Conclusions

Wireless sensors: not only to realize

smart environments, but also to implement smart objects

Friedemann Mattern

Jan Beutel, Oliver Kasten, Friedemann Mattern, Kay Römer, Frank Siegemund, and Lothar Thiele: Prototyping

Wireless Sensor Network Applications with BTnodes, EWSN,

Springer LNCS, 2004

Use today‘s technology to

prototype tomorrow‘s technology, applications, and scenarios

slide-62
SLIDE 62
  • F. Ma. 74

Friedemann Mattern

Acknow ledgements

BTnodes: Jan Beutel, Oliver Kasten, Frank Siegemund Smart Playing Cards: Kay Römer 2D-Codes: Michael Rohs Smart Vacuum Cleaner: Svetlana Domnicheva

slide-63
SLIDE 63
  • F. Ma. 75

Friedemann Mattern

Other BTnode Applications

  • The Lighthouse location system [Roemer2003]
  • Smart product monitoring [Siegemund2002]
  • Bluetooth enabled appliances [Siegemund2003]
  • Smart It’s friends [Siegemund2003]
  • XHOP/R-DSR multihop prototype [Beutel2002]
  • Distributed positioning – TERRAIN implementation [Frey2003]
  • Physical activity detection network [Junker2003]
  • Better avalanche rescue through sensors [Michahelles2002]
  • Wearable unit with reconfigurable modules [Plessl2003]
  • Undergrad projects with Lego Mindstorms [Blum2003]

…

slide-64
SLIDE 64
  • F. Ma. 76

Friedemann Mattern

Selected Publications from w w w .vs.inf.ethz.ch/publ/

  • Michael Rohs, Beat Gfeller: Using Camera-Equipped Mobile Phones for

Interacting with Real-World Objects. PERVASIVE 2004, to appear.

  • Kay Römer, Thomas Schoch, Friedemann Mattern, Thomas Dübendorfer:

Smart Identification Frameworks for Ubiquitous Computing Applications. Wireless Networks, Vol. 10 No. 6, December 2004.

  • Jan Beutel, Oliver Kasten, Friedemann Mattern, Kay Römer, Frank

Siegemund, Lothar Thiele: Prototyping Wireless Sensor Network Applications with BTnodes. 1st European Workshop on Wireless Sensor Networks (EWSN), Springer-Verlag, ISBN 3-540-20825-9, pp. 323-338, Berlin, January 2004.

  • Kay Römer, Svetlana Domnitcheva: Smart Playing Cards:

A Ubiquitous Computing Game. Journal for Personal and Ubiquitous Computing (PUC), Vol. 6, pp. 371-378, 2002.

  • Lars Erik Holmquist, Friedemann Mattern, Bernt Schiele, Petteri Alahuhta,

Michael Beigl, Hans-W. Gellersen: Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts. Proc. Ubicomp 2001, LNCS No. 2201, pp. 116-122, Springer-Verlag, 2001.

  • …
slide-65
SLIDE 65

Dagstuhl March 2004

Friedemann Mattern

ETH Zurich Institute for Pervasive Computing

ET ETH

Eidgenössische Technische Hochschule Zürich

Interacting w ith Smart Objects: Application Scenarios w ith the BTnode Platform