A selfadapting User Interface for Smart Spaces Andreas Hubel - - PowerPoint PPT Presentation

▶
a self adapting user interface for smart spaces
SMART_READER_LITE
LIVE PREVIEW

A selfadapting User Interface for Smart Spaces Andreas Hubel - - PowerPoint PPT Presentation

Jun 22, 2023 342 likes •685 views

Chair of Network Architectures and Services Department of Informatics Technical University of Munich A selfadapting User Interface for Smart Spaces Andreas Hubel Advisor: Marc-Oliver Pahl Supervisor: Prof. Dr.-Ing. Georg Carle Technical

slide-1
SLIDE 1

Chair of Network Architectures and Services Department of Informatics Technical University of Munich

A self–adapting User Interface for Smart Spaces

Andreas Hubel Advisor: Marc-Oliver Pahl Supervisor: Prof. Dr.-Ing. Georg Carle Technical University of Munich (TUM) Department of Informatics Chair of Network Architectures and Services Garching, 18. October 2017

slide-2
SLIDE 2

Andreas Hubel | A self–adapting User Interface for Smart Spaces 2

Agenda

DS2OS/VSL Research Questions Methodology Problem Analysis Approach & Solution Evaluation Related Work Conclusion

slide-3
SLIDE 3

Introduction

slide-4
SLIDE 4

Andreas Hubel | A self–adapting User Interface for Smart Spaces 4

Introduction

Problems of current smart space UI:

  • not platform in-depended
  • not reusable in other spaces
  • APIs not abstract enough
  • only insufficient exchange platforms
  • not user customisable
  • semantic relationships are not modelled

…

slide-5
SLIDE 5

Andreas Hubel | A self–adapting User Interface for Smart Spaces 5

Autonomic Home Networking Lab (MI 03.05.041)

433 MHz transceiver KA host + SHE 8x ethernet power switch Arduino Relais

slide-6
SLIDE 6

Andreas Hubel | A self–adapting User Interface for Smart Spaces 6

AHN Lab: VSL context model

slide-7
SLIDE 7

Andreas Hubel | A self–adapting User Interface for Smart Spaces 7

AHN Lab: VSL types

/basic/list /basic/text /derived/percent restrictions: minimumValue: 0 maximumValue: 100 /gahu/blind /basic/composed /basic/number /gahu/genericDevice /derived/boolean /gahu/lamp angle closed isOn

slide-8
SLIDE 8

Andreas Hubel | A self–adapting User Interface for Smart Spaces 8

DS2OS overall architecture

SHE SHE

SHE NLSM svc svc SHE NLSM svc svc SHE svc NLSM SLSM

DS2OS Site

SHE NLSM svc svc SHE NLSM svc svc SHE NLSM svc svc SHE NLSM svc svc SHE svc NLSM SLSM

DS2OS Site

NLSM KA svc SLSM SHE

Service Hosting Environment Node-Local Service Manager Site-Local Service Manager VSL Knowledge Agent Other Services Logical Communication

Global

S2Store Model Repository Statistics Service Repository

Certificate Repository

svc KA NLSM SLSM KA

DS2OS Site

DS2OS Connector

HTTPS KA KA KA KA KA KA KA KA

slide-9
SLIDE 9

Andreas Hubel | A self–adapting User Interface for Smart Spaces 9

Research Questions

How to create a state of the art (web) UI? How to implement and balance:

  • user customisation,
  • self-adaption, and
  • UI extensions?

How to use floor plans in smart space UI?

slide-10
SLIDE 10

Andreas Hubel | A self–adapting User Interface for Smart Spaces 10

Scenario: Extensible UI

slide-11
SLIDE 11

Andreas Hubel | A self–adapting User Interface for Smart Spaces 11

Methodology

slide-12
SLIDE 12

Problem Analysis

slide-13
SLIDE 13

Andreas Hubel | A self–adapting User Interface for Smart Spaces 13

Use case model

This thesis Back-end WebUI2 Owner, Operator, Installer/Fitter Admin Occupant Support debugging mange user, permissons mange floor plan mange device location

  • perate/control

device S2Store Developer, Expert implement service publish service install service configure service Store Light, Switch, Gateway, Device, Service

slide-14
SLIDE 14

Andreas Hubel | A self–adapting User Interface for Smart Spaces 14

Floor plans as smart space UI

D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D D E C K E N K A N T E U H D 03.12.036 03.12.03 03.12.037/1 03.12.038/1 03.12.040/1 03.12.051/3 03.12.051/1 03.12.053/1 03.12.054/1 03.12.039/1 03.12.057/1 03.12.056/1 03.12.058/1 03.12.061/1 03.12.059/1 03.12.060/1 03.12.102/1 03.12.051/2 M R D H VT G H VT G H VT G H VT G H VT G H VT G H VT G H VT G H VT G H G H G H VT G H VT G H VT G H VTS G H VT H VT G P I N N W A N D 2 , x 1 , 4 S K S K P K T A F E L 2 , x 1 , 2 W H I T E B O A R D 2 , x 1 , 2 W H I T E B O A R D 2 , x 1 , 2 T A F E L 2 , x 1 , 2 W H I T E B O A R D 2 , x 1 , 2 W H I T E B O A R D 2 , x 1 , 2 T A F E L 2 , x 1 , 2 T A F E L 2 , x 1 , 2 T A F E L 2 , x 1 , 2 T A F E L 2 , x 1 , 2 T A F E L 2 , x 1 , 2 T A F E L 2 , x 1 , 2 3 . 1 2 . 5 / 1 M R D M ITT E F L U R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . R W > 4 5 d B . . . . . . . . B 1 8
  • 1
1 B 1 8
  • 1
1 B 1 8
  • 2
1 B 1 8
  • 2
B 1 8
  • 4
B 1 8
  • 4
B 1 8
  • 2
1 B 1 8
  • 2
1 B 1 8
  • 1
1 B 2 7
  • 4
1 B 2 7
  • 1
B 1 8
  • 1
2 B 1 8
  • 1
1 B 1 8
  • 1
1 B 1 8
  • 1
1 V F 2 2121 2119 2120 2118 2116 2117 2686 2115 2114 2685 2111 2112 2113 2110 210 2 6 8 6 2 6 8 5 TREPPE 1 2 12 03 1 6 . 8 8 m 2 VF 1 1 5 B U E R O W I S S . 6 12 03 1 7 . 5 8 m 2 MA5 5.5 HNF 010111 B U E R O P R O F C 2 / 3 3 6 12 03 1 7 . 5 8 m 2 MA11 4.2 HNF 1 1 5 BUERO WIHI 5 8 12 03 1 7 . 5 8 m 2 MA5 10.1 HNF 1 1 5 B U E R O D I P L . 5 6 12 03 1 7 . 5 8 m 2 MA5 9.1 HNF 1 1 5 B U E R O W I S S . 12 03 1 7 . 5 8 m 2 MA5 5.3 HNF 4 1 1 5 B U E R O W I S S . 3 8 12 03 MA11 5.1 HNF 1 7 . 5 8 m 2 1 1 5 B U E R O D O K T . 5 4 12 03 1 7 . 5 8 m 2 MA5 8.1 HNF 1 1 5 B U E R O W I S S . 5 9 12 03 1 7 . 5 8 m 2 MA5 5.1 HNF 1 1 5 B U E R O D R I T T . 5 7 12 03 2 6 . 6 7 m 2 MA5 6.1A HNF 1 1 5 B U E R O P R O F . C 4 5 3 12 03 2 6 . 6 8 m 2 MA5 2.1 HNF 1 1 5 B U E R O S E K R . 5 1 12 03 1 7 . 5 8 m 2 MA5 3.1 HNF 1 1 5 B U E R O W I S S . 3 7 12 03 1 7 . 5 8 m 2 MA5 5.2 HNF 010111 B U E R O P R O F C 3 5 12 03 1 7 . 5 8 m 2 MA11 4.1 HNF 1 1 5 B U E R O P R O F C 2 / 3 3 9 12 03 1 7 . 5 8 m 2 MA5 4.1 HNF F L U R 5 1 2 . 03 4 . 1 6 m 2 VF F L U R 3 1 2 . 03 4 7 . 9 1 m 2 VF 1 1 5 B U E R O W I S S . 6 1 12 03 1 7 . 4 m 2 MA5 5.4 HNF AUFENTHALT 5 2 12 03 1 8 . 3 m 2 VF

BT-12 27 von 35

4604 17001 17000
  • Fiberoptik
Brüstungskanal 17003 17002
  • Fiberoptik
Brüstungskanal 17005 17004
  • Fiberoptik
Brüstungskanal 17009 17008
  • Fiberoptik
Brüstungskanal
  • 17006
17007 17012 17011 17010
  • Fiberoptik
Brüstungskanal 17014 17013
  • Fiberoptik
Brüstungskanal 17016 17015
  • Fiberoptik
Brüstungskanal 17018 17017
  • Fiberoptik
Brüstungskanal 17039 17038
  • Fiberoptik
Brüstungskanal 17037 17036
  • Fiberoptik
Brüstungskanal 17035 17034
  • Fiberoptik
Brüstungskanal 17033 17032
  • Fiberoptik
Brüstungskanal 17031 17030
  • Fiberoptik
Brüstungskanal 17029 17028
  • Fiberoptik
Brüstungskanal 17027 17026
  • Fiberoptik
Brüstungskanal D-MQ AK
slide-15
SLIDE 15

Andreas Hubel | A self–adapting User Interface for Smart Spaces 15

Floor plans as smart space UI

Graphical floor plan representation Architecture floor plan Google Maps OpenLevelUp Munimap free@home configuration usage Apple Mockup suitable amount of details no – – yes + yes + yes yes ++ yes ++ details zoom depended no yes yes yes no yes no different colours for corridors, toilets no – corridor ++ distinguish through colour corridor ++ distinguish through colour corridors dotted no – no – rooms are semantic entity don’t exist – semantically (if not IFC based) yes + yes + yes, + walkable area yes, + walkable area possibly yes doors yes, with opening direction no – – yes, as icons yes, as dark grey boxes + no – – yes, gaps in wall ++ windows yes no no wall indentions no grey gaps in wall ++ furniture to some amount (yes) no no no no yes, as grey areas ++ view angle top view top view top view top view top view perspective view

slide-16
SLIDE 16

Andreas Hubel | A self–adapting User Interface for Smart Spaces 16

Floor plans as smart space UI

Indoor Map UIE Architecture floor plan Google Maps OpenLevelUp Munimap free@home configuration usage Apple Mockup zoomable n/a yes yes yes yes n/a multiple floors n/a yes yes yes yes rather not floor selector n/a listing on bottom right corner top left, dropdown with arrows floating, per building

  • wn view + shortcut

—near zoom controls n/a yes + no – no – 50% —per building n/a no no yes no icon size zoom dependent n/a no, fixed no, fixed no, fixed yes, icons get sized down yes, UIE becomes small dot n/a Smart Space UIE Architecture floor plan Google Maps OpenLevelUp Munimap free@home configuration usage Apple Mockup device icons represent state n/a n/a n/a n/a no yes yes direct action with one click n/a n/a n/a n/a no yes n/a

slide-17
SLIDE 17

Andreas Hubel | A self–adapting User Interface for Smart Spaces 17

Web application frameworks

Angular vs. Ember vs. React

slide-18
SLIDE 18

Approach & Solution

slide-19
SLIDE 19

Andreas Hubel | A self–adapting User Interface for Smart Spaces 19

VSL Web UI 2.0

slide-20
SLIDE 20

Andreas Hubel | A self–adapting User Interface for Smart Spaces 20

Webserver (local) Webrowser Geo service Geodatabase UI Extension Store Front-end KA1 Back-end Web app

SQL

slide-21
SLIDE 21

Andreas Hubel | A self–adapting User Interface for Smart Spaces 21

Web app: Software architecture

PositionSerializer ApplicationSerializer DS.JSONSerializer ListController Service ListRoute MapRoute Device Dobject Position DS.Model ComponentTemplate Component Template Route Router Communication KA DS.RESTAdapter DS.Store MapController Controller 0..n 0..n 1 1 1 1..n

slide-22
SLIDE 22

Andreas Hubel | A self–adapting User Interface for Smart Spaces 22

Sequence diagram

User User User implicit subscription for each child, e.g. isOn: for each device: touches switch .

  • pens popup .
  • pens map .
slide-23
SLIDE 23

Andreas Hubel | A self–adapting User Interface for Smart Spaces 23

Web app: Model classes

Position device: Device center: LatLng Device position: Position icon Dobject id value type restriction access parent: Dobject children: Dobject name componentName 0..1 parent children 0..n

slide-24
SLIDE 24

Andreas Hubel | A self–adapting User Interface for Smart Spaces 24

Web app: Model classes and Components

Ember::Component BasicList BasicText BasicNumber GahuLamp BasicComposed Position device: Device center: LatLng Device position: Position icon Dobject id value type restriction access parent: Dobject children: Dobject name componentName 0..1 parent children 0..n

slide-25
SLIDE 25

Andreas Hubel | A self–adapting User Interface for Smart Spaces 25

Ember: Component templates

b) templates/components/basic-composed.hbs (Fig. 5.1, 5.2, 5.3)

13

{{#each c.children as |child|}}

14

{{component child.componentName c=child}}

15

{{/each}}

c) templates/components/derived-boolean.hbs (Fig. 5.1, 5.2)

16

{{#paper-switch checked=c.value }} {{c.name}} {{/paper-switch}}

d) templates/components/basic-number.hbs (Fig. 5.3, 5.2)

17

{{c.name}} <br/>

18

{{#if c.restrictions}}

19

{{input type="range" min=c.restrictions.minimumValue max=c.restrictions.maximumValue value=(mut c.value) disabled=c.isReadonly }}

,→

20

{{else}}

21

{{input type="number" value=(mut c.value) disabled=c.isReadonly}}

22

{{/if}}

slide-26
SLIDE 26

Andreas Hubel | A self–adapting User Interface for Smart Spaces 26

Spatial relationships

Operation VSL SQL geometry position without any geometry Create set INSERT set /geoservice/geometryOf/<foobar> {"type":"Polygon","coordinates":[[[30,10], [40,40],[20,40],[10,20],[30,10]]]} set /geoservice/positonOf/<foobar> x y z Update (Modify) set UPDATE Read (Retrieve) get SELECT get /geoservice/geometryOf/<foobar> get /geoservice/positonOf/<foobar> Delete (Destroy) set DELETE set /geoservice/geometryOf/<foobar> del set /geoservice/postionOf/<foobar> del VSL SQL geometry position without any geometry everything get SELECT get /geoservice/positonOf/* get SELECT get /geoservice/devicesWithoutLocation inside get SELECT get /geoservice/geometriesIn/<foobar> get /geoservice/positonsIn/<foobar> get /geoservice/locationsIn/<location> locationType get SELECT get /geoservice/locationsOfType/<type> locationType + inside get SELECT get /geoservice/locationsOfTypeIn/<type>//<location> coordinates get SELECT get /geoservice/locationsNear/<lat>/<lon> get /geoservice/locationsNear/<lat>/<lon>/<level> deviceType + inside get SELECT get /geoservice/typeIn/<type>//<location> reverse get SELECT get /geoservice/locationsReverse/<foobar>

<foobar> is a VSL path, name or UUID Milestone 1 Milestone 2 Milestone 3

slide-27
SLIDE 27

Evaluation

slide-28
SLIDE 28

Andreas Hubel | A self–adapting User Interface for Smart Spaces 28

Evaluation

Tests and measurements:

  • Desktop: Chrome, Firefox, Safari
  • Mobile devices: iOS and Android

Usability tests with users

slide-29
SLIDE 29

Andreas Hubel | A self–adapting User Interface for Smart Spaces 29

Evaluation: Usability tests

six users aged 20 to around 30

  • each test about 1h

setup:

  • hardware in the AHN lab (lamps, blind, misc sockets)
  • web app prototype
  • on Linux desktop PC in Chrome
  • on iPad 4 in Safari
  • on Android phone in Chrome
slide-30
SLIDE 30

Andreas Hubel | A self–adapting User Interface for Smart Spaces 30

Evaluation: Usability tests results

app intuitively to use (c.f. Google Maps) patterns from Material Design well received positively surprised about the live syncing of state save and recall of scenes was missed North-oriented floor plan …

slide-31
SLIDE 31

Andreas Hubel | A self–adapting User Interface for Smart Spaces 31

Distinction / Progress beyond Related Work

OpenHAB / ESH free@home HomeKit VSL + Web UI 2.0 target space type home home home everything vendor independent yes no (yes) yes UI requirements R.1 ui-addons no, but open source thus forks possible no no, but custom apps on top of HomeKit possible yes, but package format unclear R.2 floorplan no yes no yes R.3 zoomable n/a yes n/a yes R.4 multiple-floors via workaround yes via workaround no, requires vector data (v2.2) R.5 multiple-devices- same-XY n/a no n/a with clustering plugin (v2.1) R.6 icon-size-zoom- dependent n/a yes, become smaller n/a no, not yet R.7 room-geometry no yes no yes R.30 icon-represents- state yes yes n/a no R.31 map-rotation n/a no n/a no, not yet R.32 filter-by-category yes no, not yet

slide-32
SLIDE 32

Andreas Hubel | A self–adapting User Interface for Smart Spaces 32

Conclusion and Future Work

Foundation for adaptive, state of the art graphical user interfaces within the DS2OS project. Resulting artefacts, besides analysis:

  • Floor plan based web UI prototype
  • VSL Geo Service

Future Work

  • KA: Safari WebSocket problem, Response limit
  • S2Store, extension packaging
  • UI: Leaflet extensions, Library upgrades
  • services: group, logging
  • further UI extensions
  • more concrete (example) spaces
slide-33
SLIDE 33

Chair of Network Architectures and Services Department of Informatics Technical University of Munich

A self–adapting User Interface for Smart Spaces

Andreas Hubel <andreas.hubel@tum.de> https://andreas-hubel.de/ma/