The Importance of Being Thing Or the Trivial Role of Powering - - PowerPoint PPT Presentation

The Importance of Being Thing

Or the Trivial Role of Powering Serious IoT Scenarios

Sumi Helal1, Ahmed Khaled2 and Wyatt Lindquist1

1 School of Computing and Communication, Lancaster University 2 Department of Computer Science, North Eastern Illinois University

Sumi Helal, PhD, FIEEE, FIET

Professor & Chair in Digital Health Director of the Centre on Digital Health & QoL Technologies Faculty of Science & Technology Faculty of Health and Medicine Lancaster University

9 July 2019

Talk Overview

• Motivation: Make IoT happen (or increase its IoTility)
• We must walk before we run → Thing, before Internet of Things
• Back to the Future: We must be worth something already
• Integrate once, program everywhere
• Where are we today?: The World Cup App Benchmark
• Raising the Bar: New Thing Requirements
• Raising the Bar Even Higher: Domain Thing Requirements
• The Rise of Thing Architectures: Reporting on our Work
• Conclusions

Motivation

The Need for Thing Architecture

• Great vision, and many impressive scenarios, yet no clear

pathways to realization.

• The focus on vision & abstracting away Thing details were

intentional and productive, but they have now run their course

• Focus of this presentation is on Personal IoT

Personal IoT (Smart homes) Industrial IoT (Smart factory floor) At-Scale IoT (Smart City)

Back to the Future

What has been Accomplished in IoT, specifically in THING

• I. Integrate Once
• DDL: Device Description

Language

• Service-Oriented Device

Architecture (SODA)

<Sensor> <Description>…</Description> <Interface> <Signal id="ADC1">…</Signal> <Reading id="Temp1"> <Type>Physical</Type> <Measurement>Temperature </Measurement> <Unit>Centigrade</Unit> <Computation> <Type>Formula</Type> <Expression> Temp1 = (((ADC1/1023) * 3.3)-0.5)* (1000/10)</Expression> </Computation> </Reading> </Interface> </Sensor>

Example DDL for TMP36 ANALOG TEMPERATURE SENSOR

Back to the Future

What has been Accomplished in IoT, specifically in THING

• II. Program Everywhere

3 Sensor Platforms Powered UP 3 OSGi Service Bundles appear in the IDE

Atlas Communication Layer Atlas Processing Layer Atlas Device Interface Layer

Servo 8 Analog GPIO 32 Analog 16 Digital Processor ZigBee Ethernet Wi-Fi Antenna

Back to the Future

The ATLAS Platform

SODA Programmability:

Blessing or a Curse?

• The Ant, the Elephant, the Monkey & the

Giraffe, or:

• Bless because we can program and

reprogram any logic and any application (very expressive)

• Curse because:

1. SODA over-promises (an elephant for an ant), artificially masking failures, leading to complex uncertainties to manage 2. SODA is too powerful (too expressive) to be safe.

• Make it Work: IoT Tranx, Virtual Sensors,
• thers.

IoT Today

Sumi’s World Cup IoTility Benchmark

• How many people

missed the the Women World Cup yesterday?

Smart TV with a Digital Video Recorder (DVR) – all with Internet Connections 2014 World Cup Android App showing scores and game schedule The user acting On behalf of the Future!

IoT Today

Sumi’s World Cup IoTility Benchmark

Raising the Bar

New Requirements for Things

The Coffee Maker and & Smart Phone Scenario

• Minimal h/w to enable interactions
• Ability to chat and in multi languages (HTTP-

REST/MQTT/CoAP/…)

• Ability to chat about capabilities and sought

relationships

• Ability to create API’s on the fly
• Ability to infer and activate meaningful

relationships (seeking meaningful interactions)

• The ability to create IoT apps by the consumer

Raising the Bar

New Requirements for Things

The World Cup Scenario

• Ability for mobile apps to be and act like

any other thing in the IoT

• Ability for an App to change its behavior

to engage with other things in useful applications

• Ability for developers to develop such

powerful IoT apps without much knowledge of the smart space in which the app will run.

Raising the Bar Even Higher:

Domain-Specific Requirements: Health IoT

• Specialized smart things bring new requirements to IoT
• Health devices emphasize
• Safety
• Reliability
• Data collection
• Traditional architectures may not be sufficient
• Need to address these requirements to maximize potential
• Drive the next generation of generalized IoT systems

• Interactions between user and

device

• Proper use
• Notifications/Reminders

(empowerment)

• Identity
• Interactions between devices
• Mobile apps as things
• Democratization
• APIs
• Inter-thing relationships

Raising the Bar Even Higher:

Domain-Specific Requirements: Health IoT

The Rise of Thing Architecture

Reporting on our Current Work

Atlas Thing Architecture

• IoT-DDL (Iot Device Description Language)
• Inter-Thing Relationships – an IoTility Booster
• RIDE: Programming Model and IDE for the Consumer
• Mobile Apps as Thing (MAAT) and IDE for future IoT

software developers

• Multi lingual IoT Communication Broker (HTTP-

REST/MQTT/CoAP/…)

Atlas Thing Architecture

• Utilizes IoT-DDL and thing’s OS services to provide new

functionalities a thing needs to engage in interactions.

• New set of software operating layers allows the thing to:
• Self-discover its own capabilities and services.
• Generate APIs for the offered services.
• Announce its presence and interact with other things.
• Be configured and managed by the developer/user

Atlas Thing Architecture

Interactions and Tweeting Engine IoT-DDL Manager API Engine Identity Parser Attachment Manager Interface and Communication Engine Identity and Knowledge Tweets

Atlas IoT Platform

DDL Sublayer Tweeting Sublayer Interface Sublayer Host Interface Layer Network Manager Process Manager Memory Manager Device Secure Elements Services Interfaces

IoT OS Services

Main Controller and Messaging Backbone Device Manager Meaningful Interactions Security Engine and Application Run-Time Knowledge Engine

Atlas Thing Architecture

IoT-Device Description Language (DDL)

• Machine- and human-readable XML-based descriptive

language for things in smart space.

• Describes a thing in terms of the inner components,

capabilities, services and cloud-based accessories for the seamless integration and management.

Atlas Thing Architecture

IoT-DDL: Coffee maker example

Image link IconExperience.com

<Metadata> Name: Coffee Machine Vendor: Bosch ... <Resources> Network Properties: Wi-Fi Memory Properties: RAM ... <Services> Service1: Turn On For Duration Service2: Turn Off ... <Attachments> Attachment1: Log Server ...

Atlas Thing Architecture

Inter-Thing Relationships Framework

• Utilize a set of service-level relationships that tie services

logically across different things

• Built-in relationships (via IoT-DDL), and run-time inferred

relationships.

Cooperative Relationship Competitive Relationship

prepare breakfast toast by 7:00 am Coffee machines from different vendors

Atlas Thing Architecture

Runtime-Interactive Development Environment (RIDE)

• RIDE extends the Inter-thing relationships framework and

targets smartphone users with no programming experience (the consumer of the Personal IoT product space)

• RIDE main functionalities:
• Listen to things’ services and relationships
• Establish relationships, recipes and apps
• Infer existence of new relationships and recipes
• Accept the established/inferred apps and generate and install

IoT apps as mobile apps.

Development Interface Application Engine Repository Inference Engine Explorer

• Applications
• Results
• Recommendations
• Feedback
• Established Primitives
• User Preferences

Primitives

Developers

• App Development
• Preferences
• Recommendations
• Programming Opportunities

Unicast Interactions Multicast Tweets

Atlas Things

…. App 1

Directory of developed and generated Atlas IoT apps

App 2 App n

RIDE

Overall Architecture

RIDE

Application Engine

• Accepts app description, checks the

correctness through semantic rules.

• Transforms the structural

representation of the app into a descriptive manifest.

• Passes the manifest to an on-cloud

service and receives an executable app to install and run on demand.

<App> <-----> …… </----> </App> Manifest User Interface Gateway App

Unicast Interactions

Atlas Things Atlas IoT App Execution Engine

RIDE

Example IoT Application

Relationship Recipe Toggle the light and play a movie, if light is on Window blinds supports watching movie on screen

Check if the light is on Tilt the blinds down Play a movie on the screen Toggle the light Atlas Thing Architecture IoT-DDL Atlas Thing Architecture IoT-DDL Atlas Thing Architecture IoT-DDL

RIDE

Mobile Apps As a Thing (MAAT)

Actionable Keywords (AKW)

• Describe data available in the app (types and keywords)
• Things: look for AKWs that can be used as inputs
• On the app: associated with a specific layout element
• ”Placeholder” (a button) to control unknown interaction
• Thing responds to AKW
• Button appears with label specified by thing
• Click sends the specified data to the thing

Mobile Apps As a Thing (MAAT)

Actionable Keywords (AKWs)

Germany v. Brazil Tonight @ 8PM On Sky Sports { “keywords”:[“TV”, “Program”], “type”: “button”, “value”: “ProgramID” } HIDDEN RECORD { “service”:“RecordProgram”, “owner”: “DVR”, “inputs”: [“ProgramID”] } Germany v. Brazil Tonight @ 8PM On Sky Sports

New Extend Relationship DVR Actionable Keyword

Mobile Apps As a Thing (MAAT)

Developer IDE Support

• Developer might not be an IoT expert
• Choosing keywords is hard
• Minimal changes to development environment
• Android Studio Plugin
• Choose layout to make actionable
• Search through existing thing keywords and data types (IoT-DDL)
• Complete code snippet with offered data
• Plugin + library handles the rest, app is IoT enabled

Mobile Apps As a Thing (MAAT)

Developer IDE Support

Multi-lingual IoT Communication Broker

Atlas Communication Framework

• Utilizes the minimum delimiter among the various protocols to provide

homogenous links through Atlas topics and Atlas protocol translator.

MQTT-speaking thing Smart space broker (SSB) CoAP-speaking thing Off-thing Atlas protocol Translator (attachment) API Notifications Data HTTP REST-speaking thing On-thing Atlas protocol translator

Smart Space

Atlas Communication Framework (Cont.)

• Utilizes the minimum delimiter among the various protocols to provide

homogenous links through Atlas topics and Atlas protocol translator.

Smart space broker (SSB) CoAP-speaking thing Off-thing Atlas protocol Translator (attachment) API Notifications Data CoAP homogenous communication CoAP-speaking thing

References

1. Khaled, A. E., Helal, A. S., Lindquist, W., & Lee, C. (2018). IoT-DDL—Device Description Language for the “T” in IoT. IEEE Access, 6 (1), 24048-24063. 2. Khaled, A. E., & Helal, S. (2019). Interoperable communication framework for bridging RESTful and topic-based communication in IoT. Future Generation Computer Systems, 92, 628-643. 3. Helal, S., Khaled, A. E., & Gutta, V. (2017). Atlas thing architecture-Enabling mobile apps as things in the IoT: 23rd Annual International Conference on Mobile Computing and Networking, MobiCom 2017. 4. Khaled, Ahmed, Wyatt Lindquist, and Sumi Helal. "DIY Health IoT Apps." Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems. ACM, 2018. 5. Khaled, A. E., & Helal, S. (2018, February). A framework for inter-thing relationships for programming the social IoT. In IEEE 4th World Forum on Internet of Things (WF-IoT), 670-675. 6. Lindquist, W, A. Helal, Khaled, A., Lee, J, and G. Katonya, “MAAT: Mobile Apps As a Service,” Under revision in ACM IMWUT Transactions.

Conclusions

• Let us walk before we run
• When we run we can also run marathons and deliver on

the powerful visions of IoT

• Let us not part away from our past – pervasive computing
• Let us define IoTility
• Create Iotility Boosters
• Dedicated Thing architectures
• Mobile Apps as Things, and many other things as Things