Development of Hybrid Mobile Application EEE 27 Li Jiaqi Raffles - - PowerPoint PPT Presentation

Development of Hybrid Mobile Application —EEE 27

Li Jiaqi Raffles Institution Koh Rei Min Ashley Victoria Junior College

• Prof. Perry Ping Shum

Nanyang Technological University

Introduction

What are hybrid mobile applications?

Hybrid mobile applications

• 1. Not exclusive to any operating server.
• 2. Built with a combination of web

technologies like Hypertext Markup Language (HTML), Cascading Style Sheets (CSS), and JavaScript as well as native applications.

• 3. Utilises a WebView of a mobile platform

enabling access to the device’s capabilities and facilities such as the accelerometer, camera, contacts, and more.

Native mobile applications

• 1. Built to work on a specific mobile
• perating system with the necessary tools

for developers to use to create an app. E.g. Apple IOS app has to be written in Xcode using Swift or Objective C languages and it only works on Apple products

Advantages of hybrid mobile applications

• 1. For developers, the origination cost is of hybrid mobile apps is lower

and initial speed is faster.

What is Augmented Reality(AR)?

Augmented reality (AR) is a technology that superimposes a computer-generated image on a user's view of the real world, thus providing a composite view. Less than 25% of the AR world is consisted of digital images.

Applications of AR —Education

Blippbuilder — AR learning platform

Applications of AR — Entertainment

Popular AR game - Pokemon Go Movie scenes showing AR technology in the future

Applications of AR — Medication

Use AR in surgery

Therefore, ...

while a considerable amount of resources and effort has been put into research of augmented reality and its application in various domains, such as aforementioned entertainment, education and medication,

the shopping environment has yet to be well explored. Thus this gap in the current market spurred us to explore in this direction.

Overview of

• ur research

This research will explore the potential use and development

• f MAR in a shopping environment by developing a hybrid

AR shopping assistant mobile application — ARShop.

Aims and Objectives

“ARShop” mobile application

Function 1: providing additional computer-generated perceptual information that is difficult to be presented on normal 2D packaging, such as videos, audios, to shoppers. Function 2: enabling sellers to put various animated tags on commodities that can attract shoppers attention when they scan through shelves. Built with Placenote SDK Built with Unity-Arkit-Plugin

Methodology

UNITY:

DEVELOPMENT SOFTWARE USED

ARKIT 2.0: PLACENOTE:

• Cross-platform game engine
• Can be used to create both 3D and 2D

games as well as simulations for its many platforms.

• easily accessible
• Most widely used AR and VR

development platform

• Has a highly optimized rendering

pipeline and rapid iteration capabilities.

• Combines device motion tracking,

camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.

• Offers full 2D image tracking
• Allows for quickly build cloud-based AR

apps that permanently save AR content in physical locations.

• Lets users scan any space and turn it

into a smart canvas for positioning AR

• bjects.
• Integrates with ARKit on iOS devices

PROVIDING ADDITIONAL PERCEPTUAL INFORMATION

FUNCTION 1 — “Scanning”

BUILDING PROCESS

For 3D animation, build animations in Unity with animation player function Duplicate plane of image target. Insert trailer/animation onto the duplicated plane. This will be called upon when the image target is captured on camera Import reference image (Covers of CDs or Games) This will be the image target. Create new scene in Unity

End Result

Trailers of songs and movies will be played when the camera in the app recognises the image target. 3D animation of how the board game is played will show up when the camera in the app recognises the image target (board game packaging) Video of trailer

• f the album

3D animation demonstrating the gaming process

1) ALLOW SELLERS TO SCAN SHELVES AND ADD TAGS 2) ALLOWS CUSTOMERS TO LOAD MAPS WITH TAGS ADDED TO COMMODITIES

FUNCTION 2 — “Trial”

INTRODUCTION TO PLACENOTE

Placenote allows users to scan any space and turn it into a smart canvas for positioning AR objects.

space AR objects

Shelves within a certain space in supermarkets or stores Animated tags to attract consumers’ attention

DEVELOPMENT OF INTERFACE FOR SELLERS

Start a new map button for sellers

Feature points

1) Sellers start a new map

FEATURE POINTS

• Embedded function of Placenote SDK.
• Created by scanning through the space.
• Allow the space to be relocated when the map is

reloaded by API key.

When a hit test was activated on the screen, a new tag would be created at the nearest plane detected and a new info list of this tag would also be created. Position Rotation type

2) Adding tags to commodities

• 5 types of tags sellers can choose to put on

commodities.

• Users can click on the bottom at the button
• f the screen to switch type of tags added.

TAGS

E.g. “Recommend” Tag E.g. “Discount” Tag

• Built with embedded function of

Placenote SDK.

• Sellers can assign a name for the map

to help consumers find the correct map when they reload it.

3) Save Map with Name

End Result —Sellers

DEVELOPMENT OF INTERFACE FOR CONSUMERS

➔ We made reference to the scripts in the example. These included loading a map list from API key online and loading the chosen map by On Load Map Clicked function.

ENABLING CUSTOMERS TO VIEW MAPS

End Result

Animated tags attached to board games with sparkles emerging from them to attract consumers’ attention

➔ We created a Home page scene in Unity which appears first when consumers launch the app

INTEGRATING THE 2 FUNCTIONS

Subsequently, we built this Unity project in Xcode and Android Studio, enabling the app to be installed onto the selected iOS and Android devices. Function 1 Function 2 Switch between functions

End Result —Consumers

Results

1) AR hybrid app as a means to help consumers make a better decision 2) AR hybrid app as a means to achieve more enjoyable shopping experience

Conclusion

1) App facilitates information transfer and provides customers with an interesting shopping experience 2) Other potential developments: AR trying on of clothes

Vector Icons by Matthew Skiles

Wei Zhu (2006, November). Recent advances in augmented reality. Retrieved on December 25, 2018, from https://apps.dtic.mil/dtic/tr/fulltext/u2/a606245.pdf Bristowe, J. (2015, March 2519). What is a Hybrid Mobile App?. Retrieved September 1, 2018, from https://developer.telerik.com/featured/what-is-a-hybrid-mobile-app/

REFERENCES

Connolly, P., Chambers, C., Eagleson, E., Matthews, D., & Rogers, T. (2010, October). Augmented reality effectiveness in advertising. In proc. 65th Midyear Conference on Engineering Design Graphics Division of ASEE.

Acknowledgement

We would like to take this opportunity to thank our mentor, Professor Shum Ping, for his constant support throughout the whole of this project. We would also like to thank our teacher-in-charge Dr. Lena Lui Wai Yi and Mr Chan Soo Heong Julius for their guidance and support.

Thank You

Any questions?