Oscar Gilbert Department of Computer Science and Computer - - PowerPoint PPT Presentation

Oscar Gilbert

Department of Computer Science and Computer Engineering

Sarah Marsh

Department of Computer Science and Computer Engineering

Faculty Mentor: Craig Thompson, Ph.D.

Department of Computer Science and Computer Engineering

5th Annual FEP Honors Research Symposium April 20, 2012

1 Gilbert and Marsh 5th Annual FEP Honors Research Symposium

Think about your electronics

• You probably have….
• A personal computer
• A smartphone
• A printer
• A television
• A gaming system
• A GPS
• These “network devices” can already wirelessly

interact with each other.

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 2

• WHAT IF…
• every real-world object was a network object?
• all these objects could interact wirelessly?
• This is one of Dr. Craig Thompson’s ideas about

pervasive computing in the future.

• If this is to come about, computational recognition
• f human activities will become very important.

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 3

• Computers lack the ability to recognize human

behavior

• This presents an impediment to their use in

applications that involve interacting with humans.

• If this could be solved, computers would become

more helpful to us in our daily lives.

4 Gilbert and Marsh 5th Annual FEP Honors Research Symposium

• Gather small amounts of data about human

activity.

• Write a computer program to interpret the data in

such a way that the computer can extrapolate a meaningful workflow from simple activities.

5 Gilbert and Marsh 5th Annual FEP Honors Research Symposium

• Our program recognizes workflows (sequences of

activities)

After prior training, if our program receives input strings: 1. You pick up the trash 2. You go outside 3. You place the trash in the trashcan Then our program will predict possible next step(s)

• You will move the trash can OR
• You go back inside

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 6

• Currently, computers cannot hear or see like

humans can.

• To sense their world, computers could rely on small

amounts of data about the world, also known as sensory traces (because the data streams in from sensors).

• These traces will be the foundation upon which our

workflows are built.

• For our purposes, each sensory reading can be treated as

a string (as explained next).

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 7

• String = Sequence of characters
• Ex: “Hello World” or “345sdfds” or “Take the trash can outside”
• Substring = String inside a string
• Ex: Valid Substrings of “Hello World” = “Hello” / ”o World” / ”Worl”
• Example of a Function:

Function printHelloWorld(){ Print “Hello World” }

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 8

• A workflow is a sequence of steps that is organized

in time

• This concept is significant because our program

interprets human behaviors in terms of workflows

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 9

• A bad recursive function ….

Function printHelloWorld(){ Print “Hello World” printHelloWorld() }

• A better recursive function…..

Function printHelloWorld(){ Print “Hello World” If (Hello World has been printed ten times) then (Exit Program) Otherwise (printHelloWorld()) }

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 10

• Inputs from RFID devices and/or Machine Vision
• RFID is good at identifying objects
• Kinect is good at tracking movement
• An alternative used in our project: Simulated Data

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 11

Kinect + RFID?

• Represent each activity as a unique string
• Finds closest match to current history of activity
• Uses match from past history to predict future

activity

• Deals with branching workflows

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 12

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 13

• Our program is capable of learning sequences of

human behavior and predicting future actions

• There are many applications for this technology

• C. Thompson (2004). “Everything is Alive,”

Architectural Perspective Column, IEEE Internet Computing, January-February 2004.

• Radio Frequency Identification, Accessed

November, 2011, URL: http://en.wikipedia.org/wiki/Radio- frequency_identification

• A. Eguchi (2013). “Object Recognition Based on

Shape and Function: Inspired by Children’s Word Acquisition,” Inquiry J. Undergraduate Research, University of Arkansas.

Gilbert and Marsh 5th Annual FEP Honors Research Symposium 14