Real-time Tracking of Human Arm Movements Jacob Phillips, Erik - PowerPoint PPT Presentation

Real-time Tracking of Human Arm Movements Jacob Phillips, Erik Guetz, Dr. Mohammad Imtiaz

Project Overview

Problem • Be able to track, diplay, and predict human arm movements • Provide a way to access the data for various applications

Problem Overview

Project Goals • Accurate arm motion tracking – Filtering and prediction • Long battery life – Efficient embedded system • Minimal human interference – Easy to set up and run • Easy to read and understandable display – Mobile phone app with graphs and data readouts

Problem Solution • Inertial Measurement Units (IMU) – Three sensors on arm • Embedded system – Custom PCB with RTOS • Mobile phone app – Android application on smartphone

Problem Solution System Diagram

Functional Requirements • Three IMU’s placed on arm • Data streaming wirelessly • Data storage facility • Stream data for downstream systems

System Design Process

Embedded Systems Specifications • 32-bit Atmel SAM4S32B Microcontroller – Up to 120MHz, 2MB FLASH, 160KB SRAM • 4Gbit NAND FLASH • STC3100 battery “gas gauge”/Coulomb counter • Sparkfun Bluetooth Mate Silver – RN-42 Bluetooth module • LSM6DS3 Inertial Measurement Unit – Up to 6.6KHz and 8Kb on board FIFO

Custom PCBs • Two boards – IMU and main board • Created in Eagle PCB – Custom made libraries and parts • Main board contains 32-bit ARM microcontroller – FLASH memory, Bluetooth, battery charging • IMU board designed as small as possible – Final dimensions: 18.5x15.5 mm

IMUs

IMU • Small, simple device • Low power • Fast communications

IMU PCB ● Kept as simple and small as possible ● Only four components total

Sensor Controller

Sensor Controller • Low power • Multi-day storage capability • Wireless streaming • On-board battery charging

Sensor Controller PCB ● Optimized for low power consumption ● Simple operation with only a single hardware switch ● All control besides power-on handled through app

Power and Charging Schematic

Connector Schematic

FLASH Memory Schematic

Embedded System Initialization

Application Interface System

Application Interface System • Data port • Application Program Interface (API) • Sample Applications – Predictive model • LSTM Neural Network – Visualization

Data Acquisition

Data Filtering and Estimation

Data Prediction

Data Visualization

Accomplishments to Date

Sensor Reading ● Only read data if sensor gives proper “Who am I” ● Data is collected from on-board FIFO ● Sent to host over UART using CMOS to RS232

Initial Sensor Measurement • Used Realterm to save text files containing measurement data • Used Python to interpret saved measurement files and convert them to a MATLAB readable format • Used MATLAB to visualize the data and estimate position

Work In Progress

Revised Sensor Measurements • Used Python to read serial data from COM port • Used Python to estimate velocity and position data from raw acceleration data • Used Matplotlib to plot acceleration, velocity, and position

Revised Sensor Measurements (Cont.) Estimated Velocities Estimated Displacement

Kalman Filtering • Also called Linear Quadratic Estimation • Uses series of measurements containing errors and statistical noise • Produces estimates of unknown variables • More accurate than filtering using one measurement by using a joint probability distribution

Our Use of the Kalman Filter • To estimate velocity and position of the IMU from the acceleration given • To remove any noise and error attached to the incoming measurement data (i.e. sensor drift)

Future

RTOS • Used to keep precision timing on IMU samples • Will feature priority system and task scheduling – IMU sampling is high priority, FLASH writes are medium priority, and battery status reads are low priority • Allows better use of system resources • Avoids wasting processor time in delays • Also will feature a “diagnostic system” to alert host device of errors

Embedded Memory Controller • FLASH memory requires 8-bit parallel writing • SRAM FIFO used in combination with FLASH – Allows bulk writes to FLASH – Minimize time writing to FLASH • No hardware memory controller, must be created through software

Position Prediction Neural Network • Used to predict the next arm position based on past positions • A Recurrent Neural Network (RNN) with built-in long term memory also known as a Long Short Term Memory network (LSTM)

Smartphone Application • Receives streams of data • Used to visualize data • Used to send commands to the sensor controller

Division of Labor Jacob Erik • Design PCBs • Adding components to the • Develop RTOS IMU PCBs • Develop embedded • Developed Application subsystems (IMU, memory Interface controller, etc) • Worked on Android app development

Timeline

Questions?