No Hands Landline Answering System By Vidit Talati, Gokul Krishnan, - - PowerPoint PPT Presentation

No Hands Landline Answering System

By Vidit Talati, Gokul Krishnan, and Mahendra Reddy (Group 15)

Client Need

Client seeks solution for patient, RB:

A quadriplegic due to multiple sclerosis Can speak, has normal cognitive function, and minimal use of right hand Cannot move any major limbs or use fingers to press with force to depress buttons

There is a need for the ability to use a landline telephone

specifically, answering incoming calls and placing calls Need for no physical exertion

Project Scope

Deliver a system that will enable a quadriplegic patient to answer landline phone calls using only voice commands and/or head movements Software component:

must allow for answering of phone calls, as well as disconnecting from calls should save contacts into a “phonebook” on the app should enable patient to dial at least ten digit numbers by inputting each digit individually through voice commands

Hardware component:

should enable to patient to attend calls away from the system, such as while in bed should maintain a sense of privacy

Design Specifications

Design Category Specific Design Requirement Metric Hardware (speakers, landline phone) Speaker frequency 100 Hz-23.5 kHz Speaker volume 70-80 dB Speaker weight less than 0.38 kg (see explanation right below). Speaker-to-landline range at least 10 m Speaker casing size no more than 65 mm x 65 mm x 40 mm Landline phone sensitivity Detect voices/voice commands as quiet as 45 dB Spatial speaker

• rientation

Can rotate from 0 to 80 degrees Speaker distance from client No less than 10 cm from ears Software GUI simplicity Train in 30 minutes or less Calling interface Allow digit-by-digit calling of 10 or more digits Memory Store a “phonebook” of at least 30 names Run speed App processing and action should take no more than 10 seconds after voice command Accuracy Maximum of 0 wrong numbers allowed; a restart button must be created in case of voice misinterpretation Portability Landline system weight No more than 2 kg Cost N/A less than $1000 Privacy N/A Person farther than 3 feet away cannot hear phone conversation Battery life N/A All pieces should last at least 12 hours Safety Injuries from hardware 0 injuries allowed from any of the hardware components.

Major Hardware Components

HS-422 Servo with Spline Arm Raspberry Pi (Model B, Revision 2.0) Jackco ZT51000 Bluetooth Speakers 4th generation iPad CanaKit WiFi Adapter

Raspberry Pi/Servo Circuitry Design

Circuit Schematic PCB Layout

Casing and Mounting Designs

Right Side Motor Mount Circuit Casing/Left Mount Lid Right Side Speaker Platform Left Side Speaker Platform

Design Feasibility - Hardware Calculations

When using a 6 V (or higher) power supply, the servo can produce a torque of 4.1 kg*cm Maximum mass movable by servo:

Battery Life

Servo motors draw 8.8 mA when still and 180 mA when “in motion”

batteries will only last

The Raspberry Pi Minicomputer draws a peak current of 515 mA

Raspberry Pi can be powered for a minimum of

Software Considerations

Twilio was chosen and is more feasible due to usability of its API Raspberry Pi was chosen over Arduino Voice Recognition Platforms (NSSpeechRecognizer, Siri, OpenEars)

OpenEars = very integrable API, end user usability

Software Design

Software Design cont.

Incoming calls - Answering phone call solution:

Initial problems - listening for voices while answering calls Solution Incoming Call is immediately rejected Phone number of caller shows up on app screen User says “Accept” to call person back

End calls by saying “Stop”

Software Design cont.

Contact Book

Press “Add a Contact” UIButton to add names NOT hands free

UISwitch toggle switches added for nurse’s convenience

Moving speakers inactivating voice recognition

Reset Button closes app

Software Design cont.

Parts, Materials, and Manufacturing

Quantity Part Product ID/Model Number Website/Store Obtained from Cost 2 Servo motor - HiTec 422 (Standard Size) HS-42217 SparkFun Electronics $9.95 x 2 = $19.9517 2 Standard Spline Arm 563611 ServoCity $5.99 x 2 = $11.98 1 1 VELCRO - Industrial Strength Low Profile - 10' x 1" Tape - Black 9110037 VELCRO brand (from Amazon.com) $10.6237 1 Jackco Black and Red Portable Bluetooth Speaker ZT5100020 Jackco (from Amazon.com) $36.9920 2 Eveready Super Heavy Duty Zinc Carbon 9V Battery 1222SW12 Eveready (from batteryjunction.com), but obtained from Widder’s Lab $2.49 x 2 = 4.9812 2 9 Volt Battery Snap-I Style-3" Wire Leads BS3I8 Digi-Key Electronics, but obtained from Widder’s lab $0.63 x 2 = $1.268 1 Raspberry Pi Model B, revision 2.0 RASPBRRY- PCBA51232 Raspberry Pi (from Amazon.com), but obtained from Widder’s lab $30.0032 1 Raspberry Pi Model B, revision 2.0 Case (Discontinued by Adafruit) Adafruit Product ID: 114027 adafruit.com/products/1140

• riginally, but obtained from

Widder’s lab $10.0027

Parts, Materials, and Manufacturing

CanaKit Raspberry Pi WiFi Adapter CKXW100033 Micro Center (87 Brentwood Promenade Ct, Brentwood, MO 63144) $9.9933 Gorilla Glue Single-Use Tubes, 4- Pack 500050216 Wal Mart (1900 Maplewood Commons Dr, Maplewood, MO 63143) $4.7516 Printable Circuit Board for our Circuit N/A http://fab.fritzing.org/ fritzing-fab13 For 1-4, $17.15 each; for 5-9, $12.24 each; for 10 or more, $9.79 each13 Aukey 12000 mAh Portable Power Bank Charger PB-N286 Amazon.com $14.996 Photive 12000mAh Dual USB Portable Battery Charger PH-PB12B26 Amazon.com $34.9926 Apple iPad 4 with Retina Display - 16GB, Wi-Fi, Black, with iOS 8.3 IPAD 4 16B BLK CR5 Apple (on Amazon.com) $314.995 Fing - Network Scanner Version 3.2.014 Apple App Store Free14

Hardware Assembly

After circuit was made, it was housed in a casing and mounted

mounted and lid attached using Velcro straps

Servos mounted onto case (left) and right side motor mount (right)

Speakers placed on platforms and mounted on HS-422 spline arms using Velcro

Issue: not sturdy, platforms wobble

SAFETY: Edges of platform were covered in felt-like material

Hardware Assembly cont.

*Motors are already mounted

Finished Hardware

Software Development

Integrated OpenEars platform into app Data flow within a Model-View-Controller System

Canakit WiFi adapter allows for wireless communication between Raspberry Pi and iPad app NSStream from app to Raspberry Pi’s IP address routes messages to Pi’s Linux terminal

Software/Hardware Integration

Conclusions

Speaker frequency 100 Hz-23.5 kHz No, but very close. iPads, which relays audio to the Jackco speakers are able to produce sound across 20-20k Hz. Speaker volume 70-80 dB Yes, the Jackco speakers can be set to this volume Speaker weight less than 0.38 kg (see explanation right below). Yes, the Jackco speakers weigh 8.5 ounces,

• r 0.241 kg With the platforms, they weigh

0.271 kg. Speaker-to-landline range at least 10 m Yes, the speakers-landline range can be extended to 10m using a long auxiliary cable

• r purchasing an aux-bluetooth adapter.

Landline phone sensitivity Detect voices/voice commands as quiet as 45 dB No, iPad cannot detect voice commands this

• quiet. However, client can buy an iPad

microphone Spatial speaker orientation Can rotate from 0 to 80 degrees Yes, the HS-422 motors can rotate 180 degrees. Speaker distance from client No less than 10 cm from ears Yes, 10.3cm away

Specific Design Requirement Metric Was the Design Spec Met? If Not, Why?

GUI simplicity Can be trained to use in 30 minutes or less Yes, the app is simple enough to be taught in about 10 minutes. Calling interface Allow digit-by-digit calling of 10 or more digits Yes Memory Store a “phonebook” of at least 30 names Yes, the amount of names the app can store is limited only by the storage space on the ipad, which is far more than 30 names. Run speed App processing and performing action should take no more than 10 seconds after client says voice command Yes, voice command processing takes a maximum of four seconds. Accuracy A maximum of 0 wrong numbers allowed; a restart button must be created in case the software misinterprets a voice command Yes

Conclusions

Conclusions

Cost less than $1000 Yes, the summed cost of all items in Table 2 is $505.49. Privacy Person farther than 3 feet away cannot hear phone conversation Yes, this was concluded from observations. Battery life Any piece of system not connected to

• utlet needs to last at least 12 hours

No, the batteries that run the motors will drain in 2.22 hours in the design’s current state. Injuries from hardware 0 scrapes, cuts, bruises or ear damage allowed from any of the hardware components. This is dependent on how careful the user and

• ther people walking past the system are.

Issues with Prototype

Bluetooth automatically disconnected after connection was made on Twilio went for auxiliary cable instead to extend range and save battery Mounting system is not transferrable to bed Nurse can transfer speakers to bed, but privacy will decrease Servos are constantly twitching

Future Directions

Stop servos from twitching Develop a more universal mounting system for use in wheelchair and in bed Create a way for Raspberry Pi’s IP Address to remain static Expand from single word commands to phrases

Lessons from the Design Process

Applications of computing devices like Raspberry Pi How to affect commands wirelessly Integrating multiple APIs into one application using available SDKs Troubleshooting and general circuit design

Our Design is not Intellectual Property

Much of the software has open source platforms incorporated in it

The unique aspects are methods that connect two APIs together and establish wireless connections with a computer

Need more sleek hardware Hardware may be too large and not completely up to date

Release of the Raspberry Pi Zero last week

References

1. "56361 Standard Spline." ServoCity. Robotzone, LLC, 2015. Web. 10 Nov. 2015. 2. "Activate Siri With Only Your Voice for a True Hands-Free Experience." OS X Daily. N.p., n.d. Web. 14 Oct. 2015. 3. "AN INTRODUCTION TO GPIO AND PHYSICAL COMPUTING ON THE RASPBERRY PI." GPIO: RASPBERRY PI MODELS A AND B. Raspberry Pi, n.d. Web. 20 Nov. 2015. 4. "Announced Specification of HS-422 Standard Deluxe Servo." Datasheets (n.d.): n. pag. SparkFun Electronics. Web. 19 Oct. 2015. 5. "Apple iPad 4 with Retina Display 16GB Wi-Fi Only Tablet, Black (Certified Refurbished)." Amazon. Apple, n.d. Web. 28 Nov. 2015. 6. "Aukey 12000 mAh Portable Power Bank Charger External Battery Pack with AIPower Tech for IPhone 6S 6S Plus 6 Samsung Galaxy Google Nexus LG and More (5V 2A Input, 5V 3.4A Output for Fast Charging) - Black." Amazon. Aukey, n.d. Web. 28 Nov. 2015. 7. Brown, Owen. "Arduino Tutorial: Integrating Bluetooth LE and IOS." Ray Wenderlich. N.p., n.d. Web. 14 Nov. 2015. 8. "BS31-Datasheet." Datasheets (2014): n. pag. Memory Protection Devices, Inc., 11 July 2014. Web. 20 Nov. 2015. 9. "Controlling Servo Motors." O'Reilly Raspberry Pi Cookbook. O'Reilly Media, Inc., 2013. Web. 28 Nov. 2015. 10.

• Djmason. "Create A Socket Based IPhone App and Server for Raspberry Pi."Bitcows. N.p., 5 Aug. 2014. Web. 13 Nov. 2015.

11.

• Djmason. "Controlling a Raspberry Pi with your iPhone Create A Socket Based IPhone App and Server for Raspberry Pi."Bitcows. N.p., 5 Aug. 2014. Web. 13 Nov. 2015.

12. "Energizer Eveready Super Heavy Duty Zinc Carbon 9V Battery (1222SW)." BatteryJunction.com: Gear Up. Power On. Battery Junction, n.d. Web. 20 Nov. 2015. 13. "Fab." Fritzing Fab. Fritzing, n.d. Web. 25 Nov. 2015. 14. "Fing - Network Scanner on the App Store." App Store. Apple, n.d. Web. 28 Nov. 2015. 15.

• Forcie. "RasPi Power Usage Measurements ALL Models." Raspberry Pi. N.p., 17 June 2012. Web. 28 Nov. 2015. Forcie. "RasPi Power Usage Measurements ALL

Models." Raspberry Pi. N.p., 17 June 2012. Web. 28 Nov. 2015. 16. "Gorilla Glue Single-Use Tubes, 4-Pack." Walmart. Gorilla Glue, n.d. Web. 24 Nov. 2015. 17. "HS-422 Super Sport." ServoCity. Robotzone, LLC, n.d. Web. 28 Nov. 2015. 18. "iPad (4th Generation) - Technical Specifications." Apple.com. Apple, n.d. Web. 26 Nov. 2015. 19. "IPhone Programming - Reading/Writing .plist Files." YouTube. Milmers Xcode, n.d. Web. 14 Nov. 2015. 20.

• Jackco. Sound2 by Jackco ZT51000 User Manual. Norwich: TSO, 2005. Jackco. Web. 29 Oct. 2015.

References cont.

1. "Jackco ZT51000 Black and Red Portable Bluetooth Speaker." Amazon. Jackco, n.d. Web. 29 Oct. 2015. 2.

• Matt. "Running A Python Script At Boot Using Cron." Raspberry Pi Spy. N.p., n.d. Web. 21 Nov. 2015.

3. "Mpow Streambot 2-In-1 Wireless Bluetooth Audio Music Streaming Switchable Transmitter and Receiver With 3.5mm Stereo Output - Connect Your PC, IPhone, IPod, IPad, Tablets Or MP3 Player To Speakers And Entertainment Systems, Home Or Ca." Mpow, n.d. Web. 29 Nov. 2015. 4. "NSLinguisticTagger." NSLinguisticTagger Class Reference. Apple, n.d. Web. 8 Nov. 2015. 5. "NSSpeechRecognizer." NSSpeechRecognizer Class Reference. Apple, n.d. Web. 8 Nov. 2015. 6. "Photive 12000mAh Dual USB Portable Battery Charger. Powerful External USB Battery Charger for IPad Air,Air 2, Mini, IPhone 6, 6 Plus, 5S, 5C, 5, 4S, Galaxy S5, S4, S3, Note 3, Nexus 4, 5, 7, 10, HTC One, One 2 (M8) and More." Amazon. Photive, n.d. Web. 28 Nov. 2015. 7. "Pi Shell - Clear Raspberry Pi Model A or B Case." Adafruit. Adafruit Technologies, n.d. Web. 20 Nov. 2015. 8. "Pricing." Voice for Phone Numbers & Calls. Twilio, n.d. Web. 8 Nov. 2015. 9. "Politepix." OpenEars. PolitePix, n.d. Web. 27 Nov. 2015. 10. "Power Modern Communications. Build the next Generation of Voice and SMS Applications." Twilio. Twilio, n.d. Web. 27 Nov. 2015. 11. "Raspberry Pi Model B." (n.d.): n. pag. Docs-europe.electrocomponents.com. Raspberry Pi. Web. 17 Nov. 2015. 12. "Raspberry Pi Model B 756-8308 Motherboard (RASPBRRYPCBA512)." Amazon. Raspberry Pi, n.d. Web. 18 Nov. 2015. 13. "Raspberry Pi Wifi Adapter." Raspberry Pi Wifi Adapter. CanaKit, n.d. Web. 12 Nov. 2015. 14. "Servo - Hitec HS-422 (Standard Size)." Sparkfun. SparkFun Electronics, n.d. Web. 15 Oct. 2015. 15.

• Subhransu. "Building an IOS App like Siri." Ray Wenderlich. N.p., n.d. Web. 8 Nov. 2015.

16. "Twilio Client for IOS Quickstart Tutorial." Twilio Cloud Communications. N.p., n.d. Web. 11 Nov. 2015. 17. "VELCRO - Industrial Strength Low Profile - 10' X 1" Tape - Black." Amazon. VELCRO Brand, n.d. Web. 1 Nov. 2015. 18. "Why Does My IP Keep Changing?" WhatIsMyIPAddress.com. N.p., n.d. Web. 21 Nov. 2015. 19. "Writing Hangout Apps." Google Developers. N.p., n.d. Web. 8 Nov. 2015.

Thank you!

Are there any questions?