Communication Networks and Computer Vision Based Control Nicholas - - PowerPoint PPT Presentation

Mentors

Yonggang Xu, James Riehl

Professor

João Hespanha

Ventura College Physics/Computer Science

Communication Networks and Computer Vision Based Control

Nicholas Tovar Nicholas Tovar

Communication Networks and Computer Vision Based Control

Control over a network

Use Client/Server protocols to interpret and process data

• ver the network
• Data consists of video/captured images and periodic sonar

readings.

• Networking using Ethernet(100 Mbits/s) or

Wireless(11 Mbits/s) connections

Examine the limitations of real time video processing on a lower bandwidth network

• Emphasis on real-time video interaction.
• Creating/Using algorithms for overall more efficient

client-side video processing

What is my Role?

Create our own re-usable code to control the robot

Camera manipulation and remote viewing

• PTZ (Pan-Tilt-Zoom) camera motion control
• Frame-by-frame image capture
• Client-side image processing
• Pattern recognition algorithm

Sonar sensor feedback and motion

• Sonar information retrieval and calculations
• Direct Integer or Byte Command Based movement

Network Connection

• Network programming connection using sockets

Code Example

Laboratory Process/Equipment

System Schematic

Internet & 802.11b SIP (Server Info Packet) CIP (Client Info Packet) Commands (PTZ) Frames Sonar Encoders Motors Micro-Controller Sony PTZ Camera

Beginning Steps

Images and Sonar

Real-time Examples

Real-time Video integrated with motion and sonar

Video and Network algorithm Incorporation

• High Resolution -> 640x480, 24 bit color (900KB)
• Compressed file -> 320X240, 8 bit grayscale (75KB)
• Resultant file is 12 times smaller

Object Avoidance using sonar

• Sonar directed movement.
• Robot avoids obstacles by turning when objects are

detected within a given range.

Video Footage

What I have Accomplished

Summary of Achievements

Understanding/Investigative

• Analyzed and understood robot manufacturer code and

robot specs

• Examined how the robot interacts with the network

Programming, C++

• Created my own “Basic Movement” class
• Coded an “Improved Sonar” class
• Utilized programming code that favored direct packet

communication

• Employed a pre-existing video and pattern recognition

algorithm

What’s Next?

Future Plans

• Combine Object Avoidance with vision control
• Integrate pattern recognition and image processing
• Refine sonar control

Acknowledgements

INSET, CNSI, UCSB, NSF Trevor Hirst, Liu-Yen Kramer, Nick Arnold, Mike Northern Yonggang Xu, James Riehl João Hespanha My Lab mates from ECE rm 5156 My Fellow INSET interns