Building Identification and Gaining Access Using AR Drone By: - - PowerPoint PPT Presentation
Building Identification and Gaining Access Using AR Drone By: Smith Gupta 11720 Dhruv Kumar Yadav 11253 Contents What is A.R. Drone? Problem Statement Object Classification Representation Learning Matching Dataset
By: Smith Gupta 11720 Dhruv Kumar Yadav 11253
▪ What is A.R. Drone? ▪ Problem Statement ▪ Object Classification
– Representation – Learning – Matching
▪ Dataset ▪ Future Work ▪ References
Source: appadvise.com
rotors
pitched propellers; 2 clockwise (CW) and 2 counter- clockwise (CCW)
control lift/torque
achieved by altering the rotation rate of one or more rotor discs, thereby changing its torque load and thrust/lift characteristics
Source: quadcopters.co.uk Source: wikipedia.com
Unmanned Aerial Vehicle
platform due to its robustness, mechanical simplicity, low weight and small size
following, position stabilisation, autonomous navigation and has wide applications in military reconnaissance and surveillance, terrain mapping and disaster management
Source: jazarah.net
▪ HARDWARE – WI-FI 802.11 b/g/n to communicate via ad- hoc network – 1Gbit DDR2 RAM at 200MHz – Front Camera ▪ 720p, 30fps HD Video Recording ▪ 75◦ × 60◦ field of view – Bottom Camera ▪ 176 × 144 p, 60 fps vertical QVGA camera
▪ Field of view 45◦ × 35◦
Source: ardrone2.parrot.com/ardrone-2/specifications/
Source: ardrone2.parrot.com/ardrone-2/specifications/
Source: ardrone2.parrot.com/ardrone-2/specifications/
▪ Autonomous identification of large structures like buildings from aerial imagery using an AR Drone ▪ The training and test data sets for identification include images of buildings at IIT Kanpur captured using front camera of AR Drone ▪ The identification task is to be done at run-time i.e., during the flight
▪ Upon recognition the Quadcopter gain access in the building via open portal - window or door
– UAV flying 3 Km to an abandoned village and identifying a structure based on symbol on the building – Upon identification the UAV has to access the structure through open portals (doors, windows, other openings) that had to be identified by the UAV
– Developing an autonomous system that will help people navigate through an unknown/GPS denied environment – This can be used for finding routes, or even as a tour guide
Feature Detection and Description Bag of Visual Words Model Keypoint Classification using SVM
Interesting points on the object that can be extracted to locate the
Object should be detectable under change of scale, rotation and noise
DATABASE TEST IMAGE
▪ Repeatable – Feature in one image can be found in other image ▪ Distinctive Description – Each feature has distinctive property ▪ Locally Salient – Occupy small area of image; robust to clutter
▪ Intuition: Match corners i.e. points with large variation in neighbourhood
REFERENCE: A COMBINED CORNER AND EDGE DETECTOR by Chris Harris & Mike Stephens [1988] Source: slides by Steve Seitz, Kristen Grauman, DevaRamanan
▪ Thus look at change in Intensity value ▪ Using Taylor series expansion ▪ Thus
REFERENCE: A COMBINED CORNER AND EDGE DETECTOR by Chris Harris & Mike Stephens [1988] Source: slides by Steve Seitz, Kristen Grauman, DevaRamanan
▪ Thus find the direction ([u v]) will result in largest and smallest Eigen values ▪ We can find the value by looking at Eigen vectors of H ▪ For pixel/ patch to be corner point; even the smallest Eigen value should be large enough ▪ Apart from smallest Eigen vector we can also look at Harris Corner Point
REFERENCE: A COMBINED CORNER AND EDGE DETECTOR by Chris Harris & Mike Stephens [1988] Source: slides by Steve Seitz, Kristen Grauman, DevaRamanan
▪ Harris Corner point is very sensitive to changes in image scale ▪ Although Harris corner can detect corners and highly textured points, it is not a good feature for matching images under different scales
REFERENCE: A COMBINED CORNER AND EDGE DETECTOR by Chris Harris & Mike Stephens [1988] Source: slides by Steve Seitz, Kristen Grauman, DevaRamanan
NOT DESIRABLE FOR OUR PROBLEM
▪ Intuition: Construct Scale space. Find the interest point in DoG space (Difference of Gaussian) by comparing a pixel with its 26 neighbouring pixels in the current and adjacent scales ▪ Eliminate edge points by constructing H matrix and look for Harris function with non maximal suppression
REFERENCE:Distinctive Image Features from Scale-Invariant Keypoints by David Lowe [2004]
▪ Orientation Assignment to each keypoint for rotation invariant. Descriptor now is represented relative to this orientation ▪ For each image sample L(x,y) at specific scale, gradient magnitude and orientation on the Gaussian smoothed images is pre-computed ▪ Create a weighted direction histogram in neighborhood of keypoint 36 bins ▪ Peak in the histogram correspond to the orientations of the patch
REFERENCE:Distinctive Image Features from Scale-Invariant Keypoints by David Lowe [2004]
▪ FEATURE DESCRIPTOR ▪ Based on 16*16 patches ▪ 4*4 subregions ▪ 8 bins in each subregion ▪ 4*4*8=128 dimensions in total
REFERENCE:Distinctive Image Features from Scale-Invariant Keypoints by David Lowe [2004]
Source: Jonas Hurreimann
▪ Split space of feature descriptors into multiple clusters using k- means algorithm ▪ Each resulting cluster cell is then mapped to a visual word ▪ Each Image is represented as histogram of these visual words
Vector quantization
▪ Types of Classifiers:
– One vs. All – One vs. One
▪ One vs all deals with all the data of all the samples thus consume more time ▪ Thus we plan to use one vs one to increase classification speed
Images of buildings at IIT Kanpur
AR Drone SDK 1.8 ffmpeg libraries
▪ For test data we use video streaming captured by the front camera of the drone ▪ Codec used by AR Drone 2.0 is H.264/MPEG-4
▪ Images can be extracted from this streaming video by using ffmpeg libraries ▪ These images would be given as inputs to SIFT algorithm at a certain frequency ▪ Each test image is matched with the database and probability measure is assigned ▪ If greater than threshold; image is successfully classified else look for another measure
▪ Compare between various Techniques like
– vocabulary tree – bag or words with k nearest neighbour
▪ Build SKYCALL like system at IIT KANPUR
– An autonomous flying quadcopter and a personal tour guide build at MIT Senseable City Lab – Guide prompts the users for the destination they want to reach – A mobile based application is being developed through which a user can “call” the guide for assistance
▪ [1] AR-Drone as a Platform for Robotic Research and Education-Tomas Krajnık, VojtechVonasek, Daniel Fiser, and Jan Faigl {2011} ▪ [2] Image target identification of UAV based on SIFT. - Xi Chao-jian,Guo San-xue {2011} ▪ [3] Architectural Building Detection and Tracking in Video Sequences Taken by Unmanned Aircraft System (UAS) Qiang He, Chee-Hung, Henry Chu and Aldo Camargo {2013} ▪ [4] Contextual Bag-of-Words for Visual Categorization -Teng Li, Tao Mei, In-So Kweon {2011} ▪ [5] A SIFT-SVM METHOD FOR DETECTING CARS IN UAV IMAGES - Thomas MORANDUZZO and Farid MELGANI {2012} ▪ [6] Multi-Information based Safe Area Step Selection Algorithm for UAV’S Emergency Forced Landing - Aiying Lu, Wenrui Ding and Hongguang Li {2013}
Source: http://s1.reutersmedia.net/