Moving Objects Removal PROCESAMIENTO DE IMÂGENES DIGITALES Group Project Andrea Migotto Fabio Falcioni Giacomo Santoni
INTRODUCTION Be able to get a photo of a landscape or a monument or PROBLEM: anything else without people or vehicles ruining the photo Take multiple photos with a static background and IDEA: process them to obtain a picture without unwanted objects
RESOLUTION These are the main steps of our program. In the next slide they have been schematically sketched for a better understanding. 1. Images acquisition 6. Connected components identification 2. Difference between images 7. Correlation computation and comparison 3. Canny detector 8. Final image updating 4. Binarization of the difference 9. Second execution 5. Elaboration of the binarized image
RESOLUTION
Images acquisition Two or more images can be selected for being processed using a normal pop-up dialog window. The more images are used, the easier it is to get a good final result. This is because a portion of the image is less probable to be covered in all photos by some moving object.
Difference between images The difference between two images is calculated to detect movements within the same scene. The regions that do not change between the images are black while all the moving objects are highlighted, in this way it’s possible to find what should be removed.
Edge detection The Canny filter is used to detect the edges of each moving object. An image processed by the Canny algorithm makes easier to search for individual objects in the image database. Searching for colour matching could lead to wrong results as the difference in absolute value has been used.
Binarization of the difference The difference is transformed into a black and white image thanks to the binarization. It consists in a reduction of the information on it, as you move to a representation formed only by true and false (1 and 0 respectively). A low threshold is used on the difference image to bring to 0 the background and to 1 all the differences.
Elaboration of the binarized image Through some different morphological operations and filters, the binarized image is processed to remove isolated white spots and to highlight moving objects. More precisely, a median filter, erosion, expansion and filling have been used to not consider small difference like those caused by small variation in the angle or in the brightness between the two images elaborated.
Connected components identification The connected components of the mask are extracted and their bounding boxes are identified in order to isolate and locate each moving object and store the information about its position in the image. Each component thus contains an object that now has to be recodgnize in one of the two initial images.
Correlation comparison For each connected component, in the corresponding region the correlation between the difference image and both the initial one and the provisional final one is computed and then compared. The object is then located in the image showing the higher correlation, while the other will contain the background of that part.
Final image The final image is then updated by replacing the unwanted objects with the corresponding background of the other image. Clearly having fewer background differences between the various images lead to a better result. Background differences, for example, include changes in light, shadow or small frame variations between shots
Second execution The result may be improved by choosing a “good” initial image. To do so each image of the set is compared with the final one that we just obtained and the set is sorted in order of descending correlation, then the algorithm is executed again. In practice this means choosing the image with fewer moving objects to make as few changes as possible to the image.
EXPERIMENTATION The resolution was done with a set of only two images to better explain the program. The script can also work with a larger datastore formed by many initial images in which there are numerous moving objects. In the following there are some examples with shots taken in Seville: - Municipio - Paseo del Rio - Plaza de España - Real Alcazar - Prado de San Sebastián - Campus Reina Mercedes
Municipio – Initial images
Municipio – Final image
Plaza de España – Initial images
Plaza de España – Final image
Prado de San Sebastián – Initial image
Prado de San Sebastián – Final images
Paseo del Rio – Initial Images
Paseo del Rio – Final Images
Real Alcazar – Initial images
Real Alcazar – Final image
Campus Reina Mercedes – Initial images
Campus Reina Mercedes – Final image
CONCLUSIONS After an exhaustive test phase with sets of pictures shot in different conditions, we can affirm that our method works well in real world situations and it’s rather versatile and can be implemented in a light and performing software. The vast majority of the moving objects, in particular we are interested in people, are correctly removed and the appearance of artifacts in the final image is minimal, if not inexistent. In conclusion, we can see that our method has its strengths and weaknesses, and presents interesting possibilities of improvement while it can already be used as a fully functional tool.
POSSIBLE IMPROVEMENTS Using more sophisticated and refined techniques during the processing of the images and the correlation computation will allow to achieve better results The code can be optimized to achieve better computational speed and use less resources Implement the possibility of using multiple shots even if they are not aligned using the SURF algorithm for features detection and matching
Moving Objects Removal PROCESAMIENTO DE IMÂGENES DIGITALES THANKS FOR YOUR ATTENTION
Recommend
More recommend
