Trade-off Between Computational Complexity and Accuracy in - PowerPoint PPT Presentation

Trade-off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction Wissam A. Albukhanajer, Yaochu Jin and Johann A. Briffa Wissam A. Albukhanajer (student) E: w.albukhanajer@surrey.ac.uk T: +44(0)1483 68 6051 CEC 2015, Sendai, Japan SS37: Evolutionary Feature Selection and Construction & SS13: Evolutionary Computer Vision Nature Inspired Computing and Engineering Department of Computing Faculty of Engineering and Physical Sciences University of Surrey United Kingdom GU2 7XH May 28 Thursday, 2015

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. In this Talk • Introduction – Traditional Trace Transform. – Triple Features. – Evolutionary Trace Transform. • Computational Complexity of the Trace Transform • Multi-Objective Parameters Tuning – Two-Objective Optimisation – Three-Objective Optimisation • Experimental Results – Selection of Knee Point Solutions – Performance Analysis – Complexity Analysis • Summary and Conclusion 2

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Introduction • Traditional Trace Transform (TT): – Inspired by Radon transform, proposed by Kadyrov and Petrou [15] – Trace Transform is a generalisation of Radon Transform. The functional calculated on the image pixels is not necessary the integral. – Different Trace Transforms can be produced Using different Trace functionals T . The Trace matrix An image [15] A. Kadyrov and M. Petrou, "The Trace transform and its applications," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, pp. 811-828, 2001. 3

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Introduction • Traditional Trace Transform (TT) (cont.) – Triple feature construction : by applying another two functionals: • Diametric functional D applied to the Trace Matrix to produce a vector of length ( 𝜄 ). • Circus functional C is applied to the diametric vector to produce a real number to describe the image called Triple Feature . Schematic diagram of Triple feature construction . 4

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Introduction • Evolutionary Trace Transform: Using NSGA-II [17] based Multi-objective optimisation (MOO) to search the best – combinations of Triple features functionals – The two objectives are: 1. Minimize the within-class feature variance (S w ) 2. Maximize the between-class feature scatter (S b ) where K is the number of classes, 𝐷 𝑙 is the number of samples in class 𝑙 , 𝜈 𝑙 is the mean of class 𝑙 of Ξ Triple features, Ξ 𝑘𝑙 is the j th Triple features of class 𝑙 , and 𝜈 Ξ is mean of all classes of Ξ Triple features. – NSGA-II is implemented using SHARK Machine Learning Library[16] [16] Christian Igel, Verena Heidrich-Meisner, and Tobias Glasmachers. Shark. Journal of Machine Learning Research 9, pp. 993-996, 2008. http://image.diku.dk/shark 5 [17] K. Deb, A. Pratap, S. Agarwal and T. Meyarivan, "A fast and elitist multiobjective genetic algorithm: NSGA-II," Evolutionary Computation, IEEE Transactions on, vol. 6, pp. 182-197, 2002.

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Introduction • Evolutionary Trace Transform in the Presence of Noise (ETTN) – In the evolutionary set up, sample images include three different classes, each containing five different types of changes. The major difference here is that Gaussian noise is added to the sample images apart from RST deformations: • Sample 1 : A low-resolution image from ( 64 × 64 ) generated from a randomly chosen original image ( 256 × 256 ); • Sample 2 : Random rotation, scale and translation of Sample 1 with Gaussian noise (standard deviation=4); • Sample 3 : Random rotation, scale and translation of Sample 1 with Gaussian noise (standard deviation=6); • Sample 4 : Random rotation of Sample 1; • Sample 5 : Random scale of Sample 1. – Therefore, there are 15 images in ETTN (20 sample images in ETT). [145] L. Fei-Fei, R. Fergus, and P. Perona, \One-shot learning of object categories," IEEE Transactions 6 on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 594-611, April 2006.

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Computational Complexity • Tuneable Parameters The Trace matrix of size 𝑜 𝜄 × 𝑜 𝜍 7

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Computational Complexity Traditional Trace Transform uses a thousand [87] (not optimised) features: • 𝑜 𝑢 = 𝑜 𝜍 = 𝑜 𝜄 = 100 , 0 ≤ 𝜄 ≤ 2𝜌 , and 𝑂 𝑈 = 𝑂 𝐸 = 𝑂 𝐷 = 10 • The largest number of sampling lines per rotation angle equals the number of pixels on the image diagonal, i.e., the maximum value of 𝑜 𝜍 is equal to 𝑂 2 + 𝑂 2 = 2𝑂 with Δ𝜍 = 1 . Randon Complexity (Big O): 𝑃(𝑂 2 𝑜 𝜄 ) • • The number of operations (in Big Theta notation) required to calculate a Triple feature is equal to: Θ(𝑂 𝑈 𝑜 𝑢 𝑜 𝜍 𝑜 𝜄 ) for 0 ≤ 𝜄 ≤ 𝜌 For 𝑜 𝜄 = 180, ∆𝜄 = 1 : complexity equals to Θ(𝑂 2 ) • When considering Δ𝜄, Δ𝜍 : Θ(𝑂 𝑈 𝑂 2 /Δ𝜄Δ𝜍) • [87] A. Kadyrov, A. Talepbour, and M. Petrou, \Texture Classification With Thousands of Features," in 13th British Machine Vision Conference. BMVC, 2-5 September 2002 2002, pp. 656{665. 8 [190] S. Tabbone, L. Wendling, and J.-P. Salmon, \A new shape descriptor defined on the Radon transform," Computer Vision and Image Understanding, vol. 102, no. 1, pp.42 -51, 2006.

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Multi-objective Trace Parameters Tuning • Parameters Tuning Using Two-Objective Optimisation – Two-objective minimisation with ∆𝜄 𝑏𝑜𝑒 ∆𝜍 as extra parameters to be tuned in addition to the Trace functionals T, D, and C. • Parameters Tuning Using Three-Objective Optimisation – Time complexity is added as a third objective: • Both integer and real coding for ∆𝜄 𝑏𝑜𝑒 ∆𝜍 are considered. 9

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Some Functionals 10

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Multi-objective Trace Parameters Tuning • Two-objective non-dominated fronts and a selection of solutions at the knee point Integer coding Search converges to value of 1 (better accuracy) Real coding 11

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Multi-objective Trace Parameters Tuning • Three-objective non-dominated fronts Real coding Integer coding 12

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Experimental Results • Two-Objective Parameters Tuning: RST with Gaussian Noise (COIL-20 database) Integer coding Real coding 13

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Experimental Results • Three-Objective Parameters Tuning: RST with Gaussian Noise (COIL-20 database) Integer coding Real coding 14

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Experimental Results • Average performance of three solutions of each method: 15

W. A. Albukhanajer et. al. “Trade -off Between Computational Complexity and Accuracy in Evolutionary Image Feature Extraction,” CEC2015 , Sendai, Japan. Experimental Results • Time complexity of three solutions of each method: From Table VI, we can see there are three different T functionals in all Triple feature pairs, which are T1, T3 and T5. which are And which have a time complexity O(N). The run time in the three-objective case is smaller than the run time in the two-objective case due to the time complexity being added as a third objective in the three-objective case, which provides fastest features but a relatively low performance. 16