Prototype Selection Using Polyhedron Curvature Benyamin Ghojogh, - PowerPoint PPT Presentation

Anomaly Detection and Prototype Selection Using Polyhedron Curvature Benyamin Ghojogh, Fakhri Karray, Mark Crowley Canadian AI conference, 2020 1

Anomaly Detection • finding outliers or anomalies which differ significantly from the normal data points • fraud detection, intrusion detection, medical diagnosis, and damage detection • Some methods: • Local Outlier Factor (LOF) • One-class SVM • Elliptic Envelope (EE) • Isolation forest 2

Prototype Selection • also referred to as instance ranking and numerosity reduction • Two versions: • Ranking based • Retaining based • Some methods: • Edited Nearest Neighbor (ENN) • Decremental Reduction Optimization Procedure 3 (DROP3) • Stratified Ordered Selection (SOS) • Shell Extraction (SE) • Principal Sample Analysis (PSA) • Instance Ranking by Matrix Decomposition (IRMD) 3

Polyhedron Curvature • Polytope: a geometrical object in R^d whose faces are planar • Special cases: • Polygon: polytope in R^2 • Polyhedron: polytope in R^3 • Consider a polygon where τj and µj are the interior and exterior angles at the j-th vertex • we have τj +µj = π 4

Polyhedron Curvature • Thomas Harriot’s theorem proposed in 1603: • if this geodesic on the unit sphere is a triangle, its area is µ1 +µ2 +µ3 −π = 2π −(τ1 +τ2 +τ3) • generalization of this theorem from a geodesic triangular polygon (3-gon) to an k-gon 𝑙 is µ1 + · · · + µk − kπ + 2π = 2π − σ 𝑏=1 τa 𝑙 • Descartes’s angular defect: 2π − σ 𝑏=1 τa 5

Polyhedron Curvature 𝑙 • Descartes’s angular defect: D(x) = 2π − σ 𝑏=1 τa • total defect of a polyhedron with v vertices, e edges, and f faces is: 𝑙 D := σ 𝑏=1 D(xi) = 2π(v − e + f). • T erm v − e + f is Euler -Poincare characteristic of the polyhedron • The smaller τ angles result in sharper corner of the polyhedron • So, we can consider the angular defect as the curvature of the vertex 6

Curvature Anomaly Detection (CAD) • Every data point is considered to be the vertex of a hypothetical polyhedron • For every point, we find its k-Nearest Neighbors (k-NN) • The k neighbors of the point (vertex) form the k faces of a polyhedron meeting at that vertex. • The more curvature that point (vertex) has, the more anomalous it is, i.e., far away (different) from its neighbors • So, anomaly score s_A is proportional to the curvature. 7

Curvature Anomaly Detection (CAD) 𝑙 • Descartes’s angular defect: 2π − σ 𝑏=1 . Hence, curvature is τa proportional to minus the summation of angles • S_A(xi) ∝ 1/ τ a ∝ cos( τ a) 𝑙 • S_A(xi) := σ 𝑏=1 cos(τa) = 𝑙 σ 𝑏=1 (x’_a x’_a+1) / (||x’_a||_2 ||x’_a+1||2) • x’_a := x_a − x_i • Relaxation: Relaxation is valid 8

Curvature Anomaly Detection (CAD) • Finding anomalies (training data): • Scree plot • K-means with two clusters: Cluster with larger mean is anomaly • Finding anomalies (out-of-sample): • k-NN for the out-of-sample point where the neighbors are from the training points • Calculate anomaly score • Compare with the means of clusters 9

Kernel Curvature Anomaly Detection (K-CAD) • Pattern of normal and anomalous data might not be linear. • Done in feature space: (1) finding k-NN, (2) calculating the anomaly score • Kernel: k(x1, x2) := φ(x1)^T φ(x2) • Euclidean distance in the feature space: • Normalize the kernel: 10

Kernel Curvature Anomaly Detection (K-CAD) • Score: • anomaly score in K-CAD is ranked inversely for some kernels such as Radial Basis Function (RBF), Laplacian, and polynomial (different degrees) • Reason: future work • M ultiply the scores by −1 or take the K -means cluster with smaller mean as the anomaly cluster 11

Anomaly Landscape • anomaly landscape: the landscape in the input space whose value at every point in the space is the anomaly score computed by CAD or K- CAD. • two types of anomaly landscape: • all the training data points are used for k-NN • or merely the non-anomaly training points are used for k-NN 12

Anomaly Paths • anomaly path: the path that an anomalous point has traversed from its not-known-yet normal version to become anomalous. Conversely, it is the path that an anomalous point should traverse to become normal • anomaly path can be used to make a normal sample anomalous or vice-versa 13

Inverse Curvature Anomaly Detection (iCAD) • Score: • Two versions: • Rank based: ranking the points with the ranking score • Retaining based: apply K-means clustering, with two clusters, to the ranking scores and take the points of the cluster with larger mean 14

Kernel Inverse Curvature Anomaly Detection (K-iCAD) • Scores: • iCAD and K-iCAD are task agnostic: • Classification: apply the method for every class • Regression and clustering: the method is applied on the entire data. 15

Experiments: anomaly landscape 16

Experiments: anomaly paths 17

An application in image denoising 18

Experiments: anomaly detection In most cases, K-CAD has better performance than CAD In many cases, we are better than the baseline methods We are also very fast 19

Experiments: Effect of k Almost robust to change of k 20

Experiments: prototype selection on synthetic data 21

Prototype selection Outperform many of the baseline methods: • in both accuracy and time • In both ranking and retaining based approaches 22

Future Direction • Try the idea of curvature for manifold embedding to propose a curvature preserving embedding method. 23