For the Open-Set Data Classification Zhuoyi Wang, Bo Dong, Yu Lin, - - PowerPoint PPT Presentation

UT DALLAS

Erik Jonsson School of Engineering & ComputerScience

FEARLESS engineering

Co-Representation Learning Framework For the Open-Set Data Classification

Zhuoyi Wang, Bo Dong, Yu Lin, Yigong Wang, Latifur Khan Department of Computer Science The University of Texas at Dallas, Richardson TX, USA

This material is based upon work supported by

⮚ Open Data Problem:

– Continuous incoming of instances from various source domain. – New emerged categories that may non-linear separable with

existing classes.

– Limited past data to build model.

2 FEARLESS engineering

Scene image with new object in the autonomous system Flow of news with new emerged topic in the Social Network System.

Why Open Data Classification?

Challenge: Novel Class

Novel class

x1

3 FEARLESS engineering

Traditional lower dimension space dataset (Boundary between novel and existing class is clear) Previous work: ECSMiner[1], SAND[2], ECHO[3]. Novel Class Non-linear separable space of data set FASHION-MNIST (novel categories may share some attributes with seen class).

[1]. Al-Khateeb, T., Masud, M. M., Khan, L., Aggarwal, C., Han, J., & Thuraisingham, B. (2012, December). Stream classification with recurring and novel class detection using class-based ensemble. In 2012 ICDM.​ [2]. Haque, Ahsanul, Latifur Khan, and Michael Baron. "Sand: Semi-supervised adaptive novel class detection and classification over data stream." In AAAI 2016.​ [3]. Haque, Ahsanul, et al. "Efficient handling of concept drift and concept evolution over stream data." In 2016 ICDE.

Proposed Approach: RLCN

4 FEARLESS engineering

✓ Applying deep metric-learning to :

– Extract more powerful features from deep network – Learn more suitable latent space which improve the intra-class compactness and inter-class separateness, and – detecting any significant change(s) through confidence scores.

✓ Wight Pairwise-Constraint is used for classification:

– Distance metric for existing and novel class is learned on different samples for corresponding classification. – Training is semi-supervised.

✓ Temperature Scaling (distillation process for output calibration) ✓ Novel class detection is integrated.

Co-Representation Learning Framework

FEARLESS engineering 5

Weighted Pairwise-Constraint (WPC)

6 FEARLESS engineering

We define the weighted pairwise-constraint (WPC), which aims to transform the input x into a suitable latent feature space 𝜚(x). Comparing the original space in the Figure 1 (middle), after the mapping operation after WPC, the distance between similar pairs will be smaller and clustered together; the inter-class separation also achieved with a margin value τ + γ.

Figure 1

➢ Objective function (minimization problem)

7 FEARLESS engineering

Weighted Pairwise-Constraint (WPC)

➢ Weighted Constraint (Discussion)

The weight for pair (𝑦𝑗, 𝑦𝑘) in the WPL, could be derived from differentiating 𝑀𝑞𝑏𝑗𝑠 on S(𝑦𝑗, 𝑦𝑘) as:

8 FEARLESS engineering

Weighted Pairwise-Constraint (WPC)

We can find that a negative pair with a higher similarity would be assigned with a larger weight: Negative pair (In the figure below) is more informative than the dis-similar and inter- class pairs (Upside). The two similar samples from different classes could be distinguished by our WPC . Less weight contribute to 𝑀𝑞𝑏𝑗𝑠 Larger weight contribute to 𝑀𝑞𝑏𝑗𝑠

Open-World Classifier

9 FEARLESS engineering

Aims to compute the threshold Tnovel to reject instances from novel class and classify the existing categories’ examples. Let be the estimated class label for instance xt, different with softmax output, we apply the temperature scaling to calibrated the prediction confidence of the output: Suppose the instances from total k classes in the training set D, . For the xt, we could calculate the prediction probability that it’s corresponding ො 𝑧t for class Yi , i∈ {1,2 … 𝐿}:

ො 𝑧t

The threshold Tnovelfor class Yi could be generated through:

Experiments

10 FEARLESS engineering

➢ We use the following performance metrics:

– ERR: Total misclassication error (percent), i.e., ((FP+FN+Fe)*100)/N – Mnew: % of novel class instances Misclassified as existing class, i.e., (FN*100)/Nc – Fnew: % of existing class instances Falsely identified as novel class, i.e., (FP*100)/(N-Nc) Name of Data Set Number of Instances Number of Classes Number of Features

FASHION-MNIST[1] 70,000 10 784 MNIST [2] 70,000 10 784 CIFAR10[3] 70,000 10 4096 New-York Times 100,000 20 300

Guardian

130,000 20 300 SynRBF@0.003 [4] 130,000 7 70

Visulization of latent space

11 FEARLESS engineering

TSNE visualization of embeddings in original feature space (left) and transformed latent feature space (right) on FASHION-MNIST dataset

Classification Error and Executing Time

12 FEARLESS engineering

Comparison of classification performance on competing methods over

• pen-set settings.

Analysis of the execution time for the different training of model. RLCN is more efficient than other methods during different dimension of datasets.

Performance: Novel Class Detection

13 FEARLESS engineering

Novel Class detection performance over open-set data environment. Here - denotes failure of novel class detection. Parameter analysis of τ on novel class detection

Conclusion

14 FEARLESS engineering

➢ RLCN:

– is a co-representation learning framework which handles for open-set classification problem.

• Metric learning for learned suitable latent space for filter out novel

class.

– adjusts threshold with temperature scaling to find the most probability novel class instances. – does not require so many labeled data to maintain the classification accuracy for the novel class instances. – outperforms state of the art approaches both in classification accuracy and novel class detection.