Labeling Information Enhancement for Multi-label Learning with - - PowerPoint PPT Presentation

An Tao*, Ning Xu, and Xin Geng Southeast University, China

Labeling Information Enhancement for Multi-label Learning with Low-rank Subspace

Outline

1

Introduction

2

The LIEML Algorithm

3

Experiments

4

Conclusion

1

Introduction

Introduction

We call such label as logical label.

• Each label is regarded to be either relevant or irrelevant to the instance.

Label set: Y = {sky, water, cloud, beach, plant, house}

Labeling information in multi-labellearning is categorical in essence.

Features: each pixels in the picture

In traditional multi-label learning:

Introduction

However…

Q: The two pictures can’t be differed with only logical labels. A: To describe the pictures better, we extend the logical label to be numerical. Same logical label set Y = {sky, water, cloud, beach, plant, house}

This new label is called numerical label.

Introduction

Specification of numerical label Specification of logical label

Use 𝒛" ∈ {−1,1}) to denote the logical label vector. Use to denote the numerical label vector.

• Label element of 𝒛" = 1 : relevant to the instance.
• Label element of 𝒛" = −1 : irrelevant to the instance.
• Label element of

> 0 : relevant to the instance.

• Label element of

< 0 : irrelevant to the instance.

• Absolute value of label element of

: reflects the degree to which the label describes the instance.

Introduction

Logical Label Numerical Label

Label Enhancement (LE)

Multi-label Model

Predictive Model Induction

Feature Overview of our LIEML algorithm for multi-label learning

+

Step 1: Step 2:

ü LE can be seen as a data preprocessing step which aims to facilitate in learning a better model.

2

The LIEML Algorithm

The LIEMLAlgorithm

Label Enhancement

Linear Model:

Symbol Definition:

• : Dataset
• : i-th instance vector
• : i-th logical label vector
• : i-th numerical label vector
• is a weight matrix.
• is a bias vector.

The LIEMLAlgorithm

Label Enhancement

Linear Model:

• For convenient describing, we set

.

The model becomes:

’

Label Enhancement

We then construct a stacked matrixZ:

Target Matrix

The LIEMLAlgorithm

The optimization problem of LE becomes:

Label Enhancement

’

• L(Z): logistic loss function

Not deviate too much

ü It prevents the label values in Z from deviating the original values too much.

The LIEMLAlgorithm

Label Enhancement

Low-rank Assumption A: We assume that the stacked matrix Z belongs to an underlying low-rank subspace.

’

Not deviate too much Rank(Z)↓

• L(Z): nuclear norm

and squared function Q: Why construct the stacked matrix Z? ü The stacked matrix Z is therefore an underlying low-rank matrix.

Labeling Information↑

The LIEMLAlgorithm

Label Enhancement

’

Not deviate too much Rank(Z)↓

The target function T1 for optimization is yielded as:

The LIEMLAlgorithm

Predictive Model Induction

The target function T2 we wish to minimize is: We build the learning model through an adapted regressor based on MSVR.

• , ,

, , , , , , . .

• is a nonlinear transformation of x to a higher dimensional feature space.

The LIEMLAlgorithm

3

Experiments

Experiments

Experiment Configuration

Five evaluation metrics widely-used in multi-label learning:

• BR, CLR, ECC, RAKEL, LP, and ML2

Six well-established multi-label learning algorithms:

• Ranking-loss, One-error, Hamming-loss, Coverage, and Average precision

Ten benchmark multi-label data sets:

Experiments

Experimental Results

Experiments

Experimental Results

Experiments

Experimental Results

LIEML ranks 1st in the most cases! The model of LE in LIEML is linear, but nonlinear in ML2, it is uneasy for LIEML to beat ML2 with the less efficient linear way. ü The results of the experiment validate the effectiveness of our LIEML algorithm for multi-label learning.

4

Conclusion

My personal website: https://antao.netlify.com/

Conclusion

This paper proposes a novel multi-label learning method named LIEML, which enhances the labeling information by extending logical labels into numerical labels. The labeling information is enhanced by leveraging the underlying low- rank structure in the stacked matrix.

Major Contribution More Information

Our lab website: http://palm.seu.edu.cn/

Thank You!

The LIEMLAlgorithm

Label Enhancement

To optimize the target function T1 :

Alternative Step 1: gradient decent Step 2: shrinkage ü To improve the speed of convergence, we begin with a large value µ1 for µ , and decay as .

The LIEMLAlgorithm

Predictive Model Induction

To minimize T2 (Θ ;m ), we use an iterative quasi-Newton method called Iterative Re-Weighted Least Square (IRWLS). The solution for the next iteration ( Θ(k+1

) and m( k+1) ) of

is obtained via a line search algorithm along ( Θ and m ) .

• ,

, , , , .

≈ The quadratic problem can be solved as:

Experiments

Experiment Configuration

Experiments

Experiment Configuration

Zhi-Hua Zhou, and Min-Ling Zhang. "Multi-label Learning." (2017): 875-881.

Introduction

In traditional multi-label learning:

An instance

Some features Label 1 Label 2 Label d . . . Yes Yes No learning Model . . . No No Yes . . . predicting