Analogy-Based Preference Learning with Kernels Mohsen Ahmadi Fahandar - - PowerPoint PPT Presentation

▶

Feb 23, 2024 264 likes •497 views

Analogy-Based Preference Learning with Kernels Mohsen Ahmadi Fahandar , Eyke Hllermeier Intelligent Systems and Machine Learning Group Heinz Nixdorf Institute and Department of Computer Science Paderborn University KI 2019, Wednesday,

SLIDE 1

Mohsen Ahmadi Fahandar, Eyke Hüllermeier

Analogy-Based Preference Learning with Kernels

Intelligent Systems and Machine Learning Group Heinz Nixdorf Institute and Department of Computer Science Paderborn University

KI 2019, Wednesday, September 25th

SLIDE 2

Our Contribution

Intelligent Systems and Machine Learning 2

Kernel-based Machine Learning Analogical Reasoning Generalized (fuzzy) Equivalence Relations

SLIDE 3

Analogical Reasoning

Intelligent Systems and Machine Learning 3

 Formalization of analogical reasoning based on the notion of analogical proportion (Miclet and Prade 2009; Prade and Richard 2017)

(pictures from ImageNet)

: :: :

SLIDE 4

Analogical Proportions

Intelligent Systems and Machine Learning 4

   

Formalizing the “as” part Domain-based instantiation

 Mathematically, a predicate on four objects

Satisfy the set of axioms

SLIDE 5

A nalogical Proport ions ( numerical case)

Intelligent Systems and Machine Learning 5

  Example

 Generalization of Boolean case: the four objects are in analogy to some degree

SLIDE 6

Ext ension t o Feat ure Vect ors

Intelligent Systems and Machine Learning 6

 Extension from individual attributes to feature vectors

SLIDE 7

Analogy and Kernels

Intelligent Systems and Machine Learning 7

Image from https://testinternetspeed.org/blog/internet-connection-speed/

similarity measure analogical proportion (by definition) defines a kind of similarity Key observation

SLIDE 8

Bridging Concept

Intelligent Systems and Machine Learning 8

   Hence capture the notion of similarity

Reflexive
Symmetric
T-transitive

SLIDE 9

Connection

Intelligent Systems and Machine Learning 9

Kernel-based Machine Learning

Motivation: certain types of fuzzy equivalence relations satisfy the properties of a kernel function (Moser 2006)

SLIDE 10

Kernels

Intelligent Systems and Machine Learning 10

  linearization kernel trick

 Symmetric  Positive semi-definite

SLIDE 11

Analogical Proportions as Kernels: analogy-kernel

Intelligent Systems and Machine Learning 11

 

SLIDE 12

Kernel-preserving Operations

Intelligent Systems and Machine Learning 12

 Extending the analogy-kernel from individual variables to feature vectors  To allow for incorporating a certain degree of non-linearity

SLIDE 13

An Application: Preference Learning

Intelligent Systems and Machine Learning 13

Predicted Ranking Ground Truth Ranking

(normalized) ranking loss: Query

SLIDE 14

Inference Pattern (Ahmadi Fahandar and Hüllermeier AAAI-2018)

Intelligent Systems and Machine Learning 14



(presumably)

: :: :

Analogy assumption Known knowledge

+

(pictures from ImageNet)

SLIDE 15

Analogy-Kernel-Based Object Ranking (AnKer-rank)

Intelligent Systems and Machine Learning 15

1. Pairwise preference:
2. Rank aggregation: This preference relation is turned into an overall consensus ranking

SLIDE 16

AnKer-rank: 1. Prediction of Pairwise Preferences

Intelligent Systems and Machine Learning 16

  

Predictions in the unit interval using Platt-scaling (Plat 1999)



SLIDE 17

AnKer-rank: 2. Rank Aggregation

Intelligent Systems and Machine Learning 17

 Bradley-Terry-Luce (BTL) model (Bradley and Terry 1952)   Predicted ranking: sort objects in descending order of their estimated parameter

Ahmadi Fahandar et al., Statistical Inference for Incomplete Ranking Data: The Case of Rank-Dependent Coarsening (ICML 2017)

SLIDE 18

Baselines

Intelligent Systems and Machine Learning 18

 Nearest Neighbor-based principle

able2rank (Ahmadi Fahandar and Hüllermeier AAAI-2018)

 Linear Regression-based principle

Expected Rank Regression (ERR) (Kamishima et al., 2010; Kamishima and Akaho 2006)

 SVM-based principle

RankingSVM (Joachims 2002)

 Neural Network-based principle

RankNet (Burges et al., 2005)

SLIDE 19

Intelligent Systems and Machine Learning 19

SLIDE 20

Experimental Setup

Intelligent Systems and Machine Learning 20

 AnKer-rank and able2rank: Rescaling of feature vectors to take values in the unit interval  ERR, RankingSVM and RankNet: Standard normalization  Hyper-parameters: fixed using (internal) 2-fold CV (repeated 3 times)

SLIDE 21

Intelligent Systems and Machine Learning 21

 Quite competitive in terms of predictive accuracy  On a par with able2rank and Ranking SVM  ERR and RankNet show worse performance

SLIDE 22

Summary and Future Work

Intelligent Systems and Machine Learning 22

 Connecting kernel-based machine learning and analogical reasoning in the context of preference learning  Building on the observation that analogical proportions define a kind of similarity  Utilizing generalized (fuzzy) equivalence relations as a bridging concept  Introducing analogy-kernel  Advocating a concrete kernel-based method for object ranking  First experimental results on real-world data from various domains are quite promising  To study kernel properties of other analogical proportions (e.g., geometric)  To study other types of applications, whether in preference learning or beyond  To study the use of kernel-base methods other than SVM

THANKS

My homepage https://github.com/mahmadif/able2rank