Discrete Collabora.ve Filtering Hanwang Zhang 1 , Fumin Shen 2 , Wei - - PowerPoint PPT Presentation

▶
discrete collabora ve filtering
SMART_READER_LITE
LIVE PREVIEW

Discrete Collabora.ve Filtering Hanwang Zhang 1 , Fumin Shen 2 , Wei - - PowerPoint PPT Presentation

Jan 10, 2023 386 likes •636 views

Discrete Collabora.ve Filtering Hanwang Zhang 1 , Fumin Shen 2 , Wei Liu 3 , Xiangnan He 1 , Huanbo Luan 4 , Tat-Seng Chua 1 Presented by Xiangnan He 1. Na>onal University of Singapore 2. University of Electronic Science and Technology of China

slide-1
SLIDE 1

Discrete Collabora.ve Filtering

Hanwang Zhang1, Fumin Shen2, Wei Liu3, Xiangnan He1, Huanbo Luan4, Tat-Seng Chua1

1. Na>onal University of Singapore 2. University of Electronic Science and Technology of China 3. Tencent Research 4. Tsinghua University 19 July 2016

Presented by Xiangnan He

slide-2
SLIDE 2

Online Recommenda-on

  • An Efficient Recommender System
  • Latent Model: Binary Representa>on for Users and Items
  • Recommenda>on as Search with Binary Codes

Offline Training

  • End-to-end binary op>miza>on
  • Balanced and Decorrelated Constraint
  • Small SVD + Discrete Coordinate Descent

2

slide-3
SLIDE 3

User-Item Matrix Latent Space

Latent Factor Approach [Koren et al. 2009]

3

slide-4
SLIDE 4

Recommenda>on is Search Search in Euclidean space is slow Search in Hamming Space is fast.

Ranking by <user vector, item vector> Requires float opera>ons & linear scan of the data Only requires XOR opera>on & constant-.me lookup User-Item Database Hash Table Query Code

4

slide-5
SLIDE 5

[Zhang et al, SIGIR’14; Zhou et al, KDD’12]

  • Stage 1: Relaxed Real-Valued Problem

{B, D} ß Con-nuous CF Methods

  • Stage 2: Binariza>on

B ß sgn (B), D ß sgn (D)

Quan-za-on Loss Code learning and CF are isolated

5

slide-6
SLIDE 6
  • 1. A,B,a,b are close but they are separated into different quadrants
  • 2. C, d should be far but they are assigned to the same quadrant

6

slide-7
SLIDE 7

7

slide-8
SLIDE 8

Observed ra>ng User code Item code Ra>ng Predic>on Binary Constraint

However, it may lead to non-informa>ve codes, e.g.:

  • 1. Unbalanced Codes à each bit should have split the dataset evenly
  • 2. Correlated Codes à each bit should be as independent as possible

8

slide-9
SLIDE 9

Illustra>on of the effec>veness of the two constraints in DCF

Without any constraints: 3 points are (-1, -1) and 1 point is (+1, -1), which is not discrimina>ve. Balanced: Separated in the 1st & 3rd quadrant Decorrelated: Well separated

9

slide-10
SLIDE 10

However, the hard constraints of zero-mean and orthogonality may not be sa>sfied in Hamming space!

10

slide-11
SLIDE 11

Ra>ng Predic>on Constraint Trade-off Binary Constraint Balanced Constraint Decorrelated Constraint Delegate Code Quality Constraint

Mixed-Integer Programming NP-Hard [Hastad 2001]

Objec>ve func>on:

11

slide-12
SLIDE 12

Alterna>ve Procedure

  • B-Subproblem
  • D-Subproblem
  • X-Subproblem
  • Y-Subproblem

12

slide-13
SLIDE 13

For each user code bi, op>mize bit by bit

Parallel for loop over m users for loop over r bits

D-Subproblem can be solved in a similar way

Objec.ve Func.on

13

Usually converges in 5 itera>ons

slide-14
SLIDE 14

#bits #bit-by-bit itera>ons #training ra>ngs #compu>ng threads

Linear to data size!

14

slide-15
SLIDE 15

Objec.ve Func.on

Small SVD r x m Orthogonaliza>on r x m row-centered user code matrix

Y-Subproblem can be solved in a similar way

15

slide-16
SLIDE 16

16

#bits #users

Linear to data size!

slide-17
SLIDE 17
  • Recommenda>on is search
  • We can accelerate search by hashing
  • Unlike previous erroneous two-stage hashing,

DCF is an end-to-end hashing method

  • Fast O(n) discrete op-miza-on for DCF

17

slide-18
SLIDE 18
  • Dataset (filtering threshold at 10):
  • Random split: 50% training and 50% tes>ng.
  • Metric: NDCG@K
  • Search Protocol:

Hamming ranking or hash table lookup

18

slide-19
SLIDE 19
  • MF: Matrix Factoriza>on [Koren et al 2009]

Classific MF, Euclidean space baseline

  • BCCF: Binary Code learning for Collabora>ve Filtering

[Zhou&Zha, KDD 2012]

MF+balance+binariza6on

  • PPH: Preference Preserving Hashing [Zhang et al. SIGIR 2014]

Cosine MF + norm&phase binariza6on

  • CH: Collabora>ve Hashing [Liu et al. CVPR 2014]

Full SVD MF + balance + binariza6on

19

slide-20
SLIDE 20
  • 1. DCF learns compact and

informa>ve codes.

  • 2. DCF’s performance is most

close to the real-valued MF.

  • 3. End-to-end > Two stage

20

Performance of NDCG@10.

slide-21
SLIDE 21

21

Training: full histories of 50% users Tes>ng: the other 50% users that have no histories in training Evalua>on: simulate online learning scenario.

slide-22
SLIDE 22

MF: original MF MFB: MF+Binariza>on DCFinit: the variant of DCF that discards the two constraints.

22

slide-23
SLIDE 23
  • Discrete Collabora>ve Filtering: an end-to-end

hashing method for efficient CF

  • A fast algorithm for DCF
  • DCF is a general framework. It can be

extended to any popular CF variants, such as SVD++ and factoriza>on machines.

23

slide-24
SLIDE 24

24

Code available: hups://github.com/hanwangzhang/Discrete-Collabora>ve-Filtering