BIS: Bidirectional Item Similarity for Next-Item Recommendation - - PowerPoint PPT Presentation

BIS: Bidirectional Item Similarity for Next-Item Recommendation

Zijie Zeng Weike Pan* Zhong Ming*

National Engineering Laboratory for Big Data System Computing Technology College of Computer Science and Software Engineering Shenzhen University zengzijie1991@gmail.com, {panweike,mingz}@szu.edu.cn

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 1 / 29

Introduction

Problem Definition

Figure: Illustration of next-item recommendation.

Formally, we have n users and their observed action lists, i.e. I = {I

′

1, I

′

2, ..., I

′

n}, and each list consists of items that are sorted by

the user-item interaction timestamps, i.e., I

′

u = [i1 u, i2 u, ..., i|I

′ u|

u

]. For each user, our task is to build a model capable of predicting the next item that is most likely to be interacted with by the user in the near future.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 2 / 29

Introduction

Motivation

FPMC [Steffen Rendle and Schmidt-Thieme, 2010] applies Markov chains to the process of factorizing the user-item interaction matrix. Fossil [He and Mcauley, 2016] utilizes Markov chains in a way similar to that

• f FPMC. However, Fossil is based on FISM [Kabbur et al., 2013] and

factorizes the item-item matrix instead. ...... Most sequential CF methods are model-based. However, memory-based CF methods for next-item recommendation can rarely be seen. This motivates us to develop memory-based CF methods for sequential recommendation.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 3 / 29

Introduction

Notations

Table: Some notations and explanations.

Symbol Meaning n ∈ N+ user number m ∈ N+ item number U the whole set of all users ski the similarity between item k and item i Iu/I

′

u

the set/list of user u’s interacted items I the set of all users’ lists of interacted items Ni the nearest neighbors of item i Ik

u,latest

the set of user u’s latest interacted items s(ℓ)

j→i

the sequence-oriented directional item similarity (BIS(ρ = 0)) from j to i s(ℓ,ρ)

j→i

the sequence-oriented bidirectional item similarity (BIS) from j to i δ(x) indicator function that returns 1 if x is true and 0 otherwise Tmax the maximum timestamp in the training set Tmin the minimum timestamp in the training set tuk ∈ N+ the timestamp of the record (u, k)

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 4 / 29

Introduction

Overall of Our Solution

The proposed method is based on the framework of time-aware item-based CF: ˆ rui =

• k∈Iu∩Ni

w(tuk) · ski We propose a novel similarity measurement called sequence-oriented bidirectional item similarity (BIS). We develop a compound weighting function which is based on the user’s active session window [Mobasher et al., 2002] and exponential function [Ding and Li, 2005]. We apply BIS and the compound weighting function to time-aware ICF framework and propose a novel collaborative filtering method.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 5 / 29

Method

Sequence-oriented Bidirectional Item Similarity

Mathematically, the sequence-oriented bidirectional item similarity from item j to item i is as follows: s(ℓ,ρ)

j→i =

• u∈U δ(j ∈ Iu) · δ(i ∈ Iu) · δ(−ρ · ℓ ≤ pu(i) − pu(j) ≤ ℓ)

|Uj ∪ Ui| . (1) Item position: we use pu(j) to denote the position of item j in user u’s action list. Note that the items in the list are sorted by the timestamps of the corresponding user-item interactions in ascending order. Maximum gap: we use ℓ as a threshold to identify whether two items are associated in a specific user’s action list. Reverse factor: we introduce a reverse factor ρ into BIS in order to better adapt it to real-world data.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 6 / 29

Method

A Toy Example of BIS (1/2)

We have 4 users and their action lists. Note that items are ordered by the timestamps in each list. For comparison, we first use Jaccard index to measure the similarity between item b and item c: sbc = |Ub ∩ Uc| |Ub ∪ Uc| = 4 4 = 1.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 7 / 29

Method

A Toy Example of BIS (2/2)

Table: The calculation of the BIS from item b to item c.

User pu(c) − pu(b) δ(−1 ≤ pu(c) − pu(b) ≤ 2) u1 3 − 1 = 2 1 u2 1 − 0 = 1 1 u3 1 − 0 = 1 1 u4 0 − 2 = −2 Then we use BIS to measure the similarity from item b to item c: s(2,0.5)

b→c

=

• u∈U δ(b ∈ Iu) · δ(c ∈ Iu) · δ(−1 ≤ pu(c) − pu(b) ≤ 2)

|Ub ∪ Uc| = 3 4, where we set the maximum gap ℓ and reverse factor ρ to 2 and 0.5, respectively.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 8 / 29

Method

The Proposed CF Method

With a compound weighting function and the proposed BIS, we reach

• ur proposed collaborative filtering method:

ˆ rui =

• j∈Iu∩Ni

w(u, j) · s(ℓ,ρ)

j→i =

• j∈Iu∩Ni

wactive(u, j) · we(tuj) · s(ℓ,ρ)

j→i ,

(2) where the compound weighting function consists of two functions: Exponential function [Ding and Li, 2005]: we(t) = e− Tmax −t+1

p·Tmax .

(3) User’s active session window [Mobasher et al., 2002]: wactive(u, j) =

• 1,

j ∈ Ik

u,latest

0,

• therwise

. (4)

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 9 / 29

Experiments

Datasets (1/3)

We conduct our studies on two public dataset, i.e., MovieLens 10M and Netflix. MovieLens 10M contains 10 million ratings ranging from 0.5 to 5 with a step size of 0.5. The ratings are assigned by 71567 users to 10681 movies. Netflix contains about 0.1 billion ratings in the range of {1, 2, 3, 4, 5} assigned by 480189 users to 17770 movies. We preprocess the rating records of each data as follows: We remove the records whose rating value is smaller than 5 from the raw data. We remove the records of the users who rated fewer than 10 times from the above processed data.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 10 / 29

Experiments

Datasets (2/3)

To construct validation set, test set and training set from the preprocessed data: We sort the records of each user by the timestamps in ascending

• rder.

We then split the records of each user into two parts, i.e., the first mu − 1 records and the last record (mu denotes the number of items in user u’s action list). The first mu − 1 records are used for model training while the last record is distributed to test set with a 50% probability, or to valid set with a probability of 50%.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 11 / 29

Experiments

Datasets (3/3)

Table: Statistics of the processed data used in the experiments.

User # Item # Training record # Test record # Validation record # MovieLens 10M 40,600 8,625 1,370,625 20,228 20,372 Netflix 329,549 17,747 21,856,804 164,741 164,808

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 12 / 29

Experiments

Baselines (1/7)

We conduct empirical studies in order to verify the following three hypotheses:

1

We believe that our proposed bidirectional item similarity can capture the item correlations better than the existing ones such as Jaccard index and cosine similarity.

2

We believe that the proposed compound weighting function can better weigh the importance of the corresponding similarity in the prediction rule.

3

We believe that the proposed collaborative filtering method with BIS and compound weighting function can recommend the next item more accurately. Hence, we include several baselines of collaborative filtering with different similarity measurements and weighting functions in our empirical studies.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 13 / 29

Experiments

Baselines (2/7)

JI: item-based CF with Jaccard index as the similarity measurement. CS: item-based CF with cosine similarity as the similarity measurement. JI-uWIN: time-aware item-based CF using user’s active session window in Eq.(4) and Jaccard index.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 14 / 29

Experiments

Baselines (3/7)

JI-WIN: time-aware item-based CF using WIN: ww(tuk) =

• 1,

tuk ≥ Tw 0, tuk < Tw . (5) as the weighting function and Jaccard index as similarity

• measurement. In order to make the task of tuning parameters

easier, we introduce lr: Tw = lr · (Tmax − Tmin) + Tmin, (6) where Tmax and Tmin are the maximum timestamp and minimum timestamp in the training set, respectively. In this way, we can change the value of Tw by varying lr. We select lr from {0.1,0.3,0.5,0.7,0.9} in our experiments.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 15 / 29

Experiments

Baselines (4/7)

JI-EXP: time-aware item-based CF using Jaccard index as similarity measurement and exponential function as decay function [Ding and Li, 2005]: we(t) = e− Tmax −t+1

p·Tmax ,

(7) where the parameter p is chosen from {0.001,0.01,0.1}.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 16 / 29

Experiments

Baselines (5/7)

IF-uWIN: time-aware item-based CF using user’s active session window and a similarity based on the influential neighbors [Sun et al., 2013]: s(Tgap)

j→i

=

• u∈U δ(j ∈ Iu) · δ(i ∈ Iu) · δ(0 ≤ tui − tuj ≤ Tgap)

|Uj ∪ Ui| . (8) We introduce parameter r: Tgap = r · (Tmax − Tmin), (9) where r is chosen from {0.05,0.1,0.2,0.4}. In sequence-oriented directional item similarity, we do not consider the absolute timestamps of the records and use the item position of a specific item in user’s ordered action list instead.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 17 / 29

Experiments

Baselines (6/7)

BIS-uWIN (ρ = 0): time-aware item-based CF using user’s active session window and the proposed sequence-oriented directional item similarity. BIS-uWIN: time-aware item-based CF using user’s active session window and our proposed similarity BIS. Note that we fix the reverse factor ρ in Eq.(1) to 0.2. BIS-uWIN-EXP: time-aware item-based CF using a compound weighting function in Eq.(2) combining user’s active session window and exponential function in Eq.(7). The parameter p is chosen from {0.001,0.01,0.1}. As for similarity measurement, we use BIS. Note that we fix the reverse factor ρ in Eq.(1) to 0.2.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 18 / 29

Experiments

Baselines (7/7)

For the above methods in which BIS or sequence-oriented directional item similarity (BIS(ρ = 0)) is applied, we select the maximum gap ℓ between associated items from {5,10,20,40}. For methods that use user’s active session window, we select the latest interacted item number k from {5,10,20,40}. For all the above methods, we fix the number of nearest neighbors to 20, and search the best parameters by testing the above methods on validation set. Finally, we report their recommendation performances on the test set.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 19 / 29

Experiments

Evaluation Metrics

For performance evaluation, we adopt some commonly used ranking-oriented evaluation metrics: precision recall F1 NDCG 1-call The above metrics are evaluated for top-5 recommended items.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 20 / 29

Experiments

Results (1/2)

Table: Experimental results on MovieLens 10M.

Method Prec@5 Rec@5 F1@5 NDCG@5 1-call@5 best parameter JI 0.0107 0.0536 0.0179 0.0349 0.0536 CS 0.0112 0.0558 0.0186 0.0354 0.0558 JI-WIN 0.0102 0.0509 0.0170 0.0330 0.0509 lr = 0.1 JI-EXP 0.0116 0.0580 0.0193 0.0382 0.0580 p = 0.01 JI-uWIN 0.0155 0.0774 0.0258 0.0525 0.0774 k = 5 IF-uWIN 0.0154 0.0770 0.0257 0.0522 0.0770 k = 5, r = 0.2 BIS-uWIN (ρ = 0) 0.0168 0.0838 0.0279 0.0562 0.0838 k = 5, ℓ = 10 BIS-uWIN 0.0174 0.0870 0.0290 0.0592 0.0870 k = 5, ℓ = 10 BIS-uWIN-EXP 0.0177 0.0885 0.0295 0.0603 0.0885 k = 5, ℓ = 10, p = 0.01

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 21 / 29

Experiments

Results (2/2)

Table: Experimental results on Netflix.

Method Prec@5 Rec@5 F1@5 NDCG@5 1-call@5 best parameter JI 0.0153 0.0767 0.0256 0.0522 0.0767 CS 0.0151 0.0753 0.0251 0.0509 0.0753 JI-WIN 0.0158 0.0788 0.0263 0.0538 0.0788 lr = 0.7 JI-EXP 0.0215 0.1076 0.0359 0.0766 0.1076 p = 0.01 JI-uWIN 0.0197 0.0987 0.0329 0.0689 0.0987 k = 10 IF-uWIN 0.0198 0.0991 0.0330 0.0685 0.0991 k = 10, r = 0.1 BIS-uWIN (ρ = 0) 0.0204 0.1021 0.0340 0.0706 0.1021 k = 10, ℓ = 40 BIS-uWIN 0.0215 0.1077 0.0359 0.0738 0.1077 k = 10, ℓ = 40 BIS-uWIN-EXP 0.0236 0.1180 0.0393 0.0823 0.1180 k = 40, ℓ = 40, p = 0.01

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 22 / 29

Experiments

Observations

BIS-uWIN(ρ = 0) beats JI-uWIN and IF-uWIN on both datasets, revealing the fact that sequence-oriented directional item similarity can actually capture sequential patterns within the data. BIS-uWIN achieves better performance than BIS-uWIN(ρ = 0). Such phenomenon can be explained by the ability of BIS to tolerate noisy patterns. BIS-uWIN-EXP outperforms all other methods on both datasets, demonstrating the feasibility of the proposed compound weighting function and the effectiveness of our sequential item-based CF method.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 23 / 29

Experiments

Influence of Reverse Factor ρ (1/2)

We explore how the reverse factor ρ in BIS affect the recommendation performance. We vary the reverse factor ρ in the range of {0,0.2,0.4,0.6,0.8,1.0} for BIS-uWIN-EXP and BIS-uWIN. For both of these two methods, we fix the other parameters to be the same as have been shown in preceding experiments.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 24 / 29

Experiments

Influence of Reverse Factor ρ (2/2)

0.0 0.2 0.4 0.6 0.8 1.0 Reverse factor 0.081 0.082 0.083 0.084 0.085 0.086 0.087 0.088 1-call@5 BIS-uWIN-EXP BIS-uWIN 0.0 0.2 0.4 0.6 0.8 1.0 Reverse factor 0.1025 0.1050 0.1075 0.1100 0.1125 0.1150 0.1175 1-call@5 BIS-uWIN-EXP BIS-uWIN

MovieLens 10M Netflix

Figure: Recommendation performance of BIS-uWIN-EXP and BIS-uWIN with different values of ρ on MovieLens 10M and Netflix.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 25 / 29

Conclusions

Conclusions

In this paper, we study an important problem that many businesses are concerned about, i.e., next-item recommendation. Technically, we establish our work on the framework of time-aware item-based collaborative filtering. We devise a novel sequence-oriented bidirectional item similarity (BIS). We propose a compound weighting function that exploits the complementarity between user’s active session window [Mobasher et al., 2002] and exponential weighting function [Ding and Li, 2005]. We develop a novel item-based collaborative filtering method with the proposed similarity and compound weighting function.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 26 / 29

Future Works

Future Works

For future works, we are interested in applying the main idea of bidirectionality to user-based collaborative filtering methods and even model-based recommendation approaches in order to provide more accurate recommendation services.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 27 / 29

Acknowledgement

Thank you!

We thank the support of National Natural Science Foundation of China No. 61502307, No. 61672358 and No. U1636202, and Natural Science Foundation of Guangdong Province No. 2016A030313038.

Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 28 / 29

References Chen, X., Pan, W., and Ming, Z. (2016). TOCCF: Time-aware one-class collaborative filteringtime-aware one-class collaborative filtering. In Proceedings of Workshop on Multi-Dimensional Information Fusion for User Modeling and Personalization. Ding, Y. and Li, X. (2005). Time weight collaborative filtering. In Proceedings of the 14th ACM International Conference on Information and Knowledge Management, pages 485–492. He, R. and Mcauley, J. (2016). Fusing similarity models with markov chains for sparse sequential recommendation. ArXiv:1609.09152. Kabbur, S., Ning, X., and Karypis, G. (2013). Fism: Factored item similarity models for top-n recommender systems. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13, pages 659–667. Mobasher, B., Dai, H., Luo, T., and Nakagawa, M. (2002). Using sequential and non-sequential patterns in predictive web usage mining tasks. In Proceedings IEEE International Conference on Data Mining, pages 669–672. Steffen Rendle, C. F. and Schmidt-Thieme, L. (2010). Factorizing personalized markov chains for next-basket recommendation. In Proceedings of International Conference on World Wide Web, pages 811–820. Sun, G., Wu, L., Liu, Q., Zhu, C., and Chen, E. (2013). Recommendations based on collaborative filtering by exploiting sequential behaviors (in chinese). Journal of Software, (11):2721–2733. Zeng, Pan and Ming (SZU) BIS SCF ICWS 2018 29 / 29