SLIDE 1
Beyond Classification: La Latent User Interests Profiling from - - PowerPoint PPT Presentation
Beyond Classification: La Latent User Interests Profiling from - - PowerPoint PPT Presentation
Beyond Classification: La Latent User Interests Profiling from Visual Contents Analysis Lon Longqi Ya Yang , Cheng-Kang (Andy) Hsieh, Deborah Estrin Communicati Co tions On Online P Purchases So Social Network Ou Our inte terests ts
SLIDE 2
SLIDE 3
Ou Our inte terests ts are manifeste ted online …
Po Posted/Shared Contents Pe People Connected/Followed It Items s Purch chase sed
SLIDE 4
Pr Preferences learning using small data
On Online Posts Pr Private Co Communication Sh Shared Images Pe Personal Im Image Ga Gallery Pr Preference Pr Profile Ne News Se Search Engine Di Dietary En Entertainment
SLIDE 5
Te Text/label-ce centric c ap approac ach is widely studied
River restaurant tourism landscape Topic Modeling Structure Prediction Classification/Labeling/Image-to-text Tr Travel An Animal Ar Art
SLIDE 6
Bu But preferences are sometimes not just about text...
In Intra-ca categorica cal variance ce: Hard to to captu ture with th te text/ t/label! User A User B Tr Travel Im Images
SLIDE 7
Res esea earch ques estion
Im Images’ predictive power for users’ preferences be beyond d labe bels
Task 1: Pairwise Comparison Task 2: Prediction
SLIDE 8
Pa Pairwise Comparison
Us User A Us User B Di Discr scriminative Po Power of images
IMG1 IMG2 IMGn IMG1 IMG2 IMGm …... …...
Sa Same La Label
SLIDE 9
Pr Prediction: Consistency of Pr Preferences
Us User 1 Us User N Ti Timeline Pr Predict/Retrieve
IMG1 IMG2 IMGn IMG1 IMG2 IMGm …... …... …... …...
Sa Same La Label
SLIDE 10
Da Datase set
Tr Travel bo boards ds Ba Background corpus An Anal alysis
1, 1,800 800 3, 3,990 990
5, 5,790 790 Tr Travel bo boards ds
❶ ≥ 100 pins ❷ ∃ pins after June 2014
SLIDE 11
Us User Modeling and Image Representation
Pretrained Siamese Network Pretrained Places CNN
205 dim 410 dim 205 dim
Pretrained cluster centers (200) 200 dim
…
User Profile
…
pins
- B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. “Learning Deep Features for Scene Recognition using Places Database.”
Advances in Neural Information Processing Systems 27 (NIPS), 2014
* *
SLIDE 12
Us User Modeling and Image Representation
Image 2 Image 1 x y
A (CNN) B (CNN)
Contrastive Loss f(x) f(y)
− ≈ 0 − > 𝑛 , , 𝓜 = 𝟐 𝟑𝒎𝑬𝟑 + 𝟐 𝟑 𝟐 − 𝒎 𝐧𝐛𝐲 (𝟏, 𝒏 − 𝑬) 𝟑 𝒎 = 𝟐 𝒎 = 𝟏
SLIDE 13
Pa Pairwise Comparison
Us User A Us User B Ef Effects of ba backgr ground d di distribu bution!
IMG1 IMG2 IMGn …...
Tr Travel Im Images
IMG1 IMG2 IMGm …...
SLIDE 14
Pa Pairwise Comparison
Do Document 1 Do Document 2 “a “and” ” 10% “a “and” ” 11% “f “fatuous” ” 0.001% “f “fatuous” ” 1.001% 1% 1% Ba Background “a “and” ” 11% “f “fatuous” ” 0.001%
SLIDE 15
Us User A Us User B
Pa Pairwise Comparison
Ba Background co corpus
𝜀9
:;<
𝜏>(𝜀9
:;<)
Log Log-od
- dds-ra
ratio Un Uncertainty
𝑨9
:;< =
𝜀9
:;<
𝜏>(𝜀9
:;<)
SLIDE 16
Pa Pairwise Comparison
Confidence)Level:)95% Confidence)Level:)99%
𝐧𝐛𝐲 𝒜𝒍
𝑩;𝑪
Fo For all user pairs among 3,990 boards
SLIDE 17
Pr Prediction
Us User 1 Us User N Ti Timeline Sa Sampled 100 pins 50 50 pins for test 10~ 10~50 50 pins for train
IMG1 IMG2 …... …...
IMG51 IMG100
IMG1 IMG2 …... …...
IMG51 IMG100
SLIDE 18
Pr Prediction
𝑵𝑺𝑺 = 𝟐 𝑶G 𝟐 𝒔𝒃𝒐𝒍𝒋
𝑶 𝒋L𝟐
SLIDE 19
Co Conclusion
On Online Posts Pr Private Co Communication Sh Shared Images Pe Personal Im Image Ga Gallery Pr Preference Pr Profile
Sm Small data fueled preferences learning – wh what can we we do next? v Utilities of images beyond text/labels. v Multi-modal data fusion v End-to-end learning
SLIDE 20