Decision Making and Recommender Systems Marko Tkal i Johannes - PowerPoint PPT Presentation

How it fits into decision making?  Personality is related to decision making styles:  Myers-Briggs T/F: objective principles and impersonal facts (Thinking) or do you put more weight on personal concerns and the people involved (Feeling)?  Coping with Decisional Conflict (vigilance, buck-passing, procrastination and hypervigilance) (Deniz, 2011)  Career decision-making (Pecjak, 2007) Pe č jak , S., & Košir, K. (2007). Personality, motivational factors and difficulties in career decision -making in secondary school students. Psihologijske Teme , 16 , 141 – 158. Deniz, M. (2011). An Investigation of Decision Making Styles and the Five-Factor Personality Traits with Respect to Attachment Styles. Educational Sciences: Theory and Practice , 11 (1), 105 – 114. Mann, L., Burnett, P., Radford, M., & Ford, S. (1997). The Melbourne Decision Making Questionnaire: An instrument for measuring patterns for coping with decisional conflict. Journal of Behavioral Decision Making , 10 (1), 1 – 19. McCrae, R. R., & Costa, P. T. (1989). Reinterpreting the Myers-Briggs Type Indicator from the perspective of the five- factor model of personality. Journal of Personality , 57 (1), 17 – 40. doi:10.1111/1467-6494.ep8972588

Personality in Recommender Systems  New user problem  Diversity  Preferences (genres)  Group recommenders  Browsing Styles  Mood regulation

A personality-based user similarity measure Collaborative filtering recommender (CFR) systems:  Similar users have similar preferences  Rating-based similarity measures -> Personality based similarity measure  Which content should I watch tonight? Tkalcic, M., Kunaver , M., Košir, A., & Tasic, J. (2011). Addressing the new user problem with a personality based user similarity measure. DEMRA 2011, UMMS 2011 Elahi, M., Braunhofer, M., Ricci, F., & Tkalcic, M. (2013). Personality-based active learning for collaborative filtering recommender systems. AI*IA 2013: Advances in Artificial Intelligence , 360 – 371. doi:10.1007/978-3- 319-03524-6_31

Using personality to diversify recommendations Are you satisfied FIT DIVERSE DIVERSE FIT FIT with the DIVERSE recommendations? Wu, W., Chen, L., & He, L. (2013). Using personality to adjust diversity in recommender systems. Proceedings of the 24th ACM Conference on Hypertext and Social Media - HT ’13 , (May), 225 – 229. doi:10.1145/2481492.2481521

Using personality to diversify recommendations high Are you satisfied DIVER DIVER DIVER DIVER FIT FIT with the SE SE SE SE recommendations? Openness to Are you satisfied new DIVER DIVER DIVER FIT FIT FIT with the SE SE SE experiences recommendations? Are you satisfied DIVER DIVER with the FIT FIT FIT FIT SE SE recommendations? low Wu, W., Chen, L., & He, L. (2013). Using personality to adjust diversity in recommender systems. Proceedings of the 24th ACM Conference on Hypertext and Social Media - HT ’13 , (May), 225 – 229. doi:10.1145/2481492.2481521

Personality is correlated with Music Preferences Intense/rebellious  Openness  Upbeat/Conventional  Extraversion, Agreeableness,Conscientiousness  Energetic/Rhytmic  Extraversion, Agreeableness  Rentfrow, P. J., & Gosling, S. D. (2003). The do re mi’s of everyday life: The structure and personality correlates of music preferences. Journal of Personality and Social Psychology , 84 (6), 1236 – 1256. doi:10.1037/0022- 3514.84.6.1236

Personality and group RS  (Garcia et al, 2009)  Group personality composition  TKI Conflict personality modes (assertiveness, cooperativeness)  MovieLens+70 students  „ group goes to the cinema “  (Kompan et al, 2014)  TKI, NEO-FFI  Small scale (9 users) groups of 3  Movie recommendations Recio-Garcia, J. A., Jimenez-Diaz, G., Sanchez-Ruiz, A. A., & Diaz-Agudo, B. (2009). Personality aware recommendations to groups. In Proceedings of the third ACM conference on Recommender systems - RecSys ’09 (p. 325). New York, New York, USA: ACM Press. doi:10.1145/1639714.1639779 Quijano-Sanchez, L., Recio-Garcia, J. a., & Diaz-Agudo, B. (2010). Personality and Social Trust in Group Recommendations. 2010 22nd IEEE International Conference on Tools with Artificial Intelligence , (c), 121 – 126. doi:10.1109/ICTAI.2010.92 Kompan, M., & Bieliková, M. (2014). Social Structure and Personality Enhanced Group Recommendation. EMPIRE 2014: Emotions and Personality in Personalized Services .

Mood regulation  Uses and gratification theory: music is used for mood regulation  high on openness, extraversion, and agreeableness more inclined to listen to happy music when they are feeling sad .  high on neuroticism listen to more sad songs when feeling disgusted (neurotic people choose to increase their level of worry) Ferwerda, Schedl, Tkal č i č (2014), Personality & Emotional States: Understanding User’s Music Listening Needs to Enhance Recommender Systems, submitted to CHI 2015

Music Browsing Style Ferwerda, Yang, Schedl, Tkal č i č (2014), Personality Traits Predict Music Category Preferences, submitted to CHI 2015

31 Outline  Background  The role of Personality in Decision Making and Recommender Systems  Models  Acquisition  Personality and Decision Making  Personality in RecSys – examples  The role of Emotions in Decision Making and Recommender Systems  Models  Acquisition  Emotions and Decision Making  Emotions in RecSys – examples  Focus: Emotions as Implicit Feedback for evaluation purposes  Conclusions and take away notes

Emotions in Decision Making and Recommender Systems

Overview of emotions  Emotions are complex human experiences  Evolutionary based  Several definitions  We take simple models, easy to incorporate in computers:  Basic emotions  Dimensional model  Circumplex model

Basic emotions  Discrete classes model  Different sets  Darwin: Expression of emotions in man and animal  Ekman definition (6 + neutral):  Happiness  Anger  Fear  Sadness  Disgust  Surprise

Dimensional model  Three dimensions  Valence  Arousal  Dominance  Each emotive state is a point in the VAD space

Circumplex model  Maps basic emotions  dimensional model Arousal high joy anger surpri se disgu st fear Valence neutr al negative positive sadne ss low

How to detect emotions?  Explicit vs. Implicit  Explicit  Questionnaires (SAM)  Implicit:  Work done in the affective computing community  Different modalities (sources): – Facial actions (video) – Physiological signals ( GSR, EEG) – Voice – Posture – ...  ML techniques – Classification (basic emotions) – Regression (dimensional model)

Emotion detection from videos of facial expressions  Problem statement:  Explicit affective labeling has drawbacks: – Annoying – Time consuming – Potentially inaccurate in real applications  Proposed solution:  Implicit affective labeling through emotion detection from facial video  Aggregation of emotions detected from several users

Experiment  2 datasets: Posed (Kanade Cohn)  Spontaneous (LDOS-PerAff-1)   Input: Video streams of facial expressions as responses to visual stimuli  Output: emotive states as distinct classes Gabor features kNN Emotive state

Results and conclusions  Posed dataset: accuracy = 92 %  Spontaneous dataset: accuracy = 62%  Reasons for bad results:  Weak learning supervision  Non optimal video acquisition (face rotation, occlusions, changing lightning ...)  Non extreme facial expressions Tkal č i č , M., Odi ć , A., & Košir, A. (2013). The impact of weak ground truth and facial expressiveness on affect detection accuracy from time-continuous videos of facial expressions. Information Sciences , 249 , 13 – 23. doi:10.1016/j.ins.2013.06.006 Tkalcic, M., Odic, A., Kosir, A., & Tasic, J. (2013). Affective Labeling in a Content-Based Recommender System for Images. IEEE Transactions on Multimedia , 15 (2), 391 – 400. doi:10.1109/TMM.2012.2229970

Emotion detection in social media  Microblogs: 1 tweet ->{happy/unhappy, active/inactive} Features:  Unigrams  Emoticons  Punctuation features  Negation features  Hasan, M., Rundensteiner, E., & Agu, E. (n.d.). EMOTEX : Detecting Emotions in Twitter Messages., SocialCom2014

The problem researchers face

43 DM = rational + emotional  The chooser came under the influence of emotions  Affective mechanisms that work on an unconscious automatic level  Immediate emotions OR  She intentionally consults her feelings about options, and uses that information to guide the decision process  Information value of emotions

44 The affect heuristic  A mental shortcut  Quick decision based on current emotions and feelings  No need to complete an extensive search for information  Equivalent of "going with your gut instinct" Slovic, P., Finucane, M., Peters, E., & MacGregor, D. G. (2002). The affect heuristic. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment. Cambridge: Cambridge University, Press.

45 Some affect heuristics  Risks and Benefits of options evaluated depending on the positive or negative feelings associated with a stimulus  What about buying a fast luxury car?

46 Fear Appeals Did your perception of the risks of buying a fast car change?

47 Somatic marker hypothesis  Cognitive overload of complex and conflicting choices  somatic markers and their evoked emotions can help decide  Physiological signals are consciously or unconsciously associated with their past outcomes  Somatic markers associated with positive / negative outcomes  tendency to choose / avoid options  Experience-based Damasio, A.R. (1996) "The Somatic Marker Hypothesis and the Possible Functions of the Prefrontal Cortex". Philosophical Transactions 351 (1346): 1413 – 1420, Damasio, A.R. (1994) Descartes' Error: Emotion, Reason, and the Human Brain, Putnam, 1994.

48 Emotions as Contexts in Context-aware Recommender Systems (I)  Emotions useful as contextual conditions for context- aware-splitting [Zheng2013]  Emotion-linked context makes an important contribution when added to other dimensions (e.g. Time)  Emotions incorporated into contextual modeling process to assist user-based CF [Zheng2013]  Allows to define the role of emotions with respect to the specific components of the recommendation process  Mood and dominant emotion are influencial for computing user similarity and selecting neighbors  Emotional tags associated with points of interest for location-based music recommendation [Kaminskas 2011] [Kaminskas2011] Marius Kaminskas and Francesco Ricci. 2011. Location-adapted music recommendation using tags. Proceedings of UMAP '11, Springer-Verlag, Berlin, Heidelberg, 183-194. [Zheng2013] Yong Zheng, Bamshad Mobasher, Robin D. Burke: The Role of Emotions in Context-aware Recommendation. Decisions@RecSys 2013: 21-28

Example of affective user modeling  We propose to use AFFECTIVE METADATA  Multimedia content ELICITS (induces) emotions  Underlying assumption: users differ in their preferences for emotions

Example of affective user modeling IAPS Image Stimuli generic metadata affective metadata Metadata Consumed (Item Item EMOTION Profile) INDUCTION User Machine Explicit Profile Learning Rating Ground Predicte Truth d Ratings Ratings Confusi on Matrix Tkal č i č , M., Burnik, U., & Košir, A. (2010). Using affective parameters in a content-based recommender system for images. User Modeling and User-Adapted Interaction , 20 (4), 279 – 311. doi:10.1007/s11257-010- 9079-z

Social Signal – Hesitation as Affective Feedback Vodlan, T., Tkal č i č , M., & Košir, A. (2014). The impact of hesitation, a social signal, on a user’s quality of experience in multimedia content retrieval. Multimedia Tools and Applications . doi:10.1007/s11042-014- 1933-2

ePoznan.pl- polish startup recsys company

Datasets  LDOS PerAff-1(images, small scale, emotions, personality)  LDOS CoMoDa (movies, emotions, personality)  myPersonality (facebook activity, personality) Tkal č i č , M., Košir, A., & Tasi č , J. (2013). The LDOS-PerAff-1 corpus of facial-expression video clips with affective, personality and user-interaction metadata. Journal on Multimodal User Interfaces , 7 (1-2), 143 – 155. doi:10.1007/s12193-012-0107-7 Košir , A., Odi ć , A., Kunaver, M., Tkal č i č , M., & Tasi č , J. F. (2011). Database for contextual personalization. Elektrotehniški Vestnik , 78 (5), 270 – 274. Bachrach, Y., Kosinski, M., Graepel, T., Kohli, P., & Stillwell, D. (2012). Personality and patterns of Facebook usage. In Proceedings of the 3rd Annual ACM Web Science Conference on - WebSci ’12 (pp. 24 – 32). New York, New York, USA: ACM Press. doi:10.1145/2380718.2380722

54 Outline  Background  The role of Personality in Decision Making and Recommender Systems  Models  Acquisition  Personality and Decision Making  Personality in RecSys – examples  The role of Emotions in Decision Making and Recommender Systems  Models  Acquisition  Emotions and Decision Making  Emotions in RecSys – examples  Focus: Emotions as Implicit Feedback for evaluation purposes  Conclusions and take away notes

Focus: Emotions as implicit feedback for assessing serendipity of recommendations

56 Collaborators/Contributors Marco de Gemmis Pasquale Lops S emantic W eb A ccess and P ersonalization research group http://www.di.uniba.it/~swap

Outline  Serendipity and Evaluation  Research questions  Operationally induced serendipity:  Knowledge Infusion (KI) process  Item-to-Item correlation matrix  Random Walk with Restart boosted by KI  Experimental evaluation  Noldus FaceReader ™  Dataset  Design of the experiment  Metrics  Questionnaire analysis  Analysis of user emotions  Conclusions

58 Serendipity

59 Serendipity in Information Seeking  Information seeking metaphor investigated in literature (Toms 2000)  Toms suggests 4 strategies  Blind luck or “role of chance”  random  Pasteur Principle or “chance favors only the prepared mind”  flashes of insight don’t just happen, but they are the products of a “prepared mind”  Anomalies and exceptions or “searching for dissimilarities”  identification of items dissimilar to those the user liked in the past  Reasoning by analogy  abstraction mechanism allowing the system to discover the applicability of an existing schema to a new situation (Toms 2000) E. Toms. Serendipitous Information Retrieval. Proceedings of the First DELOS Network of Excellence Workshop on Information Seeking, Searching and Querying in Digital Libraries , Zurich, Switzerland: European Research Consortium for Informatics and Mathematics, 2000.

60 Serendipitous recommendations  “ Suggestions which help the user to find surprisingly interesting items she might not have discovered by herself ” (Herlocker et al. 2004)  Both attractive and unexpected  “The experience of receiving an unexpected and fortuitous item recommendation” (McNee et al. 2006)  A response to the overspecialization problem and the filter bubble (Pariser 2011)  tendency to provide the user with items within her existing range of interests  suggesting “STAR TREK” to a science-fiction fan: Accurate but obvious , thus actually not useful  users don’t want algorithms that produce better ratings, but sensible recommendations (Herlocker et al. 2004) Herlocker, L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating Collaborative Filtering Recommender Systems. ACM Transactions on Information Systems 22(1): 5 – 53, 2004. (McNee et al. 2006) S. M. McNee, J. Riedl, and J. A. Konstan. Being accurate is not enough: How accuracy metrics have hurt recommender systems. In CHI ’ 06 Extended Abstracts on Human Factors in Computing Systems , CHI EA ’ 06, 1097 – 1101, ACM, New York, NY, USA, 2006. (Pariser 2011) E. Pariser. The Filter Bubble: What the Internet Is Hiding from You . Penguin Group, May 2011.

61 Evaluation of Serendipity: research questions  Is the emotional response of the user useful for assessing serendipity?  Can the emotions observed in facial expressions be considered as a trustworthy implicit feedback for assessing the pleasant surprise that serendipity should convey?

62 Operationally induced Serendipity: A Quick Look at the Recommendation Algorithm  Novel method for computing item similarity  tries to find “hidden associations” instead of computing attribute similarity  knowledge intensive process that allows deeper understanding of item descriptions  Knowledge Infusion (KI)  provides the RecSys with a background knowledge made from external sources  Content-based approach that exploits the knowledge base to compute a correlation index between items

63 Operationally induced Serendipity: Knowledge Infusion (KI) “Words” Recommender System  Which “words”?  Words that induce positive emotions  Relevant/attractive words able to surprise the conversation partner “Language is the Skin of my Thought” Arundhati Roy. Power Politics . South End Press, January 2001.

Recommending Words: the Architecture of the KI process sci-fi conflicts/ fights

65 KI@Work CLUE#1 CLUE#2 CLUE#3 CLUE#4 CLUE#5 BACKGROUND KNOWLEDGE . . . Knowledge Knowledge Knowledge Knowledge Source #n Source #2 Source #3 Source #1 KEYWORD 1 SPREADING KEYWORD 2 NEW KEYWORDS ACTIVATION ASSOCIATED … NETWORK WITH CLUES G. Semeraro, M. de Gemmis, P. Lops, P. Basile. An Artificial Player for a Language Game. IEEE Intelligent Systems 27(5): 36-43, 2012. P. Basile, M. de Gemmis, P. Lops, G. Semeraro. Solving a Complex Language Game by using Knowledge-based Word Associations Discovery. IEEE Transactions on Computational Intelligence and AI in Games , 2014 (in press).

66 KI as a novel method for computing associations between items clues BM25 retrieval score

67 KI as a Serendipity Engine: Item-to-Item similarity matrix  Item-to-Item correlation matrix w ij computed in different ways  #users co-rated I i and I j  cosine similarity between item descriptions Knowledge Infusion  Correlation index  w ij Recommendation list computed by Random Walk with Restart (Lovasz 1996) augmented with KI (RWR-KI) (Lovasz 1996) L. Lovasz. Random Walks on Graphs: a Survey. Combinatronics 2:1 – 46, 1996.

68 Evaluation of Serendipity: research questions  Is the emotional response of the user useful for assessing serendipity?  Can the emotions observed in facial expressions be considered as a trustworthy implicit feedback for assessing the pleasant surprise that serendipity should convey?

69 Experimental Evaluation: Goal  Validation of the hypothesis that recommendations produced by RWR-KI are serendipitous ( relevant/attractive & unexpected/surprising )  Not only an issue of metrics!  Difficulty of detecting and providing an objective assessment of the emotional response conveyed by serendipitous recommendations  Difficulty of assessing the user perception of serendipity of recommendations and their acceptance (in terms of relevance and unexpectedness )  Difficulty of assessing unexpectedness M. de Gemmis, P. Lops, G. Semeraro. An Investigation on the Serendipity Problem in Recommender Systems. submitted manuscript, 2014.

70 Experimental Evaluation  2 experiments  In-vitro  User study  In-vitro experiment  Unexpectedness measured as deviation from a standard prediction criterion (Murakami et al. 2008)  Standard prediction criterion: (non-personalized) popularity  User study  Analysis performed using Noldus FaceReader ™  Allows to analyze users’ facial expressions and gather implicit feedback about their reactions (Murakami et al. 2008) T. Murakami, K. Mori, R. Orihara, Metrics for Evaluating the Serendipity of Recommendation Lists, in K. Satoh, A. Inokuchi, K. Nagao, T. Kawamura (Eds.), New Frontiers in Artificial Intelligence, Lecture Notes in Computer Science 4914, pp. 40 – 46, Springer, 2008.

71 Noldus FaceReader ™  Recognize 6 categories of emotions, proposed by Ekman (1999)  fear  happiness  disgust  anger  surprise  sadness  Classification accuracy  ~ 90% on Radboud Faces Database (RaFD) (Langner et al. 2010) (Ekman 1999) P. Ekman, Basic Emotions, in T. Dalgleish, M.J. Power (Eds.), Handbook of Cognition and Emotion , 45 – 60, John Wiley & Sons, 1999. (Langner et al. 2010) O. Langner, R. Doetsch, G. Bijlstra, D.H.J. Wigboldus, S.T. Hawk, A. van Knippenberg. Presentation and Validation of the Radboud Faces Database, Cognition and Emotion 24(8), 1377-1388, 2010.

72 Experimental Evaluation: Noldus FaceReader ™

73 Experimental Evaluation ( user study ): Dataset  Experimental units: 40 master students (engineering, architecture, economy, computer science and humanities)  26 male (65%), 14 female (35%)  Age distribution: from 20 to 35  Dataset  2, 135 movies released between 2006 and 2011  Movie content – title, poster, plot keywords, cast, director, summary – crawled from the Internet Movie Database (IMDb)  Vocabulary of 32, 583 plot keywords  Average: 12.33 keywords/item

74 Experimental Evaluation ( user study ): Design of the experiment  Between-subjects controlled experiment  20 users randomly assigned to test RWR-KI  20 users randomly assigned to test RANDOM (control group), a baseline inspired by the blind luck principle which produces random suggestions that showed surprisingly good performance in the 1 st In-vitro experiment  Procedure  Users interact with a web application – shows details of movies – displays 5 recommendations (movie poster & title) per user  Recommended items displayed 1 at a time

75 Web application

76 Experimental Evaluation ( user study ): Design of the experiment  Procedure  2 binary questions to assess user acceptance – “Did you know this movie?” “Have you ever heard about this movie?” ( unexpectedness ) – “Do you like this movie?” ( relevance ) – (NO,YES) answers  serendipitous recommendation  Video started when a movie is recommended to the user and stopped when the answers to the 2 questions are collected  5 videos per user  Noldus FaceReader ™ used to analyze videos and assess user emotional response when exposed to recommendations

77 Experimental Evaluation ( user study ): Design of the experiment  Questionnaire analysis  Quality of RWR-KI and RANDOM  Metrics Relevance @ N = #relevant_items /N Unexpectedness @ N = #unexpected_items/ N Serendipity @ N = #serendipitous_items/ N = #(relevant_items  unexpected_items)/ N N = size of the recommendation list

78 Experimental Evaluation ( user study ): Design of the experiment  Questionnaire analysis  ResQue model (Chen et al. 2010) – category: Perceived System Qualities – sub-category: Quality of Recommended Items – Relevance = perceived accuracy – Unexpectedness = novelty (Chen et al. 2010) L. Chen, P. Pu, A User-Centric Evaluation Framework of Recommender Systems, in: B.P. Knijnenburg, L. Schmidt- Thieme, D. Bollen (Eds.), Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), CEUR Workshop Proceedings 612 , 14-21, CEUR-WS.org, 2010.

79 Experimental Evaluation ( user study ): Results  Questionnaire analysis  Serendipity: RWR-KI outperforms RANDOM  Statistically significant differences (Mann-Whitney U test, p <0.05) ~ Half of the recommendations are deemed  serendipitous!  RWR-KI: a better Relevance-Unexpectedness trade-off  RANDOM: more unbalanced towards Unexpectedness

80 Experimental Evaluation ( user study ): Results  Questionnaire analysis: distribution of serendipitous items within Top-5 lists  Almost all users (19 out of 20) received  1 serendipitous suggestions  Most of RWR-KI lists: 2-3 serendipitous items  Most of RANDOM lists: 1-2 serendipitous items

81 Experimental Evaluation ( user study ): Results  Analysis of user emotions  Hypothesis: users’ facial expressions convey a mixture of emotions that helps to measure the perception of serendipity of recommendations  Serendipity associated to surprise and happiness  ResQue model: attractiveness  200 videos (40 users x 5 recommendations)  41 videos filtered out (< 5 seconds)   159 videos, FaceReader™ computed the distribution of detected emotions + duration (emotions lasting <1 sec.)

Circumplex model  Maps basic emotions  dimensional model Arousal high joy anger surpri se disgu st fear Valence neutr al negative positive sadne ss low

83 Experimental Evaluation ( user study ): Results  Frequency analysis of user emotions associated to serendipitous suggestions (69 videos=81 – 12) 39 videos 30 videos  Surprise: 17% RWR-KI vs 9% RANDOM  Happiness: 14% RWR-KI vs 9% RANDOM  RWR-KI produces more serendipitous suggestions than RANDOM! (confirm questionnaires results)  High values of negative emotions ( sadness and anger ); why?

84 Experimental Evaluation ( user study ): Results  Frequency analysis of user emotions associated to non-serendipitous suggestions (90 videos=119 – 29) 39 videos 51 videos  General decrease of surprise and happiness  High values of negative emotions ( sadness and anger ), also in this case  Explanation: Negative emotions due to the fact that users assumed troubled expressions since they were very concentrated on the task

85 Experimental Evaluation ( user study ): Conclusions  Positive emotions: marked difference between RWR-KI and RANDOM  Positive emotions: marked difference between serendipitous recommendations and non-serendipitous ones  Agreement between questionnaires ( explicit feedback ) & facial expressions/emotions ( implicit feedback )  Emotions can help to assess the actual perception of serendipity  A step forward to the creation of a ground truth for evaluation purposes

86 Outline  Background  The role of Personality in Decision Making and Recommender Systems  Models  Acquisition  Personality and Decision Making  Personality in RecSys – examples  The role of Emotions in Decision Making and Recommender Systems  Models  Acquisition  Emotions and Decision Making  Emotions in RecSys – examples  Focus: Emotions as Implicit Feedback for evaluation purposes  Conclusions and take away notes

Publishing opportunities UMUAI Special Issue on Personality in Personalized Systems (Tkal č i č , Quercia,  Graf) : 1. December 2014 UMUAI Special Issue on Physiology in Personalized Systems (Tkal č i č , Fairclough,  Conati, Valjamae): mid 2015

Concluding Remarks, Take away notes & Challenges  Wrap-up: DM = Rational + Emotional  DM is related to personality  Various models (of emotions and personality)  Acquisition is not perfect  Variety of approaches in RS  Future:  Novel RS metric required (that take into account affect and personality)  Watch out for better acquisition methods:  – affective computing community, – personality computing, – social signal processing – New devices (e.g. google glass + Fraunhofer Shore) Explore the usage of personality and emotions:  – Context – Personalized feedback acquisition (Feedback interpretation) – Visualization (browsing styles, diversity …) – Group modeling – Preference modeling – Explore the „ reasons “ for consumption (uses and gratification theory) Privacy issues (e.g. Facebook experiment ) … not addressed here 

Thanks… Questions? “Data scientists will be the rock stars of the Big Data era” (www.greenplum.com) Gregor Geršak Janko Drnovšek Andrej Košir Markus Schedl Ante Odi ć Bruce Ferwerda Toma ž Vodlan http://cp.jku.at Pierpaolo Basile Annalina Caputo Marco de Gemmis Giuseppe Ricci S emantic Leo Iaquinta Pasquale Lops W eb A ccess and Piero Molino Cataldo Musto P ersonalization Fedelucio Narducci Giovanni Semeraro research group http://www.di.uniba.it/~swap

References (Basile et al. 2014) P. Basile, M. de Gemmis, P. Lops, G. Semeraro. Solving a Complex Language  Game by using Knowledge-based Word Associations Discovery. IEEE Transactions on Computational Intelligence and AI in Games , 2014 (in press). (Chen et al. 2010) L. Chen, P. Pu, A User-Centric Evaluation Framework of Recommender  Systems, in: B.P. Knijnenburg, L. Schmidt-Thieme, D. Bollen (Eds.), Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), CEUR Workshop Proceedings 612 , 14-21, CEUR-WS.org, 2010. (de Gemmis et al. 2014) M. de Gemmis, P. Lops, G. Semeraro. An Investigation on the  Serendipity Problem in Recommender Systems. Submitted manuscript, 2014. (Ekman 1999) P. Ekman, Basic Emotions, in T. Dalgleish, M.J. Power (Eds.), Handbook of  Cognition and Emotion , 45 – 60, John Wiley & Sons, 1999. (Herlocker et al. 2004) Herlocker, L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating  Collaborative Filtering Recommender Systems. ACM Transactions on Information Systems 22(1): 5 – 53, 2004.  (Kramer et al. 2014) Kramer, Adam D. I.; Guillory, Jamie E.; Hancock, Jeffrey T. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences of the United States of America, vol. 11, issue 29, 8788-8790, 2014. (Langner et al. 2010) O. Langner, R. Doetsch, G. Bijlstra, D.H.J. Wigboldus, S.T. Hawk, A. van  Knippenberg. Presentation and Validation of the Radboud Faces Database, Cognition and Emotion 24(8), 1377-1388, 2010. (Lauw et al. 2010) Lauw, H.W., Schafer, J.C., Agrawal, R., & A. Ntoulas. Homophily in the Digital  World: A LiveJournal Case Study. IEEE Internet Computing 14(2):15-23, March-April 2010. (Lovasz 1996) L. Lovasz. Random Walks on Graphs: a Survey. Combinatronics 2:1 – 46, 1996.  (McNee et al. 2006) S. M. McNee, J. Riedl, and J. A. Konstan. Being accurate is not enough: How  accuracy metrics have hurt recommender systems. In CHI ’ 06 Extended Abstracts on Human Factors in Computing Systems , CHI EA ’ 06, pages 1097 – 1101, ACM, New York, NY, USA, 2006.

References (Murakami et al. 2008) T. Murakami, K. Mori, R. Orihara, Metrics for Evaluating the Serendipity  of Recommendation Lists, in K. Satoh, A. Inokuchi, K. Nagao, T. Kawamura (Eds.), New Frontiers in Artificial Intelligence, Lecture Notes in Computer Science 4914, pp. 40 – 46, Springer, 2008. (Pariser 2011) E. Pariser. The Filter Bubble: What the Internet Is Hiding from You . Penguin  Group, May 2011. (Roy 2001) Arundhati Roy. Power Politics . South End Press, January 2001.  (Semeraro et al. 2012) G. Semeraro, M. de Gemmis, P. Lops, P. Basile. An Artificial Player for a  Language Game. IEEE Intelligent Systems 27(5): 36-43, 2012. (Toms 2000) E. Toms. Serendipitous Information Retrieval. Proceedings of the First DELOS  Network of Excellence Workshop on Information Seeking, Searching and Querying in Digital Libraries , Zurich, Switzerland: European Research Consortium for Informatics and Mathematics, 2000. (Zuckerman 2008) E. Zuckerman. Homophily, serendipity, xenophilia . April 25, 2008.  www.ethanzuckerman.com/blog/2008/04/25/homophily-serendipity-xenophilia /

References Cain, S. (2013). Quiet: The Power of Introverts in a World That Can’t Stop Talking. Broadway Books.  (McCraMcCrae, R. R., & John, O. P. (1992). An Introduction to the Five-Factor Model and its Applications.  Journal of Personality, 60(2), p175 – 215. J.L Holland: Making vocational choices, Prentice-Hall, Englewood Cliffs (1973)  http://www.gamasutra.com/view/feature/134842/personality_and_play_styles_a_.php  El-Bishouty, M. M., Chang, T.-W., Graf, S., & Chen, N.-S. (2014). Smart e-course recommender based on  learning styles. Journal of Computers in Education, 1(1), 99 – 111. doi:10.1007/s40692-014-0003-0 Furnham, A. (1996). The big five versus the big four: the relationship between the Myers-Briggs Type Indicator  (MBTI) and NEO-PI five factor model of personality. Personality and Individual Differences, 21(2), 303 – 307. doi:10.1016/0191-8869(96)00033-5 http://homepage.psy.utexas.edu/HomePage/Faculty/Gosling/tipi%20site/tipi.htm  Gosling, S. D., Rentfrow, P. J., & Swann, W. B. (2003). A very brief measure of the Big-Five personality domains.  Journal of Research in Personality, 37(6), 504 – 528. doi:10.1016/S0092-6566(03)00046-1  Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 2 – 5. doi:10.1073/pnas.1218772110 Golbeck, J., Robles, C., & Turner, K. (2011). Predicting personality with social media. Proceedings of the 2011  Annual Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’ 11, 253. doi:10.1145/1979742.1979614 Quercia, D., Kosinski, M., Stillwell, D., & Crowcroft, J. (2011). Our Twitter Profiles, Our Selves: Predicting  Personality with Twitter. In 2011 IEEE Third Int’l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing (pp. 180 – 185). IEEE. doi:10.1109/PASSAT/SocialCom.2011.26 Shen, J., Brdiczka, O., & Liu, J. (2013). Understanding Email Writers: Personality Prediction from Email  Messages. User Modeling, Adaptation, and Personalization, 318 – 330. doi:10.1007/978-3-642-38844-6_29  Nowson, S., & Oberlander, J. (2007). Identifying more bloggers: Towards large scale personality classification of personal weblogs. International Conference on Weblogs and Social Media. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.64.2021 Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital  records of human behavior. Proceedings of the National Academy of Sciences, 2 – 5. doi:10.1073/pnas.1218772110

References Pe č jak, S., & Košir, K. (2007). Personality, motivational factors and difficulties in career decision-making in  secondary school students. Psihologijske Teme, 16, 141 – 158. Deniz, M. (2011). An Investigation of Decision Making Styles and the Five-Factor Personality Traits with  Respect to Attachment Styles. Educational Sciences: Theory and Practice, 11(1), 105 – 114. Mann, L., Burnett, P., Radford, M., & Ford, S. (1997). The Melbourne Decision Making Questionnaire: An  instrument for measuring patterns for coping with decisional conflict. Journal of Behavioral Decision Making, 10(1), 1 – 19. McCrae, R. R., & Costa, P. T. (1989). Reinterpreting the Myers-Briggs Type Indicator from the perspective of  the five-factor model of personality. Journal of Personality, 57(1), 17 – 40. doi:10.1111/1467-6494.ep8972588 Tkalcic, M., Kunaver, M., Košir, A., & Tasic, J. (2011). Addressing the new user problem with a personality  based user similarity measure. DEMRA 2011, UMMS 2011 Elahi, M., Braunhofer, M., Ricci, F., & Tkalcic, M. (2013). Personality-based active learning for collaborative  filtering recommender systems. AI*IA 2013: Advances in Artificial Intelligence, 360 – 371. doi:10.1007/978-3- 319-03524-6_31 Wu, W., Chen, L., & He, L. (2013). Using personality to adjust diversity in recommender systems. Proceedings  of the 24th ACM Conference on Hypertext and Social Media - HT ’ 13, (May), 225 – 229. doi:10.1145/2481492.2481521 Rentfrow, P. J., & Gosling, S. D. (2003). The do re mi’s of everyday life: The structure and personality  correlates of music preferences. Journal of Personality and Social Psychology, 84(6), 1236 – 1256. doi:10.1037/0022-3514.84.6.1236 Recio-Garcia, J. A., Jimenez-Diaz, G., Sanchez-Ruiz, A. A., & Diaz-Agudo, B. (2009). Personality aware  recommendations to groups. In Proceedings of the third ACM conference on Recommender systems - RecSys ’ 09 (p. 325). New York, New York, USA: ACM Press. doi:10.1145/1639714.1639779 Quijano-Sanchez, L., Recio-Garcia, J. a., & Diaz-Agudo, B. (2010). Personality and Social Trust in Group  Recommendations. 2010 22nd IEEE International Conference on Tools with Artificial Intelligence, (c), 121 – 126. doi:10.1109/ICTAI.2010.92 Kompan, M., & Bieliková, M. (2014). Social Structure and Personality Enhanced Group Recommendation.  EMPIRE 2014: Emotions and Personality in Personalized Services. Ferwerda, Schedl, Tkal č i č (2014), Personality & Emotional States: Understanding User’s Music Listening Needs  to Enhance Recommender Systems, submitted to CHI 2015

References Ferwerda, Yang, Schedl, Tkal č i č (2014), Personality Traits Predict Music Category Preferences, submitted to  CHI 2015 Tkal č i č , M., Odi ć , A., & Košir, A. (2013). The impact of weak ground truth and facial expressiveness on affect  detection accuracy from time-continuous videos of facial expressions. Information Sciences, 249, 13 – 23. doi:10.1016/j.ins.2013.06.006 Tkalcic, M., Odic, A., Kosir, A., & Tasic, J. (2013). Affective Labeling in a Content-Based Recommender System  for Images. IEEE Transactions on Multimedia, 15(2), 391 – 400. doi:10.1109/TMM.2012.2229970  Hasan, M., Rundensteiner, E., & Agu, E. (n.d.). EMOTEX : Detecting Emotions in Twitter Messages., SocialCom2014 Loewenstein, G., & Lerner, J. S. (2003). The role of affect in decision making. In R. Davidson, K. Scherer & H.  Goldsmith (Eds.), Handbook of affective science (pp. 619-642). New York: Oxford University Press. Slovic, P., Finucane, M., Peters, E., & MacGregor, D. G. (2002). The affect heuristic. In T. Gilovich, D. Griffin, &  D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment. Cambridge: Cambridge University, Press. Damasio, A.R. (1996) "The Somatic Marker Hypothesis and the Possible Functions of the Prefrontal Cortex".  Philosophical Transactions 351 (1346): 1413 – 1420, Damasio, A.R. (1994) Descartes' Error: Emotion, Reason, and the Human Brain, Putnam, 1994.  Tkal č i č , M., Košir, A., Tasi č , J., & Kunaver, M. (2011). Affective recommender systems : the role of emotions in  recommender systems. Proceedings of the RecSys 2011 Workshop on Human Decision Making in Recommender Systems (Decisions@RecSys’ 11), 9 – 13. Retrieved from http://ceur-ws.org/Vol- 740/UMMS2011_paper6.pdf [Kaminskas2011] Marius Kaminskas and Francesco Ricci. 2011. Location-adapted music recommendation  using tags. Proceedings of UMAP '11, Springer-Verlag, Berlin, Heidelberg, 183-194. [Zheng2013] Yong Zheng, Bamshad Mobasher, Robin D. Burke: The Role of Emotions in Context-aware  Recommendation. Decisions@RecSys 2013: 21-28

References [Wang2013] Licai Wang, Xiangwu Meng, Yujie Zhang, and Yancui Shi. 2010. New approaches to mood-based  hybrid collaborative filtering. In Proc. of the Workshop on Context-Aware Movie Recommendation (CAMRa '10). ACM, New York, NY, USA, 28-33. [Shi2003] Shi, Y., Larson, M., and Hanjalic, A. 2010. Mining mood-specific movie similarity with matrix  factorizationfor context-aware recommendation. In Proceedings of the Workshop on Context-Aware Movie Recommendation (CAMRa’ 10). ACM, New York, NY, 34 – 40.  Tkal č i č , M., Burnik, U., & Košir, A. (2010). Using affective parameters in a content-based recommender system for images. User Modeling and User-Adapted Interaction, 20(4), 279 – 311. doi:10.1007/s11257-010-9079-z Vodlan, T., Tkal č i č , M., & Košir, A. (2014). The impact of hesitation, a social signal, on a user’s quality of  experience in multimedia content retrieval. Multimedia Tools and Applications. doi:10.1007/s11042-014- 1933-2  Tkal č i č , M., Košir, A., & Tasi č , J. (2013). The LDOS-PerAff-1 corpus of facial-expression video clips with affective, personality and user-interaction metadata. Journal on Multimodal User Interfaces , 7 (1-2), 143 – 155. doi:10.1007/s12193-012-0107-7 Košir, A., Odi ć , A., Kunaver, M., Tkal č i č , M., & Tasi č , J. F. (2011). Database for contextual personalization.  Elektrotehniški Vestnik , 78 (5), 270 – 274.

SLIDE A SUPPORTO

Concluding Remarks, Take away notes & Challenges  DM = Rational + Emotional  New metrics, evaluation settings and measures that exploit the user’s emotional state to create a ground truth for evaluation purposes  Google Glass: real-time emotion detection  The dark side: Facebook experiment on massive emotion contagion (Kramer et al. 2014)  “personalized” models for the acquisition of affective feedback

98 Experimental Evaluation (in-vitro): Dataset  A subset of HETREC2011-MOVIELENS-2K  Freely downloadable at http://grouplens.org/datasets/hetrec-2011  Dataset of user-movie ratings  855,598 ratings, 2,113 users  10,197 items (movies)  Discrete rating between 0.5 and 5.0 (step 0.5 – 10-point Likert scale)  Sparsity 96.03%  Movies content – plot keywords & summary – crawled from the Internet Movie Database (IMDb)

The Serendipity Problem  Homophily: the tendency to surround ourselves by like-minded people (Zuckerman 2008) E. Zuckerman. Homophily, serendipity, xenophilia. April 25, 2008. www.ethanzuckerman.com/blog/2008/04/25/homophily-serendipity-xenophilia/ opinions taken to extremes cultural impoverishment threat for biodiversity?

Homophily in the digital world in the physical world, one of the strongest sources of homophily is  locality , due to geographic proximity, family ties, and organizational factors (school, work, etc.) in the digital world, physical locality is less important. Other  factors, such as common interests , might play a central role 2 main questions:  Are two users more likely to be friends if they share common interests?  Are two users more likely to share common interests if they are friends? In (Lauw et al. 2010), the answer to both questions is YES (Lauw et al. 2010) Lauw, H.W., Schafer, J.C., Agrawal, R., & A. Ntoulas. Homophily in the Digital World: A LiveJournal Case Study. IEEE Internet Computing 14(2):15-23, March-April 2010.